Camila Morais Canellas
email: camila.canellas@lip6.fr

Thibaut Arribe
email: thibaut.arribe@kelis.fr

Vanda Luengo
email: vanda.luengo@lip6.fr

Towards Learning Analytics Metamodels in a Context of Publishing Chains

Keywords: Learning Analytics, Models, Model-Driven Engineering, Publishing Chains science, statistics, computer science and, psychology

In a context of pedagogical resource production via publishing chains that are based on an model-driven engineering approach, we consider the proposition of a learning analytics implementation. We argue that, by using the same approach to carry out such an implementation versus a classical one, a series of benefits could be assessed, whether they are related to the fact that it is using this specific context, methodological approach or both. Perhaps one of the most particular benefits is the detailed knowledge of the semantics and structure of any document produced, that could therefore be automatically added to the traces/analysis. Other potential improvements discussed are: separation of content and form, interoperability, compliance with data privacy, maintainability, performance, multi format, customization and reproducibility.

INTRODUCTION

The intent of the present work is to propose an implementation of a learning analytics solution into an educational platform design framework that uses a model-driven approach to design and publish pedagogical resources via publishing chains. Although we define more precisely a publishing chain in section 4, we can consider for now that it is a technological solution that facilitates producing and publishing structured documents.

In recent years the growing number of courses and resources offered by learning platforms on the Web attracted lots of participants. Learners interactions with these systems have generated a vast amount of learning-related data. Learning Analytics (LA) focuses on the collection, processing and analysis of such data. The most common definition of learning analytics is: "Learning analytics is the measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs" [START_REF] Siemens | Learning and Academic Analytics[END_REF]. Learning analytics is an interdisciplinary field of study 1 combining

RELATED WORK

Classical approaches of LA solutions consist of analysis of vast amounts of data about learning that were collected through the increasing use of technology [START_REF] Gašević | Piecing the learning analytics puzzle: a consolidated model of a field of research and practice[END_REF]. The underlying assumption is that the interaction data is either already at hand, or will be collected via the addition of the recording of all student interactions with a given system, for example using Experience API (xAPI), IMS Caliper, Learning Context Data Model (LCDM), among others.

This approach implies that the data science methods applied to these data sets in order to calculate and report indicators -of learning processes, learning outcomes, and learning activities, among others -usually happen at a later time, after the decision to collect such data and long after the creation of the actual pedagogical resources.

This approach not only implies taking time trying to understand the traces in relation to what one is trying to analyze, but also that the details of the resources used/traced are usually not available at the traces level. Some attempts have been made to overcome this such as with UTL (Usage Tracking Language) [START_REF] Choquet | Modélisation et construction de traces d'utilisation d'une activité d'apprentissage : une approche langage pour la réingénierie d'un EIAH[END_REF], which aims at describing and operationalizing learning indicators prescribed by final users. It enables adapting to the described elements by modifying the information available, using a model-driven approach. Another approach consists in creating indicators guided by interaction's traces then modeled and leveraged by the Trace Based Management System [START_REF] Djouad | Ingénierie des indicateurs d'activités à partir de traces modélisées pour un Environnement informatique d'apprentissage humain[END_REF].

To the best of our knowledge, no work has been proposed allowing to specifically define, at the same time, both the desired LA indicators and the educational resources production. We argue that, by doing so, we could enable both the document and its usage analysis to be closely linked, thus allowing to perform detailed analysis, among other benefits discussed in detail below. As the works cited in this section, the approach we use is the model-driven engineering one, that has been used in other domains -such as automotive, banking, printing, web applications, among others [START_REF] Hutchinson | Model-driven engineering practices in industry[END_REF] -, yet not as much in LA systems.

METHODOLOGY: CREATING MODELS

When developing complex systems such as publishing chains, model-driven engineering (MDE) is a methodology that allows one to focus on a more abstract level than classical programming [START_REF] Combemale | Ingénierie Dirigée par les Modèles (IDM) -État de l'art[END_REF]. This practice allows to describe both the problem at hand and its solution. In computer science, modeling is widely used in an attempt to control such complexity [START_REF] Jézéquel | Ingénierie dirigée par les modèles -Des concepts à la pratique[END_REF], both to create software or to validate it. In model-driven engineering the focus is to create and leverage domain modelsin our case, the educational domain -as a conceptual model taking into account all the topics related to this specific task/domain. Model-driven engineering is a form of generative engineering [START_REF] Combemale | Ingénierie Dirigée par les Modèles (IDM) -État de l'art[END_REF] in the sense that it follows an approach by which all or part of a computer application is generated from models. To achieve a given objective, some aspects of reality (or a solution to a problem) are simplified. This modeling can also be used to separate the different functional needs and extra-functional concerns (such as reliability, performance, etc.).

A model is therefore a concept or object, often simplified, perceived as representative of another. It can be more or less abstract, more or less precise. In computer science, a model is used to define the design or operation of a set of computer programs. A metamodel, in its turn, is a model which defines the expression language of a model, i.e. the modeling language [START_REF] Jézéquel | Ingénierie dirigée par les modèles -Des concepts à la pratique[END_REF].

A model is an operational resource created by the modeler that designs the system. A metamodel is the expression as well as the validation language of this model. The method consists in generating artefacts (code if we refer to software development, but other areas can also be addressed) via the model validated by the metamodel and a transformation algorithm.

One example of systems based on model-driven engineering are digital publishing chains. Such systems use this approach to create models that will then be used in document production.

PUBLISHING EDUCATIONAL DOCUMENTS VIA MODELS

Pedagogical resources often have a well-known consistency in structure, i.e the parts that compose the document, and in semantics, i.e., what the type of content of each part is. For example, the parts could be an introduction, a few sections on the topic in question -a definition, an example, a practical illustration, etc. -, and a series of exercises/tests aimed at comprehension and memorization. Some systems, such as digital publishing chains, have specialized in assisting the production and publication of such structured documents, especially in their design processes [START_REF] Arribe | Chaînes éditoriales numériques : allier efficacité et variabilité grâce à des primitives documentaires[END_REF]. Crozat proposes the following definition of publishing chains: "A publishing chain is a technological and methodological process consisting in producing a document template, assisting in content creation tasks and automating formatting" (Crozat, 2007, p.2, our translation).

Digital publishing chains allow for documents homogeneity via systems that logically connect editing and publishing large amounts of documents. Naturally, the design of models is strongly linked to the profession in which it fits. The document needs are analyzed and then formalized in a model, including structural diagrams. Moreover, such systems support the separation of content and form [START_REF] Bachimont | Instrumentation numérique des documents : pour une séparation fonds/forme[END_REF], allowing for authors to concentrate on the content itself as the form is automatically applied according to each type of the parts composing the content. In this way, one format is used for the production of content and one (or several) for its publication, which can include PDF, Web, or presentations [START_REF] Bachimont | Instrumentation numérique des documents : pour une séparation fonds/forme[END_REF].

According to [START_REF] Guillaume | Chaînes éditoriales numériques[END_REF], other advantages of using such systems are making teaching practices explicit, sharing of practices, optimizing production management, and reducing costs in document production.

Typically, the document will be based on a model, that is itself based on a metamodel and sometimes a metametamodel can also be in place, depending on the level of abstraction needed. In this context, a document primitive (primitive documentaire) works like a "building brick" as it is the foundation from which the actual document model will be constructed. It is a computer code abstracting the essential principles of documentary objects that later in the chain will allow the generation of specific code instantiating multiple document models [START_REF] Arribe | Chaînes éditoriales numériques : allier efficacité et variabilité grâce à des primitives documentaires[END_REF]. We illustrate this process in the next section, allowing to understand the phases and actors involved when publishing a document using (1) model-driven engineering as the underlying methodology, and (2) publishing chains as the context/tool of editing, publishing and diffusing resources.

Illustration of the Publishing Processes

In our context, developers have defined "building bricks", called document primitives, which will serve as a basis for the creation of the document's models. This corresponds to having a metamodel in an modeldriven engineering approach (cf. example below).

In the next phase, a modeling tool allows the definition of any document model desired. In practice, the so-called modeler, will define a document model using the available document primitives. The challenge of this metamodel is to abstract the technical aspects as much as possible.

Later, an author (a teacher, for example) will use this document model to create his or her document such as a course module. The design tool (s)he uses includes transformation algorithms that automatically publish the document in several formats such as PDF, Open Document, Web, synthetic presentation, etc. As stated before, the interesting aspect here is the separation of the content and its format [START_REF] Bachimont | Instrumentation numérique des documents : pour une séparation fonds/forme[END_REF]. We are particularly interested in the document in its Web format and in the ways in which it will later be used by its final users (learners).

A Practical Example

In order to understand the phases and roles of the stakeholders involved in the process of designing documents for educational purposes via model-driven engineering, we describe here a very simplified practical example. This example uses the vocabulary of the Opale model2 , which is suitable for creating academic courses (on-site training, distance or blended learning). Other open-source models are available to carry out case studies with a gamified twist, create exercisers with different modes of execution (selflearning, evaluation), build question banks for evaluations, etc. First Phase -The Developer At first, a developer defines document primitives such as "text", "multimedia", "quiz", "organization", etc. Each of these bricks (cf. figure 1) is used to build the document model. They can look like a multimedia block or quiz in Moodle for instance. It is from these bricks that the modeler will work, by defining a document model that meets the needs of a given community of users. Besides the elements created, the language used is therefore appropriate for this group/domain, the same applies to the available transformation functions. Second Phase -The Modeler A modeler then uses the available primitives on the design tool to define document models. For example, using a particular primitive (called organization primitive), one can define a very simplified model of a "learning activity" as having one or more:

• "Introduction" (text primitive)

• "Concept" (text or multimedia primitives)

• "Content" made up of parts (text and multimedia primitives) titled "Information" and "Example" • "Conclusion" (multimedia primitive) • "Practice" made up of quizzes (quiz primitive)

As seen in figure 2, the modeler also defines that a learning module can have one or more "learning activities". A "learning activity" must have exactly one "introduction", one "conclusion" and a "practice" part at the end, and include between "introduction" and "conclusion" one or more "concept" and/or "content" parts. A "practical" part must have one or more quizzes.

COURSE MODULE

Learning Activity (n)

Content (n) Introduction (1) Concept (n) Conclusion (1)
Practice (1 -end) If publishing in Web format, the modeler may choose to have a page created for each "learning activity". They may also choose to have a menu reflecting the document structure created and displayed on the left, allowing learners to browse the module, either by the pages names or by jumps to these internal parts of the page, the blocks.

Information (n) Example (1) Concept (n) Concept
The modeler therefore defines the structure (each block and its possible/obligatory components) as well as the basic semantics (what each block is supposed to contain) for each document based on this particular document model, and this information will also be used by the publishing algorithm. In other words, the different possible "parts" of the document are preestablished at the time of modeling and the content type of each part is defined by the chosen blocks. Subsequently, the semantics is known beforehand, when creating each document, depending only on the block types chosen by authors.

Knowing the semantics, for example that a "concept" block was used 15 times in a course, may allow the creation of a table of concepts at the end of the course and to automatically create a sheet-cheat with the list of concepts and to format them differently from the rest. Thus, 1/ the author must choose the model corresponding to the needs of his profession (or type of course, in our case) and 2/ this will also have consequences in the phase of trace analysis, in particular with regard to semantics analysis.

Third Phase -The Author An author, such as a instructor, uses this model to create a course. Its course consists of four "learning activities", each with an "introduction", two "concepts", four "contents", a "conclusion", followed by a "practice" activity to check the understanding of the theoretical content. Once the course has been created, the instructor can publish it in a Web format.

CHEMICAL REACTIONS

Combustion of Methane in Oxygen

Introduction -In this module we... The concept of oxygen combustion is... Fourth Phase -The Learner Finally, learners access the published content, open pages in whatever order they want, scroll through them, take quizzes, etc.

It is worth noting that the framework where the publishing chains just described are implemented also include components for the dissemination and exploitation. These components are themselves conceived using a model-driven approach that simplifies developing new approaches/platforms and maintaining existing ones, makes those platforms reproducible, and allows including an analyst in the platform design without forcing them to go into the core code. To summarize, this preexisting system, based on a model-driven engineering approach, allows the design of resources production, the publication of such resources, to define users, store data and interact with learning management systems via, for example, Learning Tools Interoperability (LTI).

ADDING LEARNING ANALYTICS TO THE CHAIN

In a context where the production of document resources for educational purposes is based on a modeldriven engineering approach, we argue that using this same methodology to implement a learning analytics environment to analyse the use of these documents has several benefits. We started by isolating the concept of a LA indicator. We consider that a learning indicator used in a learning analytics approach is a calculated measure (quantitative or qualitative) linked to a behavior or an activity of one or more learners. The result of the calculation of the indicator is given back to a human (administrator, learner, instructor, etc), in one way or another. It may also be used in the calculation of other (more advanced) indicators. A simple indicator could be the time spent by students in each section of a course, whereas a more complex could be the prediction of dropout at week 3 (which calculation could rely, among other things, on how long students spent on each section of a course).

Subsequently, we used the model-driven engineering approach to create the corresponding learning analytics primitives, by abstracting processes and components of a learning analytics indicator, dedicated to the traces analysis cycle.

This metamodel allows modelers to cover the collection of usage traces of the documents. It has the advantage of allowing to determine, for each indicator, the traces needed and the information it holds. Thus instead of collecting traces first and then trying to find what can be done with them, our approach allows to start by asking the question of what indicator might be useful for each situation/course and only then create the tracing necessary to do so. This is also an asset regarding compliance with legal and ethical norms as only the traces needed for the indicators calculation are collected, and the purpose of such collection is clear to the user. Another great advantage here is that it allows to add to traces the information regarding the semantics/structure of the document, that is well known beforehand (cf. section 3).

Furthermore, the metamodel let modelers define how the calculations will be done and, if it is the case, from whom any input should be entered. For example, the number of concepts visualized before a feedback message is shown could be added by the instructor. It is also possible to determine any preprocessing, post-processing and frequency of (re)calculations where necessary.

Finally, completing the analysis cycle, the metamodel allows to determine how results will be presented. It can range from a raw format (i.e. algorithms results, patterns), the use of visual representations (i.e. in a dashboard and using a certain graph, alongside the menu as a percent bar) or as an automated action (a notification, an alert with a suggestion, etc.); and to whom (i.e. roles that can include administrators, pedagogical assistants, instructors, learners, etc).

Thereafter, we describe the phases and roles of implementing a LA cycle using our metamodel by providing an illustration of the implementation of an indicator. We emphasize how it differs from a classical cycle regarding the succession of the steps, actors involved and, naturally, its proximity to the content(s).

Illustration of the Learning Analytics Processes

For easiness of description we will use a simple indicator -the time spent in each part of a course module -to illustrate the roles and steps of using the proposed metamodel. A more complex indicator -aimed at predicting completion -will also be used as an example allowing to indicate the differences of implementing this type of previous data-dependent indicators.

First Step -Assessing Needs

The first step relates to the identification of the question to be answered (user needs). Usually, a need is expressed by one of the stakeholders. This stakeholder comes from the community using the document model in question: an instructor, learner, educational administrator, etc.

In our example, an instructor wishes to identify the time spent on each type of the different parts of his course, he will use the information to understand the usage of such resources by learners. With this information, he wishes to understand the importance of each type of resource and identify sections that may need a re-design.

Note that this first step does not differentiates significantly from a classical learning analytics approach, besides of the actors involved (when the community discuss the changes to the model).

Second Step -Defining Indicators

The second step consists of defining the indicators that will meet the expressed need. Modeling stems from the aforementioned definition of indicator. Therefore, on the one hand the modeler will have to understand the needs of the stakeholder and, on the other hand, he or she may consult an analyst about the indicator to conceive it holistically: To whom it is addressed? Has it already been proposed in the literature? If it is the case, with what results? How should it be shown to the users who will consult it (dashboard, type of graph, warning message, etc.)?

In our example, we assume that the instructor proposes the indicator to the modeler that confirms it is possible to implement it.

This step differs from a classical approach. Usually, the e-learning interaction traces are already collected and there is a need to select the relevant ones. Often this is done by the learning environment expert, that has in-depth knowledge of the traces collected and their structure and the analyst will need to work closely with him.

Third Step -Modeling Indicators

Once all the information has been gathered, the modeler proceeds with the modeling of the indicator. This modeling concerns three main parts: traces, indicator and visualization(s).

First, the modeler defines the traces to be generated to compute the indicator. He or she decides among other things the type of triggering event of a log. In our example, the event corresponds to each block being shown/not shown on the screen (focus in and focus out). The modeler also indicates the content of the trace, i.e. the recorded information, -in our example, the ID of the block shown and its type/name, among others. (S)He can also define which parts of the document are concerned: all the pages, all the pages and each sub-part (blocks) of the pages (our case), just the blocks which are "concept" and "introduction", any attempt to answer a quiz, etc. The modeler also defines how these traces will be periodically converted into ready-to-use tables when calculating the indicator.

Then, the modeler proceeds to model the indicator itself. He or she defines the inputs necessary. It can range from a type of trace, as just described, to an input of a start/end date coming from the instructor, or another indicator used as input, among others. The calculations to be performed are coded (application of operators) and the type of output indicated. In our example, a JSON matrix containing the total time spent grouped by each type of the parts of the document. Any preprocessing and post-processing can be done either in this code or in the previous step by directly registering information within the stored data, which can be particularly useful if it is used by multiple indicators. The modeler also defines the calculation schedule for each indicator, with the possibility to identify any incremental processing. The idea is that this way more efficient calculations can be done, avoiding to recalculate everything each time an indicator is shown. This logic is not always in place as usually the interest relies on the result of an analysis, whereas here the modeler can better care about these optimization questions and this impacts all users of the indicator thereafter.

Finally, the modeler defines the transformer used to produce the visualization through a widget. One has to define (1) how the result will be returned to the users (i.e. a bar chart), ( 2) where it will be included (i.e. a dashboard), and (3) for whom (i.e. instructors and learners).

Regarding the dropout prediction indicator: as it is a case of previous data-dependent analysis, the exploratory analysis is recommended to be made using data from a previous version of the course to carry out feature engineering and run different algorithms in order to identify the best performing one related to the data at hand. Once this analysis has been done (outside of the current system) and a data model has been identified (for example, a decision tree), it is this data model that will be embedded into the metamodel of the indicator.

This step is certainly the one that differs the most from a classical approach of a LA implementation, as an indicator is usually defined a posteriori.

Fourth Step -Evaluating Indicators

The fourth step consists of making sure results are beneficial to the stakeholders. As for any educational intervention there are risks associated such as incorrect or imprecise interpretation of the information provided, for instance a bad interpretation of a graph or an unexpected effect, such as motivation decrease when comparing to others.

Finally, the expected improvement linked to the need expressed at the start of the project can be assessed, formally or informally, using any research method allowing to assess the implementation effects, as in a classical approach. The advantage here is that, since the document model is used in different contexts, an analysis of the results can be more consistent, for example, comparing access to "concepts" in documents from courses across different knowledge areas or universities.

PROOF OF CONCEPT

Reification

As indicated in section 5.1, when compared with a classical learning analytics implementation, the main difference is in step 3, where the actual model of indicator is defined. Another difference is in the implication of different actors/roles. As the modeler is already an expert on the document model(s), he or she can define the traces needed, that relate specifically to each model structure and semantics, and by doing so, there is no need to rely on traces generically collected that frequently require the intervention of an environment expert.

We implemented the components of the proposed metamodel in the same tool used to create document models, as seen in figures 5 and 6. Note that the transformations needed for it to be functional are still under consideration. The components of the metamodel are currently under evaluation by modelers in order to assess if the components are sufficient to model a variety of indicators in different contexts and if any improvements can be made at this time.

Potential Benefits

As stated before, the differences on the approach used possibly allow for an improvement in productivity, specially regarding scalability. In this section, we describe different measures and cases that could potentially be improved by using an model-driven engineering approach to implement a LA cycle in a publishing chain context. Naturally, some of the advantages come from the approach used, some from the context and, some from the strong integration of both.

Detailed Analysis of a Document

A first aspect to be considered is that, by using modeldriven engineering to produce both the document itself and the indicator, allows to create detailed indicators (or add details to existing one) taking into account (in the traces themselves) the document semantics and/or structure. This fact allows, for instance, to easily create the indicator given as an example in section 5.1, i.e. time spent in each type of content parts (our blocks, i.e. "introduction", "concept", "example", etc.), as the information (type of block used) can be added to the traces to be collected with a few clicks, linking the document primitive to the learning analytics primitive. Whereas in a classical approach one would need to tag a posteriori each part (having the premise that the document is structured in a consistent way) that would have to be done almost manually, linking each part identifier to a type of content. And any modification of the content of the course would either require this tagging to be redone, or it would be the instructor's responsibility to make sure the tagging remain valid -an error-prone extra task that they are not an expert at.

The easiness of the implementation of this level of detailed analysis, integrated to the system, can certainly be seen as an advantage when considering indicators meant at re-designing courses. Moreover, using this detailed information as inputs when carrying out feature engineering for more complex indicators such as prediction, sequence analysis, among others, is a promising path we are willing to address in the future.

Separation of Content and Form

Habitually, indicators are implemented using one specific chart as the desired result, often available for consultation in a dashboard. Having different learning analytics primitives for calculation and visualization of an indicator has the advantage of separation of content and form, in the same way as documents created using publishing chains (see section 4. As a result, we can imagine situations where the same calculation result can be used in a dashboard, intended at the instructor, and side by side with the course menu for learners (figure 4. For our example, the time spent in each part: the former will go directly to a dashboard to see which parts are the most time consuming for learners, while the later can see the time spent without having to stop consulting the content and going to check a dashboard, maintaining the flow of a studying session.

Interoperability

The interoperability of a LA system ensures the compatibility of any type of virtual learning environment by allowing the integration of different data sources [START_REF] Dyckhoff | Design and Implementation of a Learning Analytics Toolkit for Teachers[END_REF]. More broadly, according to systems engineering, it is a characteristic of a system allowing it to work with other products or systems, currently or in the future, whether in its implementation or in access.

As stated above, traces produced using the proposed approach have the potential of being enriched with document details. However, if needed by certain situations -this may be needed when traces will be analyzed by an external tool -, it is also possible to create such traces using any given standard (for example, xAPI). Doing so may increase the interoperability of the system, but decreases the advantage of working with enriched traces.

Compliance with Data Privacy

Compliance with various data privacy laws is certainly an increasingly important point regarding the innate requirements of a personal data analysis process. The data collected as well as access to it and its processing thereof must be carefully documented.

In order to justify the tracing, for each piece of data, the documentation must determine and justify why this tracing is done, who has access to it, the resulting uses and treatments, etc. In addition, explicit consent from users is required.

This point has been briefly mentioned before, as a key difference in the approach discussed here is the fact that traces are collected because they are going to be used when calculating a certain indicator that was modeled as required by a stakeholder. This is a twist from the approach where all actions performed in the learning environment are collected and only later stakeholders will decide how they could use it.

As traces are connected to indicators, it will likely be easier to create a documentation that conforms to GDPR3 principles such as communicating specific purposes for data collected and actors involved in the data processing. The key idea of only tracing what is necessary to answer to predefined questions is also an inherently very GDPR-compliant approach.

Maintainability

Another fundamental benefit in this approach is the maintainability of the proposed solution. Maintainability measures the effort required to correct or transform the software, but also its extensibility, that is to say the little effort required to add new functions. This is potentially where we can find the greatest advantage of using model-driven engineering in order to create the learning analytics implementation.

Given our context where many document models and contexts of use will rely on the same solution, being able to easily choose and adapt indicators is essential. Thereby, a modeler can create an indicator for a specific document model and, the fact that it will potentially be used by a number of users in different contexts, i.e. any document produced using the document model, from a single source (the modeling) is a great improvement in maintainability. Moreover, on the long term, any changes or corrections needed are to be made at one single code/modelling, reducing development time, resources and sources of potential errors.

Performance

The ratio between the quantity of resources required (means including personnel, time, material, etc.) and results delivered constitutes performance.

Regarding the proposed approach, we can speak of performance with regard to the creation of an indicator used in many contexts (any using the same document model), of a series of indicators -in a dashboard, for example -or even of a kind of industrialization of analysis processes, which would allow the creation of different dashboards adapted to the specific needs of a series of document models, with a potential saving of time to do this task.

Multi Format

As we saw, in our context a single pedagogical resource can be automatically transformed into several formats as PDF, Open Document, Web, synthetic presentation, etc. In a situation where a learner will print one or two chapters of a course in order to study during his or her vacations, we can imagine that, on the resulting PDF an indicator -say the time he or her has already spent on each part of each chapter -will be printed alongside the content. Needless to say, there will be no trace recorded of his or her studying session from the resulting PDF, but any data already available at the time of the transformation of the resource at hand could be appended to it.

Customization

The possibility of customizing an indicator or a dashboard has been studied by several authors [START_REF] Dabbebi | Towards Adaptive Dashboards for Learning Analytic: An Approach for Conceptual Design and Implementation[END_REF]. This may involve offering filtering alternatives or different viewing modes, the choice of which is made by users according to their preferences and needs felt during their practices. The fact that each document model can have its specific indicators related to its actual usages can also be understood as a form of customization.

Providing the possibility of customization is certainly an asset, nevertheless requires ensuring that users understand the choices available to them, as well as special attention not to overwhelm the user with a number of choices -and therefore decisions -too important to make, which could consequently inhibit the use of the system.

Using the proposed approach, it is possible to customize an indicator in two ways: (1) by proposing more than one option, such as a visualization inside a dashboard or alongside the menu, allowing users to choose the one they prefer; and (2) by remodeling an indicator, for instance, starting with the model of an indicator analysing the time spent in each sub-part (blocks within a page) of the document and changing it to only show the results of the main parts (each page) can be done with a few clicks.

Reproducibility

Another important aspect regarding LA implementations is linked to the need to allow the reproducibility of any indicators already in place. According to [START_REF] Lebis | Capitalisation of Analysis Processes : Enabling Reproducibility , Openess and Adaptability thanks to Narration[END_REF], reproducibility is made up of three properties, namely:

Replication: the ability to reproduce an identical analysis, without necessarily taking into account the variables linked to the context. More precisely, in an analysis process, replication is considered as the ordered succession of operators of such process. This is because the operators involved in this process are clearly identified and their order of application is also well defined [START_REF] Lebis | Capitalisation of Analysis Processes : Enabling Reproducibility , Openess and Adaptability thanks to Narration[END_REF]. An indicator that has being modeled in one document model can easily be used (with or without adaptations) in another document model as the operators needed and their order of application are known, only changes to the entry traces (as model's parts are not the same) should be adapted, as well as visualization modes, if necessary/desired from the new context.

Repeatability: the property of an analysis process to be carried out several times, on the same set of data and with the same configurations, having the same output results. In other words, this quality makes it possible to trace the results produced and their consistency [START_REF] Lebis | Capitalisation of Analysis Processes : Enabling Reproducibility , Openess and Adaptability thanks to Narration[END_REF]. Checking the repeatability quality of implemented indicators should be facilitated as the same implementation would be available/tested in various practical contexts.

Reuse: the ability of software (or an analysis process) to be reused in an application other than that for which it was designed or in another context. The same process must then be easily reused on another data set which is more or less similar to the initial one. Thus, possible adaptations can be made in order to adapt the analysis to a similar context, for example, to another document model, but with particular attention to guarantee that the foundation of the analysis process (a learning, theory from which the indicator was envisaged, for example) are respected and remain consistent.

Lastly, by using our approach, a large variety of indicators can be modeled, but it is also possible to envisage the proposition of a given set of an indicators ready to use (as a single "primitive"/component). Specifically, an indicator, or series of indicators seen as most useful/standards, can be modeled with the primitives we propose and, at a later time, be "condensed" as a single primitive to be added to modeling processes, facilitating even more their implementation into other document models.

DISCUSSION

In this work we analyse the potential benefits of implementing learning analytics solutions using a model-driven approach in conjunction with digital publishing chains that are also based on the same approach. The metamodel proposed aims at being sufficiently abstract in order to allow the implementation of the vast majority of learning analytics indicators, with the advantage of counting on the prior knowledge of documents semantics and structure.

Future work will be conducted aiming at modeling a variety of indicators and measuring in more detail the benefits discussed here.

Figure 1 :

 1 Figure 1: First Phase: Metamodel with the document primitives for our illustrative document modeling.

Figure 2 :

 2 Figure 2: Second Phase: Metamodel used to define the model for our illustrative document modeling.

  Figure 3: Third Phase: Document model used in the instantiating process of our illustrative document modeling.

Figure 4 :

 4 Figure 4: Fourth Phase: Document transformed into a Web format, ready to be used by learners. We added as an example the visualization of an indicator following the menu (structure) of the course.

Figure 5 :

 5 Figure 5: Traces generated with the information of the document included.

Figure 6 :

 6 Figure 6: The learning analytics modeling of an indicator.

Available at: https://doc.scenari.software/Opale/en/

https://gdpr.eu/


Thomas Sergent
email: thomas.sergent@lip6.fr

Morgane Daniel
email: morgane@lalilo.com

François Bouchet
email: francois.bouchet@lip6.fr

Thibault Carron
email: thibault.carron@lip6.fr

Predicting Young Students' Self-Evaluation Deficits Through Their Activity Traces

Keywords: Self-Regulated Learning, Primary school, Self-evaluation, Prediction, Remediation, Adaptive polling

Self-evaluation is a key self-regulatory process that can already be mastered by young children. In order to assess self-evaluation skills of children, we introduced a random prompt asked randomly after 1 out of 15 exercises into a literacy web-application for primary school student, in order to evaluate the perceived difficulty [Too easy, Good, Too difficult] of the exercise they just solved. Comparing students' actual performance with their responses to this prompt can provide information about their ability to self-evaluate, and thus detect students who could improve their self-evaluation skills. We collected more than 1,000,000 responses from 300,000 students and used these data as well as performance data on each question of each exercise to predict a student's response to the next prompt, thereby estimating how likely they are to having a self-evaluation deficit. The results show (a) that a student's past responses to self-evaluation statements impacts the quality of future predictions (b) that the impact of past responses -vs their current performance -is greater when the student has low capacity for self-evaluation (c) that including older student data (answers from several sessions ago) helps in improving the accuracy of the prediction. These results pave the way (1) for adaptive polling by identifying when the model is unreliable, giving them the statement then instead of randomly, (2) for adaptive feedback, by knowing the students the most likely to show a deficit, to provide remediation.

INTRODUCTION

Improving children's self-regulated learning (SRL) skills is a key component of their academic performance, as selfregulated students generally know better "how to learn", which can have a positive impact in all disciplines [START_REF] Zimmerman | Investigating Self-Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects[END_REF]. A key SRL process is self-evaluation [START_REF] Schunk | Self-Regulation and Learning[END_REF], which is a skill already developed in children as young as 5 years old [START_REF] Stipek | Self-evaluation in young children[END_REF]. It is therefore a particularly interesting SRL aspect to target when working with young children. The most reliable way to assess SRL deficits is through direct questions to the students [START_REF] Barnard | Measuring self-regulation in online and blended learning environments[END_REF], but constant prompting can lead to an overall degraded perception of the learning environment [START_REF] Bouchet | Evaluating Adaptive Pedagogical Agents' Prompting Strategies Effect on Students' Emotions[END_REF] and it is therefore critical to limit prompting to the minimum. Hence we are interested here in trying to predict students' answers to assessment on perceived difficulty which are used to assess students' tendencies to overevaluate or underevaluate. The second aspect we investigate is relative to the features that are the most relevant for this prediction.

RELATED WORK

The EDM community has recognized early on the interest of studying SRL through data mining [START_REF] Winne | The Potentials of Educ. Data Mining for Researching Metacognition, Motivation and Self-Regulated Learning[END_REF], and previous works have been more particularly interested in detecting SRL behaviors from traces [START_REF] Diana | Extracting Measures of Active Learning and Student Self-Regulated Learning Strategies from MOOC Data[END_REF], discussion forums [START_REF] Huang | I'm Sure! Automatic Detection of Metacognition in Online Course Discussion Forums[END_REF][START_REF] Harrak | Automatic Identification of Questions in MOOC Forums and Association with Self-Regulated Learning[END_REF] or proxy behaviors such as gaming the system [START_REF] Dang | Exploring the Link between Motivations and Gaming[END_REF], explaining such behaviors with sequence mining [START_REF] Taub | Using Sequence Mining to Analyze Metacognitive Monitoring and Scientific Inquiry based on Levels of Efficiency and Emotions during Game-Based Learning[END_REF][START_REF] Bouchet | Impact of Different Pedagogical Agents' Adaptive Self-Regulated Prompting Strategies on Learning with MetaTutor[END_REF], mixture models (for procrastination, a proxy of SRL) [START_REF] Park | Understanding Student Procrastination via Mixture Models[END_REF] or coherence analysis [START_REF] Segedy | Using Coherence Analysis to Characterize Self-Regulated Learning Behaviours in Open-Ended Learning Environments[END_REF]. Other works have focused on analyzing the differences in use of SRL strategies [START_REF] Farhana | Investigating Relations between Self-Regulated Reading Behaviors and Science Question Difficulty[END_REF], providing feedback to encourage them [START_REF] Feild | Improving Student Performance Using Nudge Analytics[END_REF] or predicting their use [START_REF] Sabourin | Understanding and Predicting Student Self-Regulated Learning Strategies in Game-Based Learning Environments[END_REF]. However, as far as the authors are aware, no previous work has specifically attempted to predict how a student would answer to a question aiming to measure a SRL deficit (self-evaluation or any other), and none of the aforementioned work focused on young children (5-7 years old). It is worth noting that although young children's abilities to use SRL strategies may be more limited than in teenagers, they seem to have comparable monitoring skills [START_REF] Schneider | The Development of Metacognitive Knowledge in Children and Adolescents: Major Trends and Implications for Education[END_REF]. Indeed, recent work on a dashboard supporting SRL in a mathematics software program for 9-10 years old (only slightly older than our targeted students) showed a significant improvement in SRL skills for students in the dashboard group compared to those without the dashboard [START_REF] Molenaar | Personalized visualizations to promote young learners' SRL: the learning path app[END_REF].

SELF-EVALUATION ASSESSMENT 3.1 Context

Lalilo is one of the many web applications used by teachers in the classroom to help them implement a differentiated pedagogy. At the beginning of 2021, it is used by 40,000 English and French speaking kindergarten and elementary classes every week to strengthen literacy through series of exercises adapted to the students' level, while providing the teacher with a dashboard to evaluate the students' activities and progress. It is therefore a relevant testing ground for evaluating and then trying to correct some students' SRL deficits. A typical session lasts 20 minutes (on average) with the student performing around 15 short exercises with 3 to 7 questions each. Student activities (e.g. logging in, time spent on an question/exercise, mistakes) are traced and we will focus on students' answers to an exercise, thus we will call trace only the answers to this set of questions of the same type within an exercise.

Data collection

To assess some aspects of students' SRL skills, we introduced a random prompt is once every fifteen exercises when a student finishes an exercise (i.e. on average once per typical learning session). This prompt includes a perceived difficulty statement asking the student "How difficult was this exercise for you?" with 3 possible answers: "Too hard", "Just-right", "Too easy"). Comparing the answer to the perceived difficulty statement with the real performance aims at measuring the self-evaluation ability of the students, i.e. their ability to correctly estimate the difficulty of the questions they just answered. Before introducing the assessments, we checked qualitatively in a classroom using Lalilo that statements were understood by 1 st grade students (details not presented here). We collected traces from Kindergarten, 1 st grade and 2 nd grade classes based in France, Canada and USA learning in French (FR) or English (EN) between January 18 and February 24, 2021 on the Lalilo platform. We kept only the traces for which students had answered to a prompt and further on we call trace the answers to the exercise with the associated answers to the prompt.

METHODS 4.1 Dataset

Given the history of a student answers to the perceived difficulty prompt and their performance on the current exercise, we want to predict which answer is the most likely to be given by the student to the next prompt (and thus extrapolate their self-evaluation skills). If the student's performancewhich will be defined in the Feature engineering subsection -was "excellent" (resp. "poor") and they answered "Too hard" (resp "Too easy"), they were considered as having an underevaluation (resp. overevaluation) deficit. We filtered out students who had strictly less than 8 traces with answers to prompts so that our model would not overfit on results of students with very few answers, as students with very few answers were overrepresented in our initial dataset. We finally had 424,173 traces with an answer to the perceived difficulty statement from 34,083 students having on average 12 traces with self-evaluation answers (SD = 5.93).

Feature engineering

We engineered several new features that could have a predicting power in our results. Basic performance feature. In addition to the trace and student IDs, used for filtering but not for prediction, we extracted for each trace the answer correctness list, a boolean vector of a length of 3 to 7 (number of questions per exercise). Enriched performance features. From the answer correctness list, we extracted 5 additional features: the good answer count (i.e. the number of 1s in the vector -the higher the value, the better the student may feel they have succeeded), the total answer count (i.e. the length of the vector), the success rate (i.e. the ratio between the good answer count and the total answer count) and the second half success rate (i.e. the success rate on the last half of a trace -a student with self-evaluation deficit may suffer from a recency bias, influencing positively [resp. negatively] their perception of their performance when answering correctly [resp. incorrectly] to the last questions of the exercise). Exercise features. We hypothesize that self-evaluation deficits are not uniform across one's knowledge. In particular, a student's deficit may be stronger in some types of exercises or on exercises about a given topic. To assess this impact, we added 5 features that relate to the exercise finished just before the two difficulty statements: exercise template (for example a multiple choice question or a word composition exercise), lesson index (there are around 1,000 lessons in Lalilo -although they are not entirely linear, the higher the value of this feature, the more advanced the content is; when working on English (resp. French) data, the lesson index FR (resp. EN) is empty), lesson type (lessons are organized in a tree structure -lesson type represents the first level category), lesson subject (lesson subject represents the second level category in the tree), language (English or French). Previous feedback modalities. In order to help students' in their performance assessment, they are randomly given an audio feedback (such as "In the last exercise, you found 3 correct answers of 5 questions") and/or a visual gauge of as many green ticks and red crosses as they had good and bad answers in the previous exercise. Even when these synthesis exercise-level performance indicators are not there, the students always have an immediate question-level true/false feedback. We have previously shown the positive impact of these two indicators in correcting some self-regulation issues, and we therefore hypothesize that they need to be taken into account when predicting how the student will assess the difficulty. Hence we added two binary features, gauge and audio which indicate whether these performance feedback were given before the two difficulty statements. A feedback is also provided after answering to the prompt, when students display a self-evaluation deficit (as defined in 3.2), encouraging them to be more confident or warning them to be more careful; we therefore encode this as a third binary feature, remediation, indicating whether the student received a feedback the last time the difficulty was assessed. Self-evaluation deficit lag features. Self-evaluation deficits are expected to be a recurring phenomena in students' answers, i.e. a student who has under/overevaluated themselves a few times is likely to under/overevaluate themselves again in the future. Hence we added 3 lag features for the last 3 perceived difficulty assessments. Moreover, since it is possible that the last 3 assessments were not allowing the student to exhibit a deficit (e.g. a student cannot appear to be overevaluating if their performance is at 100% on the last 3 exercises where they were asked to assess the difficulty), we also added 3 lag features for the last 3 perceived difficulty assessments where the student's performance was equal to the performance on the current exercise. Performance is a categorical value which is worth "poor" if the success rate is below 34%, "excellent" if the success rate is at 100% and "medium" otherwise. Overall previous self-evaluation deficit. If students are stable over time in their assessment, we expect that taking into account the whole history would have a positive impact on the prediction. We therefore introduced 5 additional features: self-evaluation answer rank (the number of times the student has been asked a self-assessment), number of "Too easy" (resp. "Too hard") answers (the number of times the student previously answered "Too easy" (resp. "Too hard") to previous assessments), and "Too easy (resp. Too hard) answers ratio" (the ratio between the two previous features). Additionally, similarly to what was done for the lag features, we also considered the 5 equivalent features exclusively on previous assessment given after an exercise with a similar performance level.

Algorithms

We used Catboost [START_REF] Prokhorenkova | CatBoost: unbiased boosting with categorical features[END_REF] and LightGBM [START_REF] Ke | LightGBM: A Highly Efficient Gradient Boosting Decision Tree[END_REF] to perform the predictions, two gradient boosting algorithms based on Decision Trees whose main assets are: (a) their ability to natively deal with categorical features and (b) their explainability, allowing to study feature importance in each prediction with SHapley Additive exPlanations (also called SHAP values) [START_REF] Lundberg | A Unified Approach to Interpreting Model Predictions[END_REF]. They also have recently won Kaggle competitions on a variety of datasets. We used MultiClass as a loss function, set the number of iterations at 200 and kept the other hyperparameters at their default values. As their results were very similar for the global prediction task (cf. Table 1), we used CatBoost model only for the other tasks.

Analyzing features importance

We first measured the improvements allowed by feature engineering to the global prediction scores using 5-fold cross validation and stratifying by student so that no student traces are both in training and testing folds. For all feature importance measures subsequently described, we created a training and a testing set -also stratified by student -and measured the feature importance in the testing set. We studied the importance of various features in our model using the SHAP package [START_REF] Lundberg | A Unified Approach to Interpreting Model Predictions[END_REF]. We also compared the importance of features across the three classes so as to highlight the features that have the most impact on each class specifically. If students showed a given deficit regularly -we defined a threshold at 50% of "underevaluation" (resp. "overevaluation") over the traces with 100% (resp. below 34%) success rate -we tagged the student as having an "underevaluation (resp. overevaluation) deficit". We then trained our algorithms on a dataset with students tagged as having a deficit and on a dataset with students tagged as having no deficit. Our hypothesis was that feature importance would vary between these two models: the predictions were expected to depend more on past answers than current performance for students tagged with deficits compared to the other students. Finally, we measured the evolution of the performance of the model depending on the self-evaluation answer rank that is predicted. To do so, we analyze the evolution of Cohen's Kappa coefficient, measuring the quality of our prediction of the perceived difficulty, on a cohort of students of the testing set having answered a given amount of prompts. We computed this coefficient for each self-evaluation answer rank and expected the quality of the prediction to improve as students would be better and better characterized by their features.

RESULTS AND DISCUSSION

Global features importance analysis

Firstly, both Catboost and LightGBM allow to predict with a reasonable overall performance the students' perceived difficulty (cf. right part of Table 1). Secondly we see that all features additions improve the model except the previous feedback ones. Specifically, adding features describing what a student did in the past improves the predictions significantly.

Table 2 ranks the top 10 features with their mean absolute SHAP values. Interestingly, the success rate of the student and the success rate on the second half of the correctness list are only the 5 th and 8 th ranked features. It means that past information about the previous answers of students to self-evaluation prompts influences more the prediction than their current performance, although they are being asked "How difficult was this exercise for you?". Specifically, the last three answers to the perceived difficulty question bear a significant weight in our model's predictions, as well as the global "Too easy" and "Too hard" ratios. As expected, the "Too easy" ratio has a huge importance for the "Too easy" class as has the "Too hard" ratio for the "Too hard" class and both ratio are highly ranked for the "Just-right" class. Indeed, we did not input the "Just-right" ratio as the model can learn it from the combination of "Too hard" and "Too easy" ratio. We can note that the success rate feature is mainly important for the "Too easy" and "Too hard" class, which is logical as an excellent (resp. poor) performance is not likely to lead to a "Too hard" (resp. "Too easy") answer. We can also see on Figure 1, as the "Too hard" ratio is equal to 0, it drives the prediction score of the "Too hard" class downwards while it drives the prediction score of the "Just-right" class upwards. Furthermore, the 3 last answers to the perceived difficulty statement of this student were "Just-right", "Too easy", "Too easy" and their success rate on this trace was 100%; the predictions of the "Too easy" class are therefore also driven upward by these features.

Features rank from self-evaluation deficits

Figure 2 shows the feature importance rankings for students detected as having or not self-evaluation deficits. Students with deficits consistently choose how to answer to the prompt more based on past answers (in particular the "Too easy/hard" ratios), as opposed to students with no deficits who rely more on the success rate to this exercise, as one should. These results are in line with our hypotheses.

Predicting power based on answer rank

Figure 3 shows the kappa evolution depending on the number of past self-evaluation assessments. The kappa for the first answer is around 0.13, then quickly climbs around 0.4 for the next four traces; and finally slowly increases until plateauing around 0.6. The kappa of 0.13 for the first answer is consistent with Table 1: at the beginning, Student features are empty and the model can only rely on Trace features. With Trace features only, the model reached a Kappa of 0.1084 which coincides with the kappa value of 0.13 in Figure 3. We can then deduce that the model is more and more able to predict answers to the perceived difficulty statement.

Table 1: Prediction metrics after 200 iterations of the Catboost and LightGBM algorithms depending on the features used. We measure mean and standard deviation (in parenthesis) on 5-fold cross validation.

CONCLUSION AND FUTURE WORKS

Using a large volume of trace data from primary school students, we leveraged students' past data to significantly improve the prediction of the answers to future self-evaluation prompts. The results also indicate that the more data we have about a student, the better our predictions are. Using feature engineering, we ranked features by the additional predicting power they provide, and found results consistent with SRL theories (in particular that prediction of answers for well-regulated students depends mostly on their success rate). This paves the way for adaptive polling (as opposed to the current random one), prompting only students likely to display a self-evaluation deficit, allowing us to better target remediation. The main limit of the current work is the specificity of the context: it would be particularly interesting to study the main features used in another context with a different type of students. We are also targeting one of many existing SRL deficits, and expanding research on predicting other deficits to encourage the training of multiple SRL skills seems important as well. Future works also include further feature engineering to refine what features may have more impact than the current ones.

Figure 1 :

 1 Figure 1: Feature impact in the prediction of each class for a randomly chosen trace in the testing pool. Top: "Too easy" class, middle: "Too hard" class, bottom: "Just-right" class.

Figure 2 :Figure 3 :

 23 Figure 2: Top 10 features impact for class prediction of students detected as having (top) or not (bottom) a selfevaluation deficit

Table 2 :

 2 Average feature importance rank by class, sorted by total SHAP value (top 5 in bold)

	Features	Too easy Just-right Too hard
	"Too easy" ratio	1	1	14
	1 to last perc. answer	2	2	3
	"Too hard" ratio	6	4	1
	2 to last perc. answer	3	3	9
	success rate	4	9	2
	3 to last perc. answer	5	5	10
	1 to last perc. answer P	7	6	12
	second half success rate	8	19	4
	3 to last perc. answer P	9	7	11
	exercise type	10	15	5


G Boschetti
email: giorgio.boschetti@gmail.com

A Mariscotti
email: andrea.mariscotti@unige.it

Integrated Electromechanical Simulation of Traction Systems: Relevant Factors for the Analysis and Estimation of Energy Efficiency

Keywords: Davis equation, Electromechanical simulation, Guideway transportation systems, Model validation

Electromechanical analysis and simulation of traction systems are required to estimate the power consumption, to size the power system and to optimize system parameters for energy saving. The number of variables and parameters (mechanical and electrical) is huge and they are deemed by various degrees of uncertainty. An integrated simulation approach is presented, where some of the major sources of uncertainty are addressed and discussed.

I. INTRODUCTION

Electromechanical simulation of electric transportation systems has several applications: during design, the sizing and rating of electric substations, of traction supply conductors and supply feeders and the optimization of the feeding points, with respect to the maximum absorbed power and the tolerated supply voltage drops along the line; moreover, under major system revamping and modernization, the calculations may be repeated to push optimization even further, reducing power absorption peaks, resizing some feeders and conductors, adjusting the time table and train scheduling. In general, also with the aim of optimizing the power consumption and demonstrating overall system efficiency, electromechanical simulation is a precious tool to support direct measurements.

Electromechanical simulation is a known discipline, in that it results from the combination of mechanical equations of movement of the trains and the electrical network equations, that rule the way the electric power is delivered to the trains at their respective current collection points (called "pantograph" in the following and including also third rail shoe current collection). The first part is normally identified as the mechanical subsystem and the second one is the electrical subsystem.

Electromechanical simulations are normally used during the design of the traction supply network, supporting the sizing of the traction power stations (or substations), the optimization of their separation, the verification of the attainable speed and the correct exploitation of the train characteristics (by monitoring the line voltage). The needed accuracy is in this case around some or several %, largely covered by the tolerances and the design margins.

On the other hand, when the simulation is used for the evaluation of the overall system energy efficiency, the accuracy required is in the range of a fraction of %. When this figure is applied to the yearly power consumption of a modern electric transportation system, it represents already a tangible difference in terms of money. This translates into a demanding requisite concerning the overall simulator accuracy. Several are the related aspects, but here the focus is on the representation of the two subsystems and the data exchange between the two, in order to ensure an adequate and comprehensive representation of the phenomena.

For the mechanical subsystem both the completeness of the equations and the determination of the parameters and equations coefficients have impact on the accuracy of the estimation of the dragging resistance, or resistance force, R. Through the electromechanical traction chain, it is translated into the absorbed electrical power P el . It is underlined then that it is simplistic to assume that the electrical power is delivered to the train only through the fundamental components of the pantograph variables, voltage V p and current I p . With a fraction of % of accuracy as the overall target, the representation must be as sophisticated as possible, in particular where the information on components and parameters is rather complete.

The integrated simulator developed in the last year, and that is used here as a reference for discussion, features thus an electrical subsystem derived from the complete network representation using distributed transmission line equations, and it is believed to be a leap ahead with respect to other solutions based on lumped component simplified network representations. The mechanical subsystem implements the full Armstrong-Swift equations [START_REF] Armstrong | Lower energy technology. Part A, identification of energy use in multiple units[END_REF], where the coefficients may be derived from available databases. On the contrary the efficiency of the traction chain that translates mechanical power into requested electrical power at pantograph cannot be more accurate than the information supplied by the vehicle or train manufacturer. The interface at the train pantograph is responsible for the iterative convergence of V p and I p at each operating point during the simulation.

II. MECHANICAL SUBSYSTEM

An electrical transit system in general consists of a combination of a number of trains (or streetcars, or track-free trolley buses), each moving as an independent unit within the general constraints of the operating schedules. All these vehicles are electrically interconnected through the power distribution system (third rail or overhead conductors) fed at specific locations by electrical substations, that in turn draw power from the local utility.

Before the exploitation of regenerative energy and other forms of stored energy, transit vehicles took the entire power for mechanical motion (kinetic energy and drag resistance) as well as on-board auxiliaries and other loads from substations. The introduction of regenerative traction, however, changed the relatively simple scheme of an electrical transit system just considered due to:

• introduction of additional mobile temporary substations, that represents vehicles or trains in regenerative braking; • need for establishing a maximum line voltage limit. When considering regenerative braking and the distribution of, the active elements of the power distribution system, the latter are characterized by a variable location (vehicle mobility) and duration of regeneration. In addition, the limitation of the line voltage V max introduces a major non-linearity: regenerative power is controlled either by switching to an alternative local braking system (dissipative braking) or reducing the amount of regenerative power fed back to the system.

The electrical subsystem solves the electrical network of the traction system at each iteration with movable and fixed lumped sources and loads. The mechanical subsystem implements the modules that calculate time-variable loading (train power draw) and time-variable geometry (train motion). The first point is covered by a step-by-step calculation of the resistance to motion of the train; the latter point is addressed implementing a set of rules that cover vehicle operating conditions and scheduling, timetabling and traffic constraints. In the following both aspects are considered, as well as the main parameters for the calculation of the absorbed power profile, depending on kinetic factors and relative vehicles position and motion.

A. Davis and Armstong-Swift equations

The train resistance force R is approximated by a quadratic function that is variously known as the "von Borries Formel", the "Leitzmann Formel", the "fonction de Barbier" and, in the Anglo-Saxon world, the "Davis equation" [START_REF] Rochard | A review of methods to measure and calculate train resistances[END_REF]:

2 Cv Bv A R + + = (1)
The coefficients A and B include the mechanical resistances and depend on the train mass, so that at lower speed (≤ 30 m/s, that is about 100 km/h) the resistance force R is mainly dependent on the train mass. At higher speed, the Cv 2 term related to the aerodynamic resistance becomes dominant. The values of the coefficients in (1) are usually set for open air conditions and require modification for the tunnel environment, where the C term, in particular, is larger.

Armstrong and Swift [START_REF] Armstrong | Lower energy technology. Part A, identification of energy use in multiple units[END_REF] proposed a set of empirical expressions to determine the coefficients A, B and C of the Davis equation for the electric multiple units (EMU) in service at that time on the former British Rail lines. The coefficients A, B, C in (1) are put in relationship with the following constants:

PC TC m a m a A 2 1 + = (2) P n b n b m b B PC TC 3 2 1 + + = ( 3
) P B B x PC TC g x n c n C c n n dI c dl c S C c C 5 4 3 2 1
) 1 (

+ + + - + + + = (4)
where: m TC in tons is the total mass of trailer cars; m PC in tons is the total mass of power cars; m in tons is the train mass; n TC is the number of trailer cars; n PT is the number of power cars; P in kW is the total power; C x is the head/tail drag coefficient; S in m 2 is the cross-sectional area; d in m is the perimeter; l in m is the train length; I g in m is the inter-vehicle gap; C x B is the bogie drag coefficient; n B is the number of bogies; n P is the number of pantographs.

The units of the empirical coefficients a 1 to c 5 are chosen to results in the correct units for A, B and C, e.g. c 3 is in [N s 2 /m 3 ]. Armstrong and Swift provide values for a 1 to c 5 that lead to the following expression:

A = 6.4 m TC + 8.0 m PC B = 0.18 m + 1 n TC + 0.005 n PC P C = 0.6125 C x S + 0.00197 d l + + 0.0021 d I g (n TC + n PC -1) + + 0.2061 C x B n B + 0.2566 n P
Different authors [START_REF] Rochard | A review of methods to measure and calculate train resistances[END_REF][START_REF] Armstrong | Lower energy technology. Part A, identification of energy use in multiple units[END_REF][START_REF] Gawthorpe | Train drag reduction from simple design changes[END_REF][START_REF] Lukaszewicz | Running resistance -results and analysis of full-scale tests with passenger and freight trains in Sweden[END_REF] in recent years have described the approaches of various national railway undertakings to the calculation of train resistance. Most of them are empirically modified versions of the Davis equation and include coefficients related to particular types of rolling stock, putting A, B and C in relationship with different figures not considered in Armstrong-Swift equations (such as the number of axles and the axle load). This of course adds a further uncertainty contribution including additional terms in the expressions above for the coefficients A, B and C.

B. Other terms of train resistance force

In addition to the three friction forces originated by the mass, viscous component of mass and aerodynamic characteristic described by [START_REF] Rochard | A review of methods to measure and calculate train resistances[END_REF], the traction equipment of trains also has to counteract the resistance to acceleration, the gradient force and the curving resistance.

Acceleration is simply a function of the masses with the necessary allowance for the rotating components (also known as "rotary allowance"), while curving is highly dependent on wheel and rail profiles, track cant and the geometry of the vehicle concerned.

Gradient resistance

The gradient force is mass-related and can be added as an equivalent linear force:

mgi R slope = (5)
where: m in kg is the mass of the train; g in m/s 2 is the gravity acceleration; i in % is the track grade (slope).

The grade i can be expressed as: i = 100 tan(α) = 100 ∆y/∆x [START_REF] Scibor-Rylsky | Road vehicle aerodynamics[END_REF] where: α in rad is the slope angle; ∆y in m is the rise; ∆x in m is the horizontal run.

Resistance due to track curvature The curve resistance is an estimate of the added resistance a vehicle has to overcome when operating through a horizontal curve. The exact details of the mechanics contributing to curve resistance are not easy to define. The effects of resistance due to track curvature are small for curves with radius larger than 250 m.

A common formula for calculating the resistance due to track curvature, provided by Profillidis [START_REF] Profillidis | Railway Engineering[END_REF], is

        = c c R k r 01 . 0 (7)
where: r c in kN/t is the specific resistance force, assuming that the gravity acceleration is 10 m/s 2 ; k is a dimensionless parameter, depending on the train design and varying in general from 500 to 1200; R c in m is the curve radius in a horizontal plane. However it is generally accepted in the railway industry that the curve resistance is approximately the same as a 0.04% up grade per degree of curvature for standard gauge tracks [START_REF]Practical Guide To Railway Engineering[END_REF]. At very slow speed (up to 5 km/h) the curve resistance is closer to 0.05% up grade per degree of curve.

C. Operating schedules and timetabling

To solve the power consumption of the train operation and to investigate the energy efficiency problem, it is necessary to consider the dynamic load behavior of the train set along the route. Trains have allocated time-space slots that are coordinated in a specified timetable and in general in the traffic management rules.

For the scope of this work, it is considered that the timetable takes into account safety issues and main infrastructure constraints (e.g. speed and acceleration limits), while any disturbances (occurring due to unexpected situations caused by unpredicted behavior of infrastructure, trains and staff) are not considered.

For each operation mode, the power demand of the train set can be solved based on the acceleration and train resistance to motion. equation ( 8) describes the motion equation of a train set:

a M R F eq t t = - (8)
where: F t in N is the traction effort; R t in N is the total resistance to motion; M eq in kg is the equivalent mass of the train set; a in m/s 2 is the acceleration.

The total resistance to motion can be calculated through the set of equations implemented into the mechanical subsystem, while the train set acceleration is imposed in accordance to vehicles timetabling. Some of the network infrastructure and trains data needed to characterize the system from the traffic perspective are listed below:

• number of stations and block sections;

• number of line and platforms which are existing in the stations; • track maintenance periods (section unavailability times); • length of the block sections;

• train set maximum speed; • train set priority; • train dispatching frequency; • train route in the network;

• train initial departure interval with tolerance;

• section free running time and scheduled stop times; • safety rules;

• headway times between trains.

D. Mechanical subsystem database

In order to guarantee an efficient data access and data management during the simulation process, data are stored into specific database tables, designed to host with an integrated approach the data required, processed and managed by each simulation module. Of course, this and the electrical database are two views of the same database where the transportation system elements are stored.

Basic Database

The Basic Database refers to the system data required when the simulator is operating, mainly including line database, traffic database, locomotive database and car database. The abundance of basic data is powerful for the simulator to meet the simulation requirements of different trains and lines. The above databases can be further supplemented or extended by users.

Line Database Line Database contains the line geometry and mechanical properties that influence train acceleration and movement trajectory. Examples are gradient values, curvatures, rail type and conditions, tunnel/viaduct/full line, station information, etc..

Trainset Database

The information in the database is crucial to the electromechanical simulator. The detailed parameters related to locomotives, vehicles, EMUs, train consists and simple cars are provided in the database. Examples of these parameters are identifier, description, weight, length, construction speed, resistance equations coefficients, list of on-board loads, energy efficiency of the various power-train elements, regenerative braking policy and characteristics, etc..

Traffic Database

It contains all the rules to be implemented (jointly or in alternative) for the circulation on the simulated lines. Examples of criteria are the fulfillment of a timetable, of a speed profile, of acceleration/deceleration constraints. Other criteria that may be implemented beforehand, during the preliminary sizing of the system, are the maximization of the number of passengers in the unit of time or optimization of the origin-destination matrix entries. the predefined train formations are used to achieve the high level of transportation service. To need the transportation service, the locomotives and cars are accordingly chosen from the train formations.

III. ELECTRICAL SUBSYSTEM

The electrical subsystem consists of the representation of the electrical network that was adopted in the past using a Multiconductor Transmission Line (MTL) approach, as reported in [START_REF] Mariscotti | Distribution of the traction return current in AC and DC electric railway systems[END_REF] [START_REF] Cella | Measurement of AT Electric Railway System currents and validation of a Multiconductor Transmission Line model[END_REF] (to exemplify applications to dc, 16.7 Hz and 50 Hz systems). The overall set of parameters of the electrical subsystem is very complex and articulated, regarding the perunit-length parameters of conductors and conductor groups [START_REF] Mariscotti | Determination of the Electrical Parameters of Railway Traction Lines: Calculation, Measurement and Reference Data[END_REF], the equivalent circuits of electrical equipment and machines (autotransformers, substations -either dc or ac -, impedance bonds, etc.), the inclusion of practical corrections for many configurations and phenomena that would require an impressively increase of the model complexity, going towards finite element techniques and equivalent circuit analysis (effects of viaducts, of buried cables, etc.).

Yet, such a detailed and complete representation of the traction line network is not needed for the purpose of electromechanical simulations, notwithstanding a requirement for the accuracy of the results and of the correct representation of phenomena. The MTL representation mentioned above is being used for interference analysis over a wide frequency range [START_REF] Mariscotti | Voltage coupled to wayside interconnecting cables[END_REF] [START_REF] Mariscotti | Modeling of Audiofrequency Track Circuits for validation, tuning and conducted interference prediction[END_REF], and necessitates a simplification before being used for the electromechanical simulation. This is necessary if the difference in length and extension of the line sections taken into account for interference analysis and electromechanical simulation is considered: a few km for the former, several tens of km, or hundreds of km, for the latter.

What an integrated simulator shall offer is the same input interface and representation for the same object, whatever the actual use in the specific case under study. For this reason not only the electrical and mechanical representations of the line are integrated, with overlapping layers that can be inspected at once and located on a geographical map, but the simplified electrical network for electromechanical simulation shall be (and is) derived automatically from the most complete MTL representation [START_REF] Mariscotti | Simplified modelling of 2x25 kV AT Railway System for the solution of low frequency and large scale problems[END_REF].

The requirements for the accuracy of the simplified representation are now briefly reviewed, in the light of the requirements for the pantograph interface. It was decided that during an electromechanical simulation (intended in a wide sense) the following quantities are of interest (maybe, of course, not at the same time): active power at the fundamental, P 1 (also including dc systems accepting some notation approximations), total active power (including harmonic active power P h ), P t = P 1 +P h , reactive power at the fundamental Q 1 and total reactive power Q t . The term "harmonic" is abused to cover both the harmonics of an ac fundamental, and all the components at frequencies not harmonically related to the fundamental (such as the emissions of on-board dc/dc converters and inverters).

The power exchanged with the traction line by modern rolling stock is only partially located at the system fundamental (either dc or ac). Traction drives and auxiliaries are characterized by significant conducted emissions; the product of the pantograph voltage and current spectra leads to several active and reactive power terms located on the frequency axis, that represent a significant fraction of the total power and that necessitate adequate modeling of both the line and the source. The former is readily obtained by the MTL formulation, while the latter is related to the models adopted for the rolling stock.

An example follows of the distribution of the power terms for a dc 3 kV, a 50 Hz 25 kV and a 16.7 Hz 15 kV vehicle, based on real measurements. The power supply indexes that are shown are the vehicle absorbed active power P 1 and the pantograph current I 1 at fundamental, the active harmonic power P h over P 1 and the total harmonic power A h over the apparent power at fundamental A 1 , the reactive power over apparent power at fundamental Q 1 /A 1 and the active power over apparent at harmonics P h /A h . The calculations have been performed for each vehicle over long enough time intervals of about 400 to 900 s, experiencing much different operating conditions.

The harmonic content of dc systems is almost always negligible and that the related calculation of power flow at dc is satisfactory. The considered dc system is a heavy railway system, so with a large supply power with respect to the absorbed power; it is expected that for metro systems the relevance of the largest harmonics is higher. On the contrary, in ac systems, and in particular in 16.7 Hz systems, the simulation of power flow cannot be done satisfactorily without including also the power terms at harmonic components. Also the electrical subsystem is to be supported by a database of parameters that are relevant to the determination of electrical parameters. This database contains geometrical data of cross sections, electrical properties of materials (e.g. resistivity, dielectric constant, magnetic permeability, etc.), types of longitudinal connections, etc. Even if the mechanical parameters and the electrical parameters are not so strictly related, so that the two database do not need to be tightly integrated, a certain degree of integration is necessary: it ensures a homogeneous representation of the line, a correspondence to real elements with relevance for both mechanical and electrical aspects.

IV. INDEXES FOR ENERGY EFFICIENCY

Modern electrified traction systems include switched resistor, regenerative chopper, onboard and wayside energy storage devices. The calculation of energy consumption requires consideration of energy drawn, energy available for return, and energy flows actually occurred between different vehicles for the various operating conditions of the particular transit system. Due to nonlinear characteristics introduced by regenerative traction systems, straightforward average value techniques (similar to those used for sizing substations) cannot be relied upon to generate the average for all cases. It is necessary, therefore, to analyze each transit system operating condition in detail.

The task of enumerating the energy evaluation factors on a per-traction system basis is not straightforward, since the recoverable kinetic energy is a function of various factors: the type and distribution of the other trains and the degree of multiple-route electrical interconnection on a traffic/mechanical level, the impedance at the pantograph, the longitudinal voltage drops, the available short circuit power and the allowed voltage rise in the power distribution system.

Considering the energy flows between vehicles -generated by regenerative breaking -the receptivity index can be used to evaluate the energy efficiency of the transit system. The premise here is that receptivity is a measure of the fraction of power available for recovery that, after being subjected to certain necessary constraints, is actually recovered.

Various definitions have been applied to the term receptivity by numerous interested groups [START_REF] Phelps | Assured energy receptivity study-A project overview[END_REF], [START_REF] Mellitt | Simulation study of DC transit systems with inverting substations[END_REF][18], [START_REF] Forsythe | Light Rail/Rapid Transit: New Approaches for the Evaluation of Energy Savings, Part II -On the Receptivity of a Transit System[END_REF]. Receptivity may be defined in terms of power or energy; calculated either at the substation, train, or traction system; and may be based on a per-streetcar or total-streetcar system calculation.

Here receptivity is defined as the ratio of the braking power actually regenerated to the braking power available for regeneration (also named "recoverable kinetic energy"), calculated at the pantograph level.

To evaluate the receptivity certain characteristics peculiar to regenerative systems must be recognized. The combination of long distance energy transmission and strong interaction between vehicles (energy flow) generally requires a complete and simultaneous simulation of the entire transit system. In addition, the nonlinear characteristics brought out by regeneration preclude the general use of techniques based on averaging of system variables and use of average values. On the contrary, the transit system must be examined over the full range of its operating conditions, considering the mix of loads/sources.

At any instant of time the important parameters for receptivity estimation are:

1) total car power drawn from the supply system; 2) total car power available for return to the supply system; 3) total car power actually returned to the supply system.

Vehicle power draw can be calculated as a function of vehicle location and time, considering the system route structure and the physical and electrical characteristics of the vehicles. The recoverable kinetic energy and the power actually recovered depend on the vehicle speed profile at any instance and on the relative location and state (e.g. acceleration, breaking, cruising) of different vehicles moving on the same line.

One solution to the problem consists in adopting the integrated electromechanical simulation approach. Through an adequate set of equations / parameters (as described above), the motion power requested by the vehicles is calculated, while the actually recover power could be estimated from the analysis of transit system operating condition, defined by: 1) vehicle scheduling, 2) vehicle speed profile per mission.

Vehicle speed profile per mission defines at any instant the state of the train and allow to calculate the total power drawn from the supply system. Furthermore the vehicle scheduling (timetable) allows to correlate each vehicle power demand and recoverable kinetic energy with relative location and state of train of other vehicles moving on the same line.

As the receptivity depends on the relative state of train of two or more vehicle on the same line, to reduce the receptivity calculations, vehicle scheduling could be quantified into a discrete number of constant vehicle density cases (e.g., daily rush hour, daily norm, daily evening, and owl) in which the total number of car hours is maintained, and the variation in the total number of cars in the system for each case is acceptably low [START_REF] Forsythe | Light Rail/Rapid Transit: New Approaches for the Evaluation of Energy Savings, Part II -On the Receptivity of a Transit System[END_REF].

V. CONCLUSIONS

In the present work an overview of the phenomena and equations related to the mechanical motion of a train are presented. The electromechanical analysis and simulation of traction systems of various kinds (heavy and light railways, metros, etc.) are required to estimate the power consumption and where optimization can be applied more profitably for energy savings. Energy efficiency has been the target of a great research effort all over the world and large traction systems are subject to a huge number of variables (mechanical and electrical), an intrinsic difficulty in the identification of system boundaries and related power flows, and a high degree of uncertainty, in particular on mechanical variables and parameters. Here the sensitivity of the mechanical equations to the set of parameters describing the track and the trains was considered, together with the spread of the output variables for typical variations of the input parameters.

Figure 1 .

 1 Figure 1. Power indexes for (a) dc 3 kV, (b) 50 Hz 25 kV and (c) 16.7 Hz 15 kV: P1 and I1 (above), Ph/P1 and Ah/A1 (middle), Q1/A1 and Ph/Ah (below)