Jana Trgalova

Denis Bouhineau

Jean-François Nicaud

AN ANALYSIS OF INTERACTIVE LEARNING ENVIRONMENTS FOR ARITHMETIC AND ALGEBRA THROUGH AN INTEGRATIVE PERSPECTIVE

Keywords: Interactive learning environments (ILE), arithmetic, algebra, analysis of an ILE

The analysis presented in this article tries to obtain a global view of the field of interactive learning environments (ILE) dedicated to arithmetic and algebra. As preliminaries, a brief overview of evaluation methods focusing on educational software is given and a short description of ten ILEs concerned by the study is provided as a kind of a state-of-the-art. Then the methodology of ILEs analysis developed in the TELMA project is explained consisting in the design and the refinement of an analysis grid and its use on the ten ILEs is mentioned. Next, a first level analysis of results leading to a compiled, analytic and synthetic view of the ILEs available and/or missing functionalities is given. A second level of the analysis is also proposed, with two concise representations of the ILEs, composed of graphical representations of the previous results, leading to a 3D map of ILEs dedicated to arithmetic and algebra. This map provides, as promised, a global view of the field and permits to define five sorts of ILEs according to two criteria: the first one is teacher-oriented and concerns usages enabled by the ILE; the second one is student-oriented and concerns control provided by the ILE to accomplish such usages.

I. Introduction

This paper is devoted to a study of Interactive Learning Environments (ILEs) in the field of arithmetic and algebra carried out within the framework of the TELMA project (Bottino & Kynigos, this issue). Most of the TELMA teams are involved in the design and development of educational software, namely arithmetic, algebraic, dynamic geometrical and geometric-constructionist software, and all of them are interested in its use for the teaching and learning of mathematics. Getting a deeper insight to the landscape of state-of-the-art ILEs for mathematics became one of the objectives of the project. The aim of this study is not to judge the quality of the systems analysed, but rather to gain a global view of educational software for arithmetic and algebra, to identify and compare didactical functionalities (Cerulli et al. 2005, Artigue et al. this issue) of these systems in terms of available (or not available) features and functionalities.

Analysis of educational software is not a new issue and various methods have been proposed to study and evaluate computer-based learning environments. In section II, we give a brief overview of research studies dealing with analysis, evaluation and classification of learning environments. However, these studies concern usually the artefact only, and many issues of interest for TELMA teams, such as motivation of the designers, rationale for the design choices, difficulties encountered in the development process, remain hidden and unattainable by classical evaluation methods. Therefore, TELMA teams needed to develop a specific methodology enabling them to get light also on such implicit and obscure issues.

A methodological tool (section III), in the form of an analysis grid, has been designed to analytically describe interactive learning environments so that a better understanding is gained about all important issues in such environments. The tool aims at framing an analysis by inspection of the ILEs in order to gather as much information as possible not only about the artefacts, but also about their life cycles, starting from the design choices up to their experimentations and/or usages. The tool has been designed and improved through collaborative work between TELMA teams. The tool has been consequently applied to the analysis of ten existing ILEs for arithmetic and algebra: three ILEs developed by TELMA teams and seven ILEs developed outside TELMA (a brief presentation of these ILEs is provided in section IV). Two independent studies of the ILEs have been performed by two different TELMA teams and the results were compared and combined to obtain a unique analysis grid filled in for each ILE. A first analysis of the grids, done by the authors of this article, allowed comparing the ILEs based on features related to the knowledge domain, the user-ILE interaction and their technical aspects (section V). A second level of analysis is proposed in section VI, with two concise representations of the ILEs. They consist of graphical representations of the first level analysis results and lead to a 3D map of the studied ILEs providing a global view of the landscape of educational software for arithmetic and algebra.

II. A brief overview of the evaluation methods of educational software

There seems to be a general agreement that evaluation of educational software is crucial and often constitutes an important phase of its development cycle [START_REF] Nogry | Evaluation des EIAH : une nécessaire diversité des méthodes[END_REF][START_REF] Vivet | Evaluating Educational Technologies : Evaluation of Teaching Material Versus Evaluation of Learning[END_REF][START_REF] Squires | Predicting quality in educational software: Evaluating for learning, usability, and the synergy between them[END_REF][START_REF] Senach | L'évaluation ergonomique des interfaces homme -machine[END_REF]. Various methods for ILE evaluation are proposed in the literature. Tricot et al. (2003, p. 392) classify such methods according to two aspects:

(1) according to the way the evaluation is done, they distinguish between evaluation by inspection, which is performed by experts who use a set of predefined criteria to evaluate various aspects of the software, and empirical evaluation, which consists in observing users and interpreting their behaviours, attitudes and opinions. These two kinds of evaluation are obviously distinct and complementary.

(2) according to the moment when the evaluation is done, they distinguish between evaluation during the design phase, evaluation at the end of the design phase (validation), and a posteriori evaluation, i.e., when the system is not only designed but also realised. Some authors suggest integrating the evaluation of the software from the very beginning of the design process [START_REF] Nanard | La conception d'hypermédias. Hypertextes et Hypermédias[END_REF].

In order to clarify relationships between functionalities of an ILE, their interest for the user and the ease of their use, [START_REF] Senach | L'évaluation ergonomique des interfaces homme -machine[END_REF] considers two main dimensions in the ILE evaluation: (1) utility, which expresses the pedagogical efficiency of the ILE (does it allow achieving the specified educational goals?), and ( 2) usability, which evaluates the easiness of manipulating the ILE (is the ILE userfriendly, easy to use and reuse?). Tricot et al. (op. cit.) add a third dimension of the ILE to be taken into account in the evaluation process: its acceptability, which is related to the decision to use the software (is the ILE in accordance with the values, culture, and organisation of the institution where one wants to use it?). The authors provide sets of criteria allowing evaluation of an ILE from the points of view of utility, usability and acceptability, both by inspection and empirically.

While numerous research works deal with the issue of evaluation of educational software, a few propose concrete methods to evaluate its quality. [START_REF] Hû | A tool for evaluation using dynamic navigation in a set of questions[END_REF] have developed a method to evaluate interactive learning multimedia systems organised around six themes (see also [START_REF] Trigano | CEPIAH, a method for Evaluation and Design of Pedagogical Hypermedia[END_REF]:

(1) General feeling (i.e., user's feelings about the use of the tool: active/passive, simple/complex, innovative/traditional, reassuring/confusing…);

(2) Technical quality (i.e., evaluation of the technical realisation: functioning, speed, installation, bugs…);

(3) Usability (i.e., ergonomic quality of the interface: manipulation, flexibility, homogeneity, adaptability…);

(4) Multimedia documents (i.e., quality and relevance of texts, sounds, images…);

(5) Scenario (i.e., quality of presentation and assembly of information: structure, browsing…);

(6) Pedagogical tools (i.e., pedagogical possibility of the software: quality and validity of content, variety and richness of activities offered by the tool, diagnosis tools, tutoring tools, hints…).

Evaluation of educational software for algebra is of special interest for TELMA teams. [START_REF] Nicaud | A questionnaire for educational environments for algebraic reasoning[END_REF] elaborated a specific questionnaire to analyse ILEs for algebraic reasoning. He considers five categories of features:

(1) Domain/capacity of the ILE (e.g., extent of the domain covered by the ILE, power of the ILE in the domain, way of editing expressions…);

(2) Inference/interaction with the ILE (e.g., who selects the rule to be applied? who applies it? who selects the sub-expression?…);

(3) Adaptability of the ILE (e.g., possibility to activate/deactivate pieces of knowledge…);

(4) Heuristic feature of the ILE (e.g., is backtracking possible?…);

(5) Tutoring features of the ILE (e.g., nature of feedback or hints when provided…).

The above mentioned methods allow evaluating an artefact by inspecting its various functionalities and features, but they do not take into account issues related either to the design and development phase or to the usage of the artefact.

These issues are of crucial interest for TELMA teams since, on the one hand, most of them are themselves involved in the development of educational software and learning about design and development issues related to similar systems may be an interesting and rewarding experience. On the other hand, information about actual or experimental usages of the system would be valuable in order to get a more complete picture of the artefact, gained both from the analysis by inspection and from results of empirical analyses.

Since the design and development of educational software bring together researchers and developers from different scientific fields, such as computer science, education, ergonomics, psychology, cognitive sciences, they need to face a multiplicity of points of view, which often results in deep misunderstandings and can thus turn to be a brake, rather than a catalyser of a fruitful cooperation among the disciplines. To cope with this diversity of approaches, [START_REF] Tchounikine | Platon-1: quelques dimensions pour l'analyse des travaux de recherche en conception d'EIAH[END_REF] propose a set of dimensions for ILE analysis offering a framework for a better communication and mutual understanding. The authors specify that the framework concerns both the "ILE design project" and the "ILE" itself considered as a result of the design. The dimensions taken into account are organized into four groups:

[A] dimensions related to the definition of the research project, such as research aims, constraints on the ILE development, purpose of the artefact, actors involved in the design, social context of the research;

[B] dimensions related to the theoretical framework of the research, such as reference to the knowledge domain, theoretical frameworks specified, their role and the way they are used in the design;

[C] dimensions related to the results of the research, such as nature of the results, kind of their validation, their impact;

[D] dimensions related to the life cycle of the research, such as context of initiating the research, its history.

One of the authors of the present paper participated at the elaboration of the two above mentioned tools for ILE analysis, and has also been involved in the design of the methodological tool developed by TELMA teams. This tool, presented in the following section, draws on the above mentioned methods and criteria for evaluating educational software by inspection. Since the tool is aimed at studying specifically systems for teaching and learning arithmetic and algebra, the items of the grid are inspired by Nicaud's questionnaire (2001). Moreover, as suggested by [START_REF] Tchounikine | Platon-1: quelques dimensions pour l'analyse des travaux de recherche en conception d'EIAH[END_REF] study, the tool takes into account the whole life cycle of the development of a system, starting from the aims of the design, through a detailed analysis of the artefact, up to the validation phase by means of empirical studies.

III. TELMA methodology for the ILEs analysis

III.1. Preliminary remarks

Our study concerns educational software already in use or at least at the stage of an advanced prototype. Therefore, referring to [START_REF] Tricot | Utilité, utilisabilité, acceptabilité: interpréter les relations entre trois dimensions de l'évaluation des EIAH[END_REF] classification, the analysis TELMA teams carried out can be considered, as an a posteriori evaluation. Organizing conditions for an empirical evaluation of the selected systems was beyond the scope of the TELMA project, thus the analysis was done exclusively by inspection. Evaluation by inspection raises two fundamental issues.

The one is linked to the fact that the knowledge of experts involved in the evaluation is individual, acquired by a professional practice, and is thus seldom made explicit (Senach 1990, p. 39). The other comes from the fact that an expert's evaluation is necessarily biased by focusing on aspects of the system relevant to her/his speciality and neglecting others (ibid.). An evaluation cannot be legitimate unless experts from different scientific domains are involved in it. To cope with these limits of the evaluation by inspection, a usual method consists of defining an evaluation grid providing a list as exhaustive as possible of features and aspects of the system to be evaluated. The first step in the TELMA study was therefore to define an analysis grid that would shape the inspection of the relevant aspects of the selected ILEs.

III.2. Design of an analysis grid

Recall that one of the main objectives of the TELMA project was to investigate the role theoretical frameworks play in the design both of educational software and of teaching experiments. Therefore, highlighting the theoretical background of the design of the ILEs was of particular interest for the TELMA study.

Moreover, since some TELMA teams are involved in a process of design and development of educational artefacts, we supposed that considering the artefact as a result of a research project, and thus analysing not only the artefact itself but all possible aspects of its life cycle would be much more insightful with respect both to the research interests of the TELMA teams and to the purpose of our study.

Therefore, [START_REF] Tchounikine | Platon-1: quelques dimensions pour l'analyse des travaux de recherche en conception d'EIAH[END_REF] analysis framework seemed particularly relevant to our objectives. Since our study concerned educational software for arithmetic and algebra, most of the criteria from [START_REF] Nicaud | A questionnaire for educational environments for algebraic reasoning[END_REF] questionnaire were applicable to our study. Drawing on both above mentioned analysis frameworks, a specific methodological tool was elaborated through a collaborative work between TELMA teams. The initial version of the analysis grid was organized around three periods of life of an ILE (see Appendix 1).

-The Design and development part aims at highlighting the objectives of the authors (who and why), the underlying principles and theoretical background (how) and the intended audience (to whom).

-The Artefact part addresses technical aspects of the ILE, issues related to the interface (representation and manipulation of mathematical objects), the domain covered by the ILE, the interactivity and tutoring features (types and nature of feedback, student's autonomy, ILE's knowledge of the task), and the role of teacher.

-The Validation part aims at gathering information about empirical validation of the ILE from published papers and available research reports. This aspect seems particularly important in our study to complement our analysis by inspection.

This version of the analysis grid was used to analyse 9 of the ten ILEs concerned by the study. Each ILE was studied by researchers from two different TELMA teams. For some ILEs, two independent grids were filled in, for others, the results of the second study complemented the grid of the first study. The completed grids were then sent to authors of the ILEs to inform them about the study and to get their feedback. Results of an analysis of the completed grids are reported in a compiled and synthetic form in section V. Besides gathering information about the ILEs analysed by using the grid, the teams were invited to report about the methodological tool itself focusing on relevance, appropriateness and clarity of the items, on their applicability to the software analysed, as well as on missing aspects if any.

Taking into account all teams' remarks and suggestions concerning the initial analysis grid, a new, refined version of the analysis grid has been elaborated (see Appendix 2). The most significant change was making an effort to explicit the nature of expected answers. The refined version of the analysis grid was subsequently used to analyse one additional ILE.

IV. ILEs concerned by the study IV.1. Criteria for the choice of the ILEs

The selection of the ILEs to be studied was based on the following criteria (Nicaud 2006, p. 3):

(a) they were produced by research laboratories or companies;

(b) they were in use in schools on a regular basis or they have been at least experimented by a significant number of students;

(c) they were either products or advanced prototypes;

(d) they have a high level of interactivity.

IV.2. Brief description of the ILEs developed outside TELMA and

selected for the study

IV.2.1. ActiveMath (http://www.activemath.org/)
ActiveMath is a rich web-based learning environment for mathematics [START_REF] Melis | ActiveMath: A web-based learning environment[END_REF][START_REF] Melis | Design of Erroneous Examples for ActiveMath Proceedings of the 12th International Conference on Artificial Intelligence in Education[END_REF][START_REF] Melis | How ActiveMath Supports Moderate Constructivist Mathematics Teaching[END_REF]. It integrates several tools such as a function Plotting Tool, a Computer Algebra System or an interactive Concept Map Tool, which can be used when necessary. Based on the system's competence assessment, a learner is presented with theoretical and interactive material and has great freedom in the way s/he learns. Interactive exercises respond with feedback and hints to the learner's input. ActiveMath permanently records and assesses the performance of a learner in the exercises and the result can be accessed by the learner. The learner can ask for more information by clicking on mathematical terms or on definitions and by making use of the semantic search facility.

ActiveMath provides a variety of contents, e.g. calculus, algebra, combinatorics. IV.2.2. AnimalWatch (http://k12.usc.edu/AW/) AnimalWatch is a web-based tutoring system for middle school mathematics with a specific focus on helping students with the transition from arithmetic to algebra [START_REF] Arroyo | Animalwatch: an arithmetic ITS for elementary and middle school students[END_REF][START_REF] Arroyo | Learning within the ZPD with the AnimalWatch intelligent tutoring system[END_REF]. The system proposes word problems about endangered animal species (e.g., Giant Panda, Right Whale, Mongolian Takhi Wild Horse). For each problem, students can request a multimedia explanation, a worked example requiring the same skills, or a video lesson. They complete the word problems, organized into "virtual narratives". Their progress is customized on the basis of prior math achievement, motivation, and ongoing performance.

Current content includes arithmetic, fractions, pre-algebra, and data analysis.

IV.2.3. Cognitive Tutor (http://www.carnegielearning.com/)

The Cognitive Tutor curricula for middle school and high school students include Bridge to Algebra, Algebra I and II, Geometry, Integrated Math and Math Prep products (Koedinger et al. 1997, Aleven and[START_REF] Aleven | An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor[END_REF]. Each curriculum Immediate feedback is provided to each step of the solution, and guidance and help is available on demand. MATH-TEACHER series consists of modules that cover much of the math curriculum of grades 7 through 12, e.g. algebra, calculus, geometry, probability, etc. MATH-TEACHER includes a Test Generator utility which enables the teachers to easily prepare homework and tests. IV.2.5. MathXpert (http://www.helpwithmath.com/index.php) MathXpert is a computer program designed to help students learn algebra, trigonometry, and one-variable calculus by assisting the student in developing his/her own step-by-step solution [START_REF] Beeson | Mathxpert, a computerized learning environment for Algebra, Trigonometry and Calculus[END_REF][START_REF] Beeson | Design Principles of Mathpert: Software to support education in algebra and calculus[END_REF]. The design of MathXpert follows principles dictated by its intended use to support learning, such as any error has to be avoided because a slight error can throw you completely off track, mathematics is cumulative, therefore you must master each part of the subject before moving on to the next one, or transparency principle meaning that the user chooses the steps and thus controls the development of the computation, while the computer carries out the low-level details.

IV.2.7. T-algebra

T-Algebra is an interactive learning environment for step-by-step solving of algebra problems [START_REF] Lepp | Expression Editor Features That Simplify Student Work On Manipulating Expressions[END_REF][START_REF] Prank | Problem solving environment T-algebra[END_REF]. The following problems can be worked out by the software: calculation of the values of numerical expressions, operations with fractions, solving linear equations, inequalities and systems of linear equations, and operations with monomials and polynomials. Talgebra makes possible the diagnosis of errors by defining each solution step as consisting of three stages: selection of the transformation rule, marking the parts of the expression the rule will be applied on, entering the result of the operation. Aplusix is an ILE for teaching and learning secondary school arithmetic and algebra [START_REF] Nicaud | A computer program for the learning of algebra: description and first experiment[END_REF], Nicaud et al. 2004). It lets students solve exercises and provides two fundamental feedbacks: it verifies the correctness of the calculations and of the end of exercises. Aplusix has been designed to be integrated into the regular class work: it is close to the paper-pencil environment, it uses a very intuitive editor of algebraic expressions (in two dimensions). It contains 400 patterns of exercises. A number of parameters allow for customisation (e.g., limit or disable commands, prevent the system from giving the solution). Aplusix records all students' actions which can be then observed by the student and the teacher by means of a Replay system. Teachers can also access to statistics concerning their classes indicating amounts of exercises the students have worked on, amounts of well-solved exercises, amounts of incorrect calculations, and scores. IV.3.3. E-Slate (http://e-slate.cti.gr/) E-Slate is an exploratory learning environment that provides a workbench for creating dynamic software with rich functionality by non-programmers [START_REF] Kynigos | E-slate Logo as a basis for constructing microworlds with mathematics teachers[END_REF][START_REF] Kynigos | Using half-baked microworlds to challenge teacher educators' knowing[END_REF]. Educational activity ideas can be turned into software with minimal authoring effort in the form of interactive Microworlds which contain specially designed educational components. A Logo-based scripting language can be used to programme the behaviour of the components and the kind of connections between them. E-Slate components are provided as a kit of pre-fabricated, interoperable computational objects. Software Microworlds can be constructed easily, by plugging components in various configurations.

The "construction kit" is intended to provide developers and authors with tools for producing educational software. However, the educational contingent of the development team has also produced microworlds within E-Slate (e.g., Fraction microworld) that may be used by teachers and students as exploratory learning

V. Results of the ILE study

The grids filled in collaboratively for each ILE1 were analysed by the authors with the aim of providing a first level comparison of the main aspects of the ILEs. The features and functionalities to be compared were organized into three categories:

(1) knowledge domain of the ILE, corresponding roughly to the domain/capacity of the ILE defined by [START_REF] Nicaud | A questionnaire for educational environments for algebraic reasoning[END_REF], ( 2) user-ILE interaction covering interaction and tutoring features of the ILE as well as its adaptability (ibid.), and (3) aspects related to the accessibility and availability of the ILE. For each category, several criteria have been chosen to describe the corresponding features of the ILEs as precisely as possible and the results for each criterion and for all 10 ILEs are reported in the tables in Appendices 3-5. Below, we present some of the most salient features of the ILEs and attempt to situate each of the ten ILEs with respect to the others.

V.1. Features related to the knowledge domain

V.1.1. Algebra content covered by the ILEs

The analysis of the ILEs from the point of view of mathematical content covered by the ILEs shows that some ILEs are devoted to very specific algebraic domains or tasks, such as fractions (E-slate), symbolization, i.e., translation of word

V.1.2. Representation systems available in the ILEs

The ILEs differ from the point of view of implemented representation systems allowing to display and manipulate algebraic expressions. For example, T-algebra uses a unique, usual algebraic representation system to handle the expressions, while ActiveMath offers textual, graphical and geometric representation systems in addition to the usual one. Figure 12 represents a ranking of the 10 ILEs with respect to the number of available representation systems.

V.1.3. Manipulation of algebraic expressions allowed by the ILEs

When analyzing the way algebraic expressions are manipulated in the ILEs, three categories can be identified:

1. input-based systems, such as Aplusix, ActiveMath or MathTeacher+, in which a user enters algebraic expressions and transforms them more or less freely;

2. rule and input-based systems, such as T-algebra or CognitiveTutor, in which the expressions can be entered freely by a user but transformations of expressions are done by choosing a rule from a list of applicable rules. The result of the transformation is provided by the user; 3. rule-based systems, such as AriLab2 or MathXpert, in which expressions are transformed by the system, i.e., a user only selects a transformation rule and a sub-expression on which the transformation should be applied, and the system transforms the expression.

V.2. Features related to the user-ILE interactions

V.2.1. Feedback provided by the ILEs

The analysis of feedback provided by the different ILEs shows that the designers of the ILEs tackle this issue in very different ways. Some ILEs are designed in a way that each student step is monitored and immediate feedback is fired when an error is detected. The ILEs provide either a visual feedback, for example the error the other half are commercial products (Aplusix, AriLab2, CognitiveTutor, MathTeacher+ and MathXpert).

VI. Comparison of the ILEs, an attempt to reach a global and visual point of view

The objective of this part is to reach a global point of view in order to compare the ILEs and to gain a global vision of the different categories of ILEs available, with some general considerations about relative position of each category with respect to the others.

The process, independent from the collaborative work described in section III, was done in two steps by the authors of the present article. In the first step, for each ILE, a radical synthesis of the study of the ILE was done in order to obtain a minimalist representation of the ILE, in the attempt to reach a visual representation of the ILE that represents it as closely to the original as possible. In the second step, the minimalist representations of all the studied ILEs were reduced again and assembled in one object to reach a global landscape of the ILEs universe.

At the same time, the objective of this section is to propose a tool for future analysis of ILEs, in the domain of mathematics, but also other domains. In the math domain, the benefit will be double: first a tool for ILE analysis will be proposed, and second, a representation of the already populated landscape of ILEs for math teaching and learning will be offered, in which a new ILE could be added and described generally.

Most of the work described below relies only on searching and counting some basic key words in the analysis grids presented in section V, but we pay a particular attention to the way the results are presented: we aim at reaching a visual representation of the results rather than provide arrays of figures of the counts, which risks not to be very expressive. We believe that a visual representation could lead to a particular insight about the ILEs of the domain.

VI.1 General considerations and fundamental choices

The first choice we have made was to determine a small set of directions in which the ILEs will be explored with the aim of reaching our global point of view. The reason was that our final result should be visually meaningful and only objects in 2D or 3D have easy-to-interpret representation.

Four directions were first considered: Knowledge, Usage, Control and Availability:

• The Knowledge direction concerns the domain covered by the ILE, in our case mathematics, and more specifically arithmetic and algebra.

• The Usage direction considers functionalities oriented towards teachers and facilities helping them build learning scenarios and organize pedagogical sequences.

• The Control direction is related to features proposed to students in order to support their learning (e.g., feedbacks, interaction features).

• The Availability direction concerns basic material and technological elements that make the ILE available in a particular circumstance. It includes price and license issues, operating system of the computer required for the running of the ILE, the natural language used by the ILE, the domain covered by the ILE. This direction is not cumulative, but Boolean and contextual, i.e., in a given context, the ILE is available or not, and it is quite easy to see, based on the characteristics given by the developers, if the ILE will be available or not in a given context. For this reason, this dimension is not taken into account in the global evaluation. This choice is slightly different from the one presented in the previous section.

The main difference consists in looking at interaction features of the ILEs. While in the first level comparison (see section V), we were interested in features from the point of view of both teachers and students, in this analysis it seems more relevant to extract features that can be considered as oriented more specifically towards the teachers and functionalities that are addressed to students. Obviously, the features for teachers and for students are not exclusive, for this reason, some features and functionalities can be relevant for two or more above mentioned directions.

VI.2 Achieving a global and visual point of view for each ILE: an ILE as a set of 3 radars

For each ILE, on the basis of the results summarized in appendices 3-5, we have searched for key words related to each of the three selected directions, and assigned points according to the following sub-directions:

• Knowledge:

-Regarding the sub-direction school level we have assigned 1 point for the ILE that can be used for only 1 to 2 school years, 2 points for the ILE that can be used during 3 to 5 years, 3 points for the ILE that can be used during 6 to 9 years and 4 points for the ILE that can be used 10 years and more. This is one of the main sub-directions of the Knowledge direction.

-Regarding the sub-direction content, we have counted the number of domains covered by the ILE among the following: Arithmetic, Algebra, Word Problem, Analysis, or Others (one point is added for Geometry, Trigonometry, Statistic, since we are interested in the ILEs for arithmetic and algebra). This is one of the main sub-directions of the Knowledge direction.

-As regards the sub-direction representation, we have counted the number of types of representation of mathematical objects available in the ILE among the following: Usual representation, Graphs, Tables, Others (one point is added for Tree, Slider, Line). Note that this sub-direction will also appear in the Control direction. It is one of the main sub-directions of the Knowledge direction.

-Concerning the sub-direction activity, we have counted the number of types of activities or exercises available in the ILE among the following activities: Formal exercises, Word problems, Open problems, Lessons and examples, Proofs and argumentation. This sub-direction will also appear in the Usage direction. This is only a secondary sub-direction of the Knowledge direction.

-Concerning the sub-direction manipulation, we have counted the number of different types of objects from the representation sub-direction that can be edited and manipulated. This sub-direction will also appear in the Usage and the Control directions. This is only a secondary sub-direction of the Knowledge direction. -The same activity sub-direction as defined in the Knowledge direction. It is one of the main sub-directions of the Usage direction.

-Concerning the flexibility sub-direction, 1 point has been assigned for the ILE with an important guidance of a student's activity (the student is not free to explore the situation, for teachers this means that the situation is reduced to the domain defined by the guidance of the ILE), 2 points have been assigned for a medium guidance and a medium freedom for the student, and as a consequence an average large space for free exploration of a situation, 3 points have been assigned for a great freedom given to the student to explore a situation, which also means that a teacher can imagine a big number of activities around the same situation. One point corresponds to Tutoring systems, three points to Microworld. It is only a secondary sub-direction of the Usage direction. This sub-direction is opposite to the rigidity sub-direction, which could be an interesting concept in another analysis.

-The same manipulation sub-direction as defined in the Knowledge direction. It is only a secondary sub-direction of the Usage direction. -The same manipulation sub-direction as defined in the Knowledge direction. It is only a secondary sub-direction of the Control direction.

For example, considering T-Algebra (the only ILE analysed with the revised grid at the end of the process) the table in appendix 3 has been analysed to construct the "Knowledge" radar. Thus for the sub-direction school level, T-Algebra gets 2 points as it is designed for students aged from 12 to 16; for the sub-direction content, it gets 1 point as it is only concerned with algebra; for the sub-direction representation, it gets 2 points as it can deal with algebraic and textual representations. Let us consider a second example. A comparison of the "Control" radars of T-Algebra (Fig. 13) and of ActiveMath (Fig. 16) shows differences in terms of control of students' actions to manage their learning. While T-Algebra enhances a hint sub-direction, ActivMath promotes rather an active manipulation subdirection.

VI.3. Achieving a global and visual point of view for all ILEs: 3Dspace of ILEs

The work done previously is intended to reduce each ILE into a point with 3 coordinates, one for each direction (as a sum of the sub-directions) and place a 3D point in a space with all the ILEs. In order to take into consideration relative importance of main and secondary sub-directions, for each direction, the sum of all points assigned to a main sub-direction is given the coefficient 1, and the sum of all points assigned to a secondary sub-direction is given the coefficient ½. In this way we obtain the figure 17 (since all the ILEs are very high in the Knowledge direction, and close one to the other, the axis for Knowledge starts with 5 points.) If the ILEs that are closest one to the other are grouped together, 5 zones emerge (see Figure 18), which can be described as follows:

• Zone A: ILEs with emphasis on Control only. Such ILEs can be used by students without a lot of help from the teacher, a control is ensured by the ILE. In this zone we will find tutoring systems. With this kind of ILE, the teacher does not have facilities to tune the ILE and forge the activity s/he wants with the ILE. Among the studied ILEs, Mrs Lindquist belongs to this zone. The ILEs from this zone can be used autonomously by students, the teacher's role may be limited to the choice of activities for the students.

This zone seems to be the one where mathematics teachers are the most active in the design of ILEs.

A

• Zone C: ILEs with equal emphasis on Control and Usage. Aplusix and ActiveMath belong to this zone. With such ILEs, the teacher needs to prepare, at least partially, the students' activities. This zone seems to be the one where computer scientists are the most active in the design of the ILEs.

• Zone D (empty zone): ILEs with tradeoffs between Control and Usage, however the usage dimension is stronger. No ILE among those studied belongs to this zone.

• Zone E: ILEs with emphasis on Usage only. It corresponds to a microworld, or an authorware ILE. Generally, such ILE does not provide any activity; therefore the teacher is in charge of designing learning activities. AriLab2 and E-Slate are within this zone.

VII. Discussion and conclusion

This paper presents TELMA approach to a study of interactive learning environments for mathematics, especially arithmetic and algebra. This study aimed at gaining a deeper insight in the features and functionalities of these systems, as well as at getting a more comprehensive view of research projects concerned by the development of such systems. Specific methodological tool in form of a grid has been elaborated by TELMA to frame the analysis by inspection of a number of aspects of the systems. It has been consequently applied to the analysis of ten existing state-of-the-art ILEs.

Although evaluation of ILEs is not a new issue and there exist numerous research works proposing methods and tools for ILE analysis and classification, it seems that they are used rather at an individual level, for evaluating a single given ILE.

These methods and tools are rarely applied, by their authors or other researchers, in order to perform comparative analyses of computer-based educational software. This situation is rather understandable: a strict separation between the definition of an evaluation and the design of an ILE has a positive aspect since it ensures a better objectivity in evaluating the ILE. On the other hand, as these methods are used rather by individuals, they are often personalized and focusing on specific features of the designed ILE. As a consequence, these evaluations can rarely be compared and a global image of the domain of ILEs can hardly be achieved.

For this reason, we believe that TELMA study reported in this paper brings a valuable contribution to the domain of ILEs for arithmetic and algebra from at least two points of view: (1) it leads to a characterization of several ILEs by means of radars, which allows comparing the ILEs with respect to some selected features, and (2) it allows a classification of the ILEs into zones determined by available features in the ILEs. Such results may be of a high interest not only from the research point of view, but also for potential users of these ILEs, teachers and students, who often complain about the difficulty to choose an appropriate computer-based program, which matches the best their own purposes.

It is worthwhile to emphasize TELMA methodological approach to the analysis of ILEs: first, each ILE developed outside TELMA has been studied by two independent teams, involving both computer scientists and researchers in mathematics education, to ensure an objective and undistorted result. Second, the ILEs developed by TELMA teams have been studied by teams who have not been involved in the design and the development of the ILE and who have used it in a teaching experiment conducted within the project. This decision was motivated by one of the results from the work done within TELMA, which is the impact of theoretical backgrounds, representation and context issues on the design, development and use of the ILE. For more details about these results, see respectively (Artigue & al., Morgan & al., Kynigos & Psycharis, all in this issue).

As was already mentioned, the ILE study was done not to evaluate one system against the others and thus establish a list of systems of better quality, but rather to gain a deeper insight in state-of-the-art technological tools devoted to the teaching and learning arithmetic and algebra. It attempted to explore transversally different directions in such tools. The results provide a better global image of the domain and allow positioning better the ILEs we develop ourselves within this landscape.

But obviously, we need to be aware of limits of this study residing mainly in the fact that the global image we give about the ILE domain is necessarily subjective.

The directions we focus on (knowledge, usage, and control) correspond to our

Figure 1 .

 1 Figure 1. ActiveMath screenshot.

Figure 2 .

 2 Figure 2. AnimalWatch screenshot.

  combines software-based, individualized computer lessons with collaborative, real-world problem-solving activities designed to emphasize connections between verbal, numeric, graphic and algebraic representations. Using the software, students receive the benefits of individualized instruction, immediate feedback and coaching. The software includes automated, dynamic assessments that provide personalized instruction for each individual student. Student's progress is constantly monitored, providing immediate feedback for both teachers and students.

Figure 3 .

 3 Figure 3. Cognitive Tutor screenshot.

Figure 4 .

 4 Figure 4. MATH TEACHER screenshot.

Figure 5 .

 5 Figure 5. MathXpert screenshots.

Figure 6 .

 6 Figure 6. Ms. Lindquist screenshot.

Figure 7 .

 7 Figure 7. T-algebra screenshot.

Figure 8 .

 8 Figure 8. Aplusix screenshot.

Figure 9 .

 9 Figure 9. ARILAB-2 screenshot.

  environments. As well as fully developed microworlds, the team has what it terms "half-baked" technological artefacts, developed specifically for use either by students or by teachers in professional development courses, with the intention that the teachers should change and customise them in order to investigate mathematics themselves or to build microworlds for students to use.

Figure 10

 10 Figure 10. E-Slate Fraction microworld screenshot.

  problems into algebraic expressions (Ms Lindquist) or solving linear equations and inequations and systems (T-algebra), while others seem to cover the whole curriculum (Cognitive Tutor, Math Teacher+ or MathXpert). The situation of each of the ten ILEs with respect to the algebraic domain is represented in Fig. 11.

Figure 11 .

 11 Figure 11. Situation of the studied ILEs with respect to the algebra domain covered.

Figure 12 .

 12 Figure 12. Ranking of the ILEs with respect to the number of different representation systems available for editing and manipulating of algebraic expressions.

•-

  Usage Regarding the customisation sub-direction, we have counted the number of ways in which the teacher can customize parts of the ILE, or activities with the ILE: Choose exercises, Create new exercises, Change important parameters of the ILE, Define scenario of use. It is one of the main subdirections of the Usage direction. -Regarding the use sub-direction, 1 point has been assigned for a declared use in laboratory, 2 points for a declared (experimental) use in a classroom, 3 points for a declared use in a large number of classes and 4 points for a widespread use of the ILE in a big number of classes. It is one of the main sub-directions of the Usage direction.

•-

  Control Regarding the feedback sub-direction, we have assigned 1 point to an ILE providing feedback on final expression, an additional point to an ILE providing feedback on intermediate expressions and another point to an ILE providing feedback on sub-expressions. It is one of the main subdirections of the Control direction. -Regarding the hint sub-direction, we have assigned 1 point to an ILE providing a simple help to students, an additional point to an ILE giving one step toward the solution, another point to an ILE providing guidance up to a solution and still another point if the ILE can solve the given problem. It is one of the main sub-directions of the Control direction. -Regarding the reaction sub-direction, we have assigned 1 point to an ILE providing a simple alert when an error occurs, an additional point when correction or hint is available when an error occurs and another point when a teacher can choose (by setting a corresponding parameter) to have progress blocked or not when an error occurs. It is one of the main subdirections of the Control direction. -Regarding the diagnosis sub-direction, we have assigned 1 point to an ILE providing a simple local diagnosis on the steps of the student's activity, 2 points when the diagnosis concerns a whole exercise or a sequence of exercises, an additional point when the diagnosis is global and concern the whole student's activity and another point when this diagnosis is shown and understandable to the teacher of the student. It is one of the main subdirections of the Control direction.-The same representation sub-direction as defined in the Knowledge direction. It is only a secondary sub-direction of the Control direction.

Figure 13 .

 13 Figure 13. Graphical representation by means of radars of the 3 directions -Knowledge, Usage and Control -of T-algebra.Two ways of using these radars can be imagined: (1) the three radars relative to a given ILE provide a global view of this ILE, (2) two radars on the same direction relative to two ILEs enable a comparison of these two ILEs as regards the direction at stake. For example, again considering T-Algebra Fig.11), the analysis of the three radars reveals the main characteristics of T-Algebra, the directions or domains where T-Algebra is strong, the direction or domains where the software is, comparatively, weak. It shows that T-Algebra is particularly student-friendly (with respect to learning) as it provides a relatively strong control for the students to manage their learning and solving processes and, on the other hand, it offers less possibilities to parameterise the software by teachers (weaker Usage dimension).

Figure 14 .

 14 Figure 14. Radars for the Knowledge direction for 9 ILEs.

Figure 15 .

 15 Figure 15. Radars for the Usage direction for 9 ILEs.

Figure 16 .

 16 Figure 16. Radars for the Control direction for 9 ILEs.

Figure 17

 17 Figure 17. A 3D map of the ILEs.

•

  Zone B: ILEs with tradeoffs between Control and Usage, but the control dimension is stronger. It is the most populated zone by the studied ILEs, it contains AnimalWatch, Cognitive Tutor, Math Teacher and MathXpert.

Figure 18 .

 18 Figure 18. Five zones in the 3D map of the ILEs landscape.

  personal choices, motivated by our personal perceptions of features and functionalities of an ILE for arithmetic and algebra. Numerous elements have not been studied or taken into account, such as availability, economic elements, basic user-interface importance, internationalization and national context etc. It is clear that considering other directions and features would lead to a different comparison and to a different global image of the ILE landscape. Nevertheless, we hope that the work initiated by TELMA contributes to a better understanding of the ILEs domain and opens ways to further research in this direction.

The grids filled in for the ILEs developed by TELMA teams, as well as for T-algebra, can be found in(Trgalova,

2007) and those for the other 6 ILEs developed outside TELMA in[START_REF] Nicaud | State of the art of didactical ICT systems[END_REF].

Acknowledgements

The study reported in this paper was conducted within the TELMA project, contract N° IST 507838, supported by the Network of Excellence Kaleidoscope.

is highlighted in red (CognitiveTutor or MathTeacher+), or a hint message is sent to the student (ActiveMath, T-algebra or AnimalWatch). Some ILEs provide feedback about the correct end of an exercise, either immediately (MathTeacher+) or on demand (Aplusix). In some ILEs, specific feedback aimed at scaffolding the student solving processes had been implemented, such as displaying contextsensitive menu and/or activating applicable rules in a list when a (sub) expression is selected (AriLab2 or MathXpert). MsLindquist provides a dialog form immediate feedback depending on chosen tutoring strategy.

V.2.2. Help and/or solution provided by the ILEs

In order to evaluate the degree of autonomy given to a user, hints provided by the ILEs were analyzed. Some of the ILEs provide hint messages as soon as an error is detected (ActiveMath or AnimalWatch), other provide them on demand (CognitiveTutor or MathXpert). The nature of the content of hint messages differs from one ILE to another. For instance, CognitiveTutor provides help messages at three levels, from the most general to the most concrete, leading eventually to the solution of a given task, MathTeacher+ or MathXpert provide step by step guidance toward the solution, and T-algebra suggests an action to use at each step.

The ILEs differ also in decisions whether and when to send a correct answer to a given exercise or activity. Thus, for instance, ActiveMath never provides a correct answer, AnimalWatch, CognitiveTutor, MsLindquist or MathTeacher+ send a correct answer after 3 or so mistakes, and finally T-algebra, Aplusix or MathXpert provide a correct answer only on user's demand.

V.3. Features related to technical aspects, accessibility and availability

V.3.1. Operating systems allowing the ILEs running

Two of the studied ILEs are web-based systems (ActiveMath and MsLindquist).

The others run under Windows, some of them are also supported by Macintosh operating system (AnimalWatch, CognitiveTutor, MsLindquist and MathXpert), and the latter runs also under Linux.

V.3.2. Accessibility of the ILEs

The accessibility of an ILE is guaranteed partly by its license.


Denis Bouhineau
email: denis.bouhineau@imag.fr

Ben-Manson Toussaint
email: ben-manson.toussaint@imag.fr

Vanda Luengo
email: vanda.luengo@imag.fr

Michael Ortega
email: michael.ortega@imag.fr

Nadine Mandran
email: nadine.mandran@imag.fr

Claire Wajeman
email: claire.wajeman@imag.fr

Open platform to model and capture experimental data in Technology enhanced learning systems

We are working in the context of our research team (multidisciplinary with numerous and various TEL systems), on the design and implementation of an open platform to collect, save and share experimental data drawn from the interaction with TEL systems as well as to build, save and share the analysis processes executed on these traces. From our point of view both data and analysis processes are worth to be stored and shared, and moreover have to be joined in a unique repository to get the whole picture. This communication presents the data part of the project.

An analysis of contemporary platforms shows the lack of solution for a repository of data joined to the analysis processes in the domain of TEL systems, and points out the limitations of current platform dedicated to TEL data. Hence a general, simple and customizable model of TEL systems interaction traces is proposed. It put emphasis on the users (learners, teachers) and allocates a reasonable weight to the pedagogical situation. This is implemented in a web platform, on one side connected to one (or several) TEL systems (via http protocol and javascript API) for the collection of the traces, and on the other side linked to predefined databases to save the data. Another added-value of the platform is to allow to share the data. The final part of the communication demonstrates several real use cases taken from our research team work.

INTRODUCTION

Early research in TEL have produced results related to models and systems to enhance learning but the data, the analyses and very often the systems couldn't be reused by others researchers. The effort to build experimental situations (ie real situations in different domains and different levels where data are collected) but also the research for replicability and the work of comparison between some equivalent models or systems show the necessity to share data, process treatments, analyses, results and benchmarks.

The rapid expansion of the use of technologies in supporting learning and the data produced by these usages allow two new research areas in TEL: learning analytic and educational data mining. In addition, classical researches propose processes to evaluate their models or systems. Like we will show in next section, in the last decade some platforms have appeared to share data. These platforms propose mainly repositories to share data and sometime some treatments. In some cases the data is linked to a learning paradigm, thus the data schema is also close to the paradigm. For example, if the platform analyses learner's knowledge, the log data has to contain knowledge evaluation. Others platforms propose to mutualise some kind of learning data but the treatments and the analysis are outside the platform. This kind of platform is useful to evaluate replicability but not to perform the comparison between equivalent models or systems.

Our objective is to mutualise experimental research in TEL System, i.e. to mutualise data but also the process to capture and analyze this kind of data. In this paper we will focus in the first part, i.e. specify and implement an open platform for modelling and capturing experimental data for TEL and will try to show how the others phases (treatment and analysis) have an influence in the choices to design these first part. In this paper, we consider the context of experimental research in TEL and therefore, in the following, data and processes are always mentioned in this context.

FRAMEWORK AND RELATED WORKS

This project is built in the framework of a multidisciplinary team (MeTAH1 ) who works in various scientific learning domains (mathematics, physics, biology, chemistry, medicine) and in different fields of TEL (student models, learning design, feedback models, inquired learning, didactics ...).

Traces, coming from the interaction between the users (learners but also teachers) and the systems, don't have the same schema and format. Also the process and used tools could be different [START_REF] Romero | Educational Data Mining: A Survey from 1995 to 2005[END_REF].

As our objective is to mutualise learning and teaching traces, and their process, it is necessary to design a model to collect traces in order to obtain TEL data.

Like written by Setoutti et al [START_REF] Settouti | A Trace-Based System for Technology-Enhanced Learning Systems Personalisation[END_REF], providing a single, common and extensible model conceptually compatible with common trace exploitation processes seems desirable. But they argue that designing common trace schema and format requires a standardized model with precise semantics and it could be too soon for TEL community to try it because it needs an agreement of semantic level. Some platforms propose TEL schemas to share data. For example DataShop [START_REF] Koedinger | A Data Repository for the EDM community: The PSLC DataShop[END_REF], which is an open repository of learning data, proposes a schema centred on learner transaction with a system, in particular intelligent tutor systems like cognitive tutors. "The DataShop logging model is represented by the following constructs: Context message: the student, problem, and session with the tutor; Tool message: represents an action in the tool performed by a student or tutor; Tutor message: represents a tutor's response to a student action" [START_REF] Koedinger | A Data Repository for the EDM community: The PSLC DataShop[END_REF], p5. The DataShop web application provides several tools to assist the analysis and visualisation of repository data. In order to take benefit from these platform functionalities (like for example calculate the learning curve) the trace has to contain in the tutor message the tutor evaluation (correct or incorrect) of the student behaviour.

The platform Mulce (MUltimodal contextualized Learner Corpus Exchange) [START_REF] Reffay | Multimodal learning and teaching corpora exchange: lessons learned in five years by the Mulce project[END_REF] proposes also a repository. They defined a Learning & Teaching Corpus (LETEC) as a package containing the data issued from an online course, the contextual information and metadata, necessary to make these data visible, shareable and reusable. Their objective is to better describe the context of the data. Indeed, they made an effort to describe this context (users, learning domain, technology, communication language, learning scenario, interactions phenomena) and the type of the corpus (global, distinguishable, pedagogical scenarios) and the descriptions of the analyses tools used for the data corpus. These analysis tools are described but externalised.

Both authors' platforms emphasize the necessity of making connection with analysis tools.

Our project is to mutualise, in an open and flexible platform, TEL data but also tools to process and analyse it. This choice, i.e. connect data and the TEL tools in the same platform, guides our choices for designing the platform and also the of the preliminary phases like model and capture data Also, because we are interested by various learning and teaching paradigms we would like to leave the possibility to share data and tools coming from several TEL paradigms.

TRACES FOR TEL

Whatever the situation, an exercise, a complex pedagogical situation, some student's resolution, the current state of a student's work, when designers define and build a TEL environment, they produce an effort to model these objects in order to save them, to retrieve them, to edit them. Very often this modelling process constitutes the backbone of the development of the TEL environment. Experimental activity traces (interaction with the learner, feedback from the TEL environment, feedback from diagnosis shown to the student, etc.) are different since explaining, recording or retrieving these traces is often not essential for the functioning of the TEL environment, and editing these traces does not seem legitimate at all. But if we want to work on those traces a modelling work is also required in order to retrieve, save and study them.

Why are we interested in studying these traces? In general, the experimentations are conducted to gain a better understanding of all kind of things concerned with TEL learning situations, such as learners' or teachers' behaviours, contents to be taught, or anything about the TEL environment itself. Therefore, this design work belongs to another world than the world of the TEL environment design and is concerned by experimentations with or about the TEL environment; despite there can be some common parts. In the present case, we focus on experimental activity traces looking for interactions between the user and the software; we consider the case where the user is a learner or a teacher, the software is a TEL environment, and the global context is a pedagogical situation where everything may change with time since learning is an evolutionary process. Although each one may think of a specific TEL environment and of a specific pedagogical situation, we aim to capture the fundamental elements that characterize such traces without being too restrictive.

Traces Usage

The way the traces will be used after being collected is highly important. The processes which are foreseen for the collection and processing of traces commonly have an influence on the way traces have to be concretely modelled and recorded. Following uses are to be expected:

• The TEL environment asks for elements from the traces in order to perform immediate diagnosis and then feedback, or personalization;

• The researcher asks for elements from the traces in order to explore the knowledge domain, the TEL environment, the way the learners do learn or the teachers do assess or give feedback;

• The end user (student, teacher, institution …) get value from the recording of the history from a replay system or global statistical report.

Our use cases belong mainly to the second type, but also include research of the first usage as shown in the next section.

From traces to data: lifecycle

The data lifecycle (figure 2) is described with three main parts: data production, processing and communicating. The cycle can also be divided into seven phases since the data production step can split in two phases: preparation and collection; the data processing phase can split into 4 phases: validation, enrichment, analysis and summarization. Archiving (not showed in the figure) is transversal to the whole process that runs from production to synthesis. All phases are shown in a sequential order, from the phase of experiment preparation to the phase of results synthesis.

But in most cases it is an iterative process and at any phase the experimenter may have to reconsider a previous phase. However in an ideally well designed experiment there should not be any step backward from the data processing to the preparation and collection phases. In this paper we focus on the preparation, collection and archiving phases.

In the preparation phase, an experimental protocol is designed to answer a (set of) research question(s). It defines the experimental factors and the variables to be collected. It provides their structure and format, the context and means of data collection. It also states all the data processing. It includes a test stage meant for the validation of the experimental design. At each phase of the data lifecycle, the experimental design must be amended to report about the conditions of the field experiment when the process happens to differ from the original experimental design. These additions include the report of errors and of bad field conditions.

The collection phase is performed according to the experimental design though under the field conditions. Two ways for collecting the data are considered: through the capture of TEL logs [START_REF] Romero | Educational Data Mining: A Survey from 1995 to 2005[END_REF]; or without TEL logs, consistently with the experimental design or because of bad field recording conditions. Other types of data are also commonly collected in situ: video and audio records, photographs, screenshots, paper and electronic documents, artefacts… Some other types of data are produced a-posteriori. For example, numeric time-coded annotations are produced from in situ data following an annotation grid. Transcriptions of verbalizations following a transcription grid are also a very common type of a-posteriori data in the context of TEL studies. TEL logs may be directly captured in the platform format. All other types of data have to be transferred and stored in appropriate format. The data that are collected in situ are the raw data and must be stored and preserved in their original form. Raw data are irreplaceable and unique since they are the only remains of the learning situation that have been observed. The data produced aposteriori must be stored along with the description of their production and control process.

The archiving phase occurs at any stage of the data lifecycle and it is meant to memorize the experimental part of the research process and all its products: research questions, experimental design, raw data, validated and enriched data, results, summarization of results … Archiving guaranties the reproducibility of the data and the results. It also allows to reuse and to share the data and the processes. The conditions and processes that lead to any data and results do not only contain the information that the replay of the processes. It also carries the meaning of data and results. Therefore this latter information must be explicit and it has to be reported and stored too.

Interaction traces

Case study

We present here briefly some TEL environments and experimental activity traces from the MeTAH team.

• Aplusix [START_REF] Bouhineau | Two Years Of Use Of The Aplusix System[END_REF] is about the learning of arithmetic and beginning of algebra in secondary school. The TEL environment is based on a microworld for algebra with semantic or syntactic feedbacks and expert system commands available, different modes also exist (exercise machine, software companion). Numerous and large experimentations have been conducted in ordinary classrooms during more than 5 years. The traces were saved in local computer (with ad hoc they described basically the students' actions and the system feedback.

• Copex-chimie is about the learning of experimental design for chemical experiments at college and university level. The TEL environment is based on a tutor which gives help and feedback to the students with a global diagnosis of the situation. Experiments have been conducted in ordinary classrooms. The traces were archived in a server. They are organised in a SQL data base which describes basically the interaction between the system and the student as well as the evaluation of the student behaviour in some interactions.

• EDBA [START_REF] Bouhineau | AJAX pour EIAH? Choix technologique pour un EIAH de l'algorithmique: EDBA[END_REF] is about the learning of algorithmic at college and university level. The TEL environment is based on a microworld for algorithmic linked with a large bank of exercises. Semantic feedbacks are available, with statistical and collaborative guidance. Experiments have been conducted in ordinary classrooms. The traces were saved online on a server in a SQL database and directly on the platform described here, see Table 1.

• Formid Suivi [START_REF] Guéraud | Teachers need support too: FORMID-Observer[END_REF] is a tool dedicated to the teachers which allows the teacher to follow in real time the students' activities while working on a simulator. Formid Suivi uses the data coming from the interaction between the student and the simulators. These data are specified by a scenario which defines the variables the system has to trace. The TEL system offers the teacher several ways to follow the students: a set of students, one student, one problem solved by the student. Furthermore, Formid-Suivi traces the teachers' activities, for instance the teacher' level of observation and the type of students' information that the teacher looks for.

The student data are saved on a server in a database while the teacher traces are saved in a local file (with ad hoc format).

• Elec+2 is an intelligent learning environment built in learning management system. It associates three kinds of resources [START_REF] Michelet | Adaptive learning scenarios for detection of misconceptions about electricity and remediation[END_REF]: the microworld TPElec3 , a simple text editor where the students propose hypotheses and physical laws and MCQ tests. The traces produced were analysed by DiagElec in order to give a knowledge diagnosis. The traces come from several sources with several schemas and formats (TPElec schema describes the circuit, the test editor is html form and MCQ are IMS-QTI standard). The system associates each interaction trace with the corresponding context data (user, problem, step ...). These traces are stored in a SQL database on a server.

• TELEOS 4 is about the learning of surgery (professional teaching of the gesture) at hospital university level. The TEL environment [START_REF] Luengo | Design and Implementation of a Visual and Haptic Simulator in a Platform for a TEL System in Percutaneuos Orthopedic Surgery[END_REF] is based on a haptic simulator, a gesture analysis system, an eye tracker and it delivers a cognitive feedback and global hints for the next activity (which exercise, lecture or clinical case).

Experiments have been conducted in laboratory context. The traces come from the simulator, the haptic device and the eye tracker. They are sent to a server and stored twice first as raw data and secondly as enriched data since the eye tracker data are transformed to be more semantic. For example the enriched data informs that the user saw the vertebra when she pushed the pin. The traces are stored on a server in a SQL database.

The panorama presented above shows clearly the diversity of domains, TEL environments, student's level, mode of experimentation encountered in the field and stresses the importance of a global design and platform to model and capture data from all experimental studies conducted with TEL environments.

Trace and data in TEL

Activity traces exist in other domains; the most common example is "log files", web server traces of activity that are presented in the form of a sequence of lines or records, each representing a similar request/result for a web page, using the same format (figure 1): For TEL environments the overall structure of activity traces can take a similar form: a sequence of timed lines with a finite, constant, number of fields. Each line represents a significant event for the situation, related to some interaction or use of the TEL environment. This event can be either an action or a state. An action can be a software interaction initiated by the user, the activity of a conversational agent, an automatic feedback from the TEL system, ... From a general point of view, it can be any event that, from the user's or the researcher's point of view, changes the state of the situation. it can also be the result of these actions, and then the trace is a collection of timed successive states of the TEL environment. There is a duality between actions and states. The recording of the overall state of the TEL environment in every timed line can be quite expensive. The recording of the actions is often much more economical. However, when a part of the global state is necessary, it may be associated with the recording of the action, and some specific action (or specific parameter associated with any action) can be defined for this purpose. Hence, we choose and promote as central in our log model the notion of action.

TEL environments are strongly concerned by the pedagogical situation and by the users (learner and teacher). It can be demanded that the trace integrates information about the situation and the users to each record temporally located, but it may cost a lot of space to archive it. The pedagogical situation can be set at the TEL environment initialization or when the TEL environment or the learner performs the selection of a specific situation. In the latter case, the pedagogic situation appears as the result of an action and it can be recorded as such. However it can be useful to keep some parts or all parts of the situation characteristics in every timed line. The choice to keep the information in each record can be made on the basis of cost consideration. From a global perspective, the recording of this particular information is not strictly necessary at each moment; it is the kind of information that can be reconstituted at the global level, after a full reading of the records, with a certain cost (time) for the execution of the retrieving procedure.

The information about the user (learner or teacher) has not the same status since its place in each record is necessary: even with a complete knowledge of situated actions along the timeline, it is not possible to assign an action to an individual user; at a given moment, many users may perform similar actions on the same activity during an experiment. This leads us to introduce the learner or the teacher as a first order element in our model of the trace.

Then our model of activity traces for TEL is a sequence of timed lines, each representing an action using a unique format build out of core elements:

• Date and time of the action

•

User information

• Action information

•

Context (pedagogical and other) information

The context information has been introduced to offer the researcher a free space to include all kind of required context information, for instance about the learning situation or about the state of the TEL environment. This model is the basis for what follows; it was discussed at length to be finally accepted by the entire team which has several systems and several experimental research frameworks (see previous section). It is simple enough to be accessible to experimenters with little computer or database skills and it is flexible enough to cope with most TEL environments and experiments.

This model focuses on the user (learner or teacher), raising this category as a first-class object (such as time and action) in an activity trace of TEL environments. This position distinguishes traces of TEL environment from traces without TEL perspective. The educational situation does not appear at the same level as learner or teacher, partly due to common efficiency issues in database usage. However, being aware of the educational situation importance as said previously, our model set that the educational situation can be taken as the result of an action, or can be included in the context information associated with an action.

UnderTracks platform 4.1 Data modelling

Describe the experiment

First of all, the experiment has to be described: who is doing the experiment, with which TEL environment, when, where, in which domain, with whom, etc. These are the standard metadata of any experiment.

Some more information should be provided in order to cope with the open aspect of the platform for data issued from experiments with TEL environments: are these data public or not? Who can access the traces? Which part of the traces can be seen?

The experiment can be of various natures: domain centred, TEL environment centred, user centred; strictly delimited in time and on a narrow research question or without well definite start and end and aiming at a large a-posteriori analysis of usage.

Describe the data Log, user, context and action description

Our model chosen in the previous section needs to be put into practice concretely. The work of implementation must meet several challenges: how to cope with each TEL environments and keep enough simple and effective, how to keep openness and freeness and give the ability to add specific elements for each TEL environment, how not to fall in a solution that would be too general and would redo existing things, as phpMySql (a general platform to collect and manage data) while keeping our specificity, the domain of TEL.

It was chosen to implement a warehouse, available on the web, where each experiment would be associated with some general information about owner, date, location, ... and with a simple database having four tables with the following structure (see Figure 4):

• A "log" table with a temporal field, a field identifying the user, a field identifying the action, a field identifying the context. In this minimal set a finite number of free fields can be added for additional dynamic parameters specific to each TEL or experiment, or for a particular purpose. This table is the main table of the device.

• A "user" table with a field identifying the user (primary key of the table, on which join can be done with the user field of the table "log"). With this minimal field can be stored additional fields specific to each ILE to describe the user (name, class ...)

• An "action" table with a field identifying the action (primary key of the table, on which joint can be done with the action field of the table "log"). With this minimal field can be stored free additional fields specific to each ILE to describe the action (action type, format of parameters, settings ...) Also, in each table we can choose the type and the role of the different variables. For instance, when in the preparation phase an experimental design is created with some independent variables or some experimental factors, it is possible to declare, in the platform, if the variables are dependant or independent. This information could be used during the analysis phase.

Data collection

The collect of the data can be online, dynamic, during the experiment itself, or offline, after the experimental phase. The required software and functionalities are not the same.

For Offline data could be uploaded to the platform in every table, from csv files (xls or xml file will be available in near future). Data can also be consulted freely or accessed via restricted password mechanism according to the public/private declaration made by the experimenter about these data.

Examples and first evaluation

Here we can give some results from our first uses with real systems.

Example 1. These three examples show different forms of use of the open data platform for different TEL environments. These constitute three proofs of our concept. In each case, the global model for the data has been met by the three environments to collect and store the traces of these research environments.

Discussion

The main goal for the platform is to share data as well as analysis processes. In order to be flexible, this first step (modelling and capturing traces) has been simplified with few constraints: specification of the logs based on the basic definition of traces: "trace as a sequence of observed elements". If more information is needed for envisaged treatments it is possible to describe the user (user table), give a description of the action (actions and describe the learning context (context table ).

The proofs of concept built show that it is possible to collect different TEL data coming from different systems, different learning paradigms and also from different experimentations. Indeed it is possible to create several experimentations with the same TEL system and describe a data structure tailored to each experiment.

Our hypothesis is that the association of the data and the tools for treatment and analysis carry the sense of the process. Despite the fact that some process tools can be more adapted to a kind of data, this relationship has to be supported by the treatment and analysis processes and not by the data collection phase.

Figure 2 .

 2 Figure 2. Data lifecycle.

Example of web server log lines (from apache server).

  80.191.60.84 --[18/Jul/2007:04:34:59 +0800] "GET /forum/archiver/tid-2135.html HTTP/1.1" 200 1682 127.0.0.1 --[18/Jul/2007:04:35:01 +0800] "GET /whm-server-status HTTP/1.0" 200 21102 80.191.60.84 --[18/Jul/2007:04:35:11 +0800] "GET /ss-xs/archiver/ HTTP/1.1" 404 -80.191.60.84 --[18/Jul/2007:04:36:38 +0800] "GET /forum/archiver/tid-781.html HTTP/1.1" 200 1870 Figure 1.

  online collect, during the experiment, the TEL environment and the open platform must collaborate. A javascript API have been made available for the open platform so that the TEL environment could send data to the log table dynamically during the experiment via http protocol. A java API and a C++ API should be available in a near future. We are thinking about similar API for the other tables and for other types of requests (select, update) but the need for such access to the other tables is not yet clear in our context. Other types of requests introduce difficulties concerning security and privacy. Other tables are more or less static and can be accessed and modified offline.

Structure of the database to collect the traces

  Table log was the only one used by EDBA (Tab. 1).The experiment is focused on the global aspect of the TEL environment (use of the different menu of the TEL environment) and on user's track (exercises attempted, success on test). An additional field has been used in the log table to describe the parameters dynamically selected for each action or result of tests. As there is no more information given in the action table associated to each action or result, the analysis of that additional field has to be done contextually (with respect to the action) by a program aware of every action or result which may occur in the log.

	Figure 4. timestamp	action	user params
	2011-10-26 19:36:27	IHM	2	lancementCommunAuChargement([ob ject Event])
	2011-10-26 19:37:20	IHM	2	montreEnBasAgranditLeHautSiNecess aire(idDivOutils)
	2011-10-26 19:38:03	Test	2	4241::47340::206::gris::1::4::1
	2011-10-26 19:38:15	IHM	2	montreTraces()

Table 1 : Log table for EDBA

 1

	timestamp	iduser action type_action params
	12/01/10 18:58:24	733	1	connexion	acces tuteur illimite
	12/01/10 18:59:04	694	3	consult_cs	/
	12/01/10 19:07:24	733	5	ajout_produit E124 /
	12/01/10 19:07:24	733	8	ajout_action	5/fiole jaugée/E124/

Table 2 : Log table for Copex-Chime Example

 2 2. Copex-Chimie experimentation (Tab 2) uses 3 tables: log, action and user with additional field for a better description of the type of the action (the action lead to a global category) and for parameters dynamically selected for each action. The action table gives the format of each parameter with respect to the action recorded in the log table. The information collected by Copex is semantically rich, associated with chemistry.Example 3. Teleos experience has the log table (see Tab.3) with additional fields which inform about the source (eye tracker, simulator, haptic) and numerical parameters. The format of these parameters was described in the table. For example, for the action "Action_EyeTracker" the parameters are X; Y; MeanRadius; MaxRadius; Duration. The raw data collected by Teleos are in part semantically poor (quite only numeric) but very numerous (eye tracks traces, haptic traces). The data on the "xp" field identify (key joint) the information of the context table. The context table describes the characteristic of the problem (kind of vertebra, devices used -haptic or not-, learning variables ...).

https://metah.imag.fr/?lang=en

https:// http://testbeda0.imag.fr

https://TpElec.imag.fr

http://TELEOS.imag.fr

In order to help this matching, one perspective is to define some ontology of the TEL traces and of the process tools used to analyse the data. This ontology can assist the association between the data and the process tools for treatment and analysis. It could for instance support the scientific analysis of results if it carries semantic information about both the data and the process tools and check the coherence. However the use of the ontology should not be mandatory for the researcher who collects data and build a process for an analysis.