Ibtissem Bennacer
email: ibtissem.bennacer@univ-lemans.fr

Remi Venant
email: remi.venant@univ-lemans.fr

Sebastien Iksal
email: sebastien.iksal@univ-lemans.fr

A self-assessment tool for teachers to improve their LMS skills based on Teaching Analytics

Keywords: Teaching Analytics, Learning Management System, Self-assessment, Peer Recommendation, Clustering Analysis, Principal Component Analysis

While learning management systems have spread for the last decades, many teachers still struggle to fully operate an LMS within their teaching, beyond its role of a simple resources repository. Moreover, there is still a lack of work in the literature to help teachers engage as learners of their own environment and improve their techno-pedagogical skills.Therefore, we suggest a web environment based on teaching analytics to provide teachers with self and social awareness of their own practices on the LMS. This article focuses on the behavioral model we designed on the strength of (i) a qualitative analysis from interviews we had with several pedagogical engineers and (ii) a quantitative analysis we conducted on teachers' activities on the University's LMS. This model describes teachers' practices through six major explainable axes: evaluation, reflection, communication, resources, collaboration as well as interactivity and gamification. It can be used to detect particular teachers who may be in need of specific individual support or conversely, experts of a particular usage of the LMS who could bring constructive criticism for its improvement. While instrumented in our environment, this model enables supplying teachers with self-assessment, automatic feedback and peer recommendations in order to encourage them to improve their skills with the LMS.

INTRODUCTION

The trend of using Learning Management Systems (LMS) is now spreading quickly across all areas of education [START_REF] Setiawan | Trend of learning management system (lms) platforms for science education before-after covid-19 pandemic[END_REF], with an acceleration observed during the COVID-19 health situations last year. An LMS is a digital learning platform for deploying and monitoring online training, managing courses and learners, and collecting user traces for reengineering [START_REF] Setiawan | Trend of learning management system (lms) platforms for science education before-after covid-19 pandemic[END_REF]. Most universities offer LMSs as a "one size fits all" technology solution for all teachers of any discipline. Despite the growing trend for LMS to facilitate educational activities, the number of teachers using it is not increasing as quickly as one might have imagined [START_REF] Wang | An empirical study of instructor adoption of web-based learning systems[END_REF], and many teachers face several difficulties to integrate these platforms into their practices [START_REF] Setiawan | Trend of learning management system (lms) platforms for science education before-after covid-19 pandemic[END_REF]. The main problems of teachers appear to be technical or organizational, due to the lack of support and the lack of time devoted to its learning [START_REF] Nashed | Educational resources recommender system for teachers: Why and how?[END_REF][START_REF] Dhahri | Teachers' information and communication technology (ict) assessment tools: A review[END_REF]. Furthermore, most universities are hiring pedagogical engineers (PE), especially to support and train teachers in order to ensure a proper use of their LMS and ensure their pedagogical fit. With few PE compared to teachers [START_REF] Daele | La formation en ligne. les conseillers et ingénieurs pédagogiques. Distances et médiations des savoirs[END_REF], the former struggle to support every teacher. For instance, in France, these problems were one of the reasons that led the Ministry of Higher Education to launch the HyPE-13 project1 (Hybridizing and Sharing Teachings) in November 2020. Carried by a consortium of 12 french universities, it aims to accompany teachers and students towards success with new learning devices promoting the hybridization of training.

On the other hand, the use of LMS allows the capture of large amounts of quantitative data concerning the behavior of users and designers, and thus paves the way for Learning and Teaching Analytics (LA, TA). Learning Analytics relates to the collection and exploitation of traces left by learners to enhance the learning process. Teaching Analytics, which are not explored as much as the former, refer to methods and tools to help teachers analyzing and improving their pedagogical designs, and more recently, to analyze how teachers deliver their lessons [START_REF] Sergis | Teaching and learning analytics to support teacher inquiry: A systematic literature review. Learning analytics: Fundaments, applications, and trends[END_REF][START_REF] Albó | Concept-level design analytics for blended courses[END_REF]. Hence, we consider Teaching Analytics as a challenging field of research that may have a great impact on teaching methods as well as the way in which courses are delivered to the students. In our University, the LMS has been in place for more than 10 years, and is considered nowadays as a critical service heavily promoted to teachers. However, the University is facing the same issues we identified previously (LMS use expectations are not met and only 5 pedagogical engineers have to deal with more than 600 teachers). Our main objective is then to provide teachers with personal and social awareness, in order for them to engage in learning situations that aim at improving their LMS skills.

To do so, we propose the design and instrumentalisation of a teachers' behavior model to support their self-assessment and leverage peer-learning through automatic recommendations. We address here two first research questions: (i) How to model the exploitation of an LMS a teacher does and could do in an intelligible way ? (ii) What TA indicators can be propose from this model to support self-assessment and enable feedback and recommendations?

The present paper details the teacher model we designed from the analysis of a mixed study in order to depict their behaviors on the platform, and exposes a first instrumentalisation based on several TA metrics we defined and implemented into a web-based environment. The next section is dedicated to the related work on teachers' behavior on an LMS, whose limits led us to consider a mixed study that we define and expose its results obtained from the qualitative and quantitative analysis in section 3. From these results we describe then our model, its instrumentalisation and present a first prototype to address these metrics to teachers and PE in section 4. We discuss our model limitations and potential bias as well as the perspectives we consider in the last section.

RELATED WORK

Many efforts have been made to understand student behaviors on LMS, but there seems to be a lack of work that aims at analyzing teacher's behavior in such platforms [START_REF] Albó | Concept-level design analytics for blended courses[END_REF]. Nevertheless, some researchers were interested in Teaching Analytics to understand how teachers deliver their lessons. For instance, [START_REF] Sampson | Teaching and learning analytics to support teacher inquiry[END_REF] proposed the concept of Teaching and Learning Analytics as a synergy between Teaching Analytics and Learning Analytics in order to support the process of teacher inquiry. The latter is defined by [START_REF] Avramides | A method for teacher inquiry in crosscurricular projects: Lessons from a case study[END_REF] as a set of actions in which "teachers identify questions for investigation in their practice and then design a process for collecting evidence about student learning that informs their subsequent educational designs". The use of this process of teacher inquiry is possible in the case of our work, however the identification of questions regarding teachers' practices is not trivial and could be accompanied to encourage and help them adopt the approach.

On the other hand, (Ndukwe and Daniel, 2020) proposed a Teaching Outcomes Model (TOM) that aims to provide teachers with guidance on how to engage and reflect on teaching data. TOM is a TA life cycle that begins with the data collection stage where teaching data are extracted and collected. Then follows the data analysis stage by applying different machine learning (ML) techniques to discover hidden patterns. After that comes the data visualization stage, where data is presented in the form of a Teaching Analytics Dashboard (TAD) for the teacher. Finally, the action phase where actions are implemented by teachers to improve their pedagogical practices.The TOM model is interesting when analyzing teacher behavior, but it seems to be limited to only quantitative analyses that may yield inconclusive or incomplete findings without a qualitative study.

A TAD is a category of dashboard for teachers that holds a unique role and value. It could allow teachers to access student learning in almost real-time and scalable manner, therefore, allowing teachers to improve their self-awareness by monitoring and observing student activities. It also tracks teachers' personal activities, as well as students' feedback on their teaching practice. For example, [START_REF] Barmaki | Providing real-time feedback for student teachers in a virtual rehearsal environment[END_REF] explored a TAD that provides automated realtime feedback based on speaker posture to help teachers perform classroom management and content delivery skills. They used different types of multimodal data, including talk-time and nonverbal behaviors of the virtual students, captured in log files; talk time and full body tracking data of the participant; and video recording of the virtual classroom with the participant. For feedback, a visual indication was used whenever the participant exhibited a closed, defensive posture. Furthermore, [START_REF] Prieto | Teaching analytics: towards automatic extraction of orchestration graphs using wearable sensors[END_REF] used TA to automatically extract teaching actions during faceto-face classrooms (explanation, monitoring, testing, etc.). They used data collected from multiple wearable sensors (including accelerometers, EEG or eyetrackers) and explored ML techniques to characterize what teachers really do during their courses. This study allowed to automatically detect the teacher's activity (explanation, monitoring, questioning ...), and to distinguish between the moment when the teacher interacts individually, in small groups, or with the whole class. The studies of [START_REF] Barmaki | Providing real-time feedback for student teachers in a virtual rehearsal environment[END_REF] and [START_REF] Prieto | Teaching analytics: towards automatic extraction of orchestration graphs using wearable sensors[END_REF] effectively allowed to explore teachers' behavior and provide them with useful feedback. However, they only used multimodal data without investigating teachers' traces on LMS, which makes their work beyond our research scope.

Some studies have been conducted to explore teacher behavior in hybrid learning (HL) systems. From that perspective, [START_REF] Coomey | Online learning: it is all about dialogue, involvement, support and control-according to the research[END_REF] propose a theoretical model (DISC) that identifies four main characteristics of e-learning which are considered essential to good practice: Dialogue, Involvement, Support and Control. Thereafter, they proposed four paradigms according to variations in locus of control (teachers or students) and task specification (strictly specified or open) with a list of advice for each paradigm. On the other hand, [START_REF] Peraya | Apprendre en présence et à distance: une définition des dispositifs hybrides[END_REF] built an empirical framework based on 5 main dimensions to describe how teachers cope with techno-pedagogical environment in HL setting. These dimensions include modalities of articulation of faceto-face and distant phase, human support, forms of media, mediation and degree of openness. While the last two works propose models to study teacher behavior, they are contextualized for HL systems (remote/ face-to-face learning), and are not appropriate to analyse the use of an LMS used mostly as a complement to face-to-face learning. In this context, other researchers were interested in analyzing the behavior of teachers. For example, [START_REF] Whitmer | Patterns in blackboard learn tool use: Five course design archetypes[END_REF] aimed to uncover archetypes of course design across multiple institutions. To this end, they performed a clustering analysis and identified five different groups consider courses with mainly content and low interactions, with one-way communication, with strong peer interactions, courses more oriented to evaluation and eventually those with a balance between content, communication and evaluation. Moreover, [START_REF] Regueras | Clustering analysis for automatic certification of lms strategies in a university virtual campus[END_REF] proposed a method to automatically certify teachers' competencies from LMS data to help universities make strategic decisions. Three clustering methods were applied, and they were able to identify 6 types of courses (non active, submission, deposit, communicative, evaluative, balance). To enable teachers to measure and evaluate their courses, [START_REF] Valsamidis | A clustering methodology of web log data for learning management systems[END_REF] used Markov Clustering and Kmeans algorithms to analyze LMS courses and student activity, then computed metrics based on the number of sessions and page views per user. While the latter allowed for a preliminary ranking of courses, they are only based on students actions and thus do not related to the activities the teacher perform on the LMS.

Overall, these analysis of teachers' actions target various purposes: some studies attempt to cate-gorize courses, to profile teachers or to analyze the overall use of LMS, while others aimed at automatically certifying teachers or to evaluate and measure their courses performance. However, it appears that none of them have targeted the modelisation of the teacher's behavior for its application to selfassessment. Such modelisations give proper insights to what is currently done on the LMS, and may be used to compare a teacher to another, but present common limits. Indeed, they use student-related data that are difficult to compare since it is not the same population, nor the same number of students, etc. In addition, the empirical models we reviewed depict current platform usage, with the rejection of unused variables and cannot adapt to future use that are expected. We suggest then to design a model from both data mining and expert knowledge when expectations involve LMS use that are not observed yet. Furthermore, the models proposed in the literature depend on the data within the LMS used by the researchers. However features used by teachers and their behavior change from one LMS to another, which requires us to create a new model.

TEACHER BEHAVIOR ANALYSIS

Methodology

In order to qualify the current and expected teachers' uses of the LMS, we applied a quantitatively driven mixed method [START_REF] Johnson | Toward a definition of mixed methods research[END_REF]. We started applying a quantitative analysis to deduce statistically different profiles of LMS use, in order to find groups of teachers or profiles of interest, based on the LMS log data. We performed a Principal Component Analysis (PCA) and a clustering analysis. PCA analysis allows to highlight diversity of the dataset in a reduced set of variables (components) while the clustering one aims to regroup the different instances of the dataset regarding their similarity. Afterwards, we conducted semi-structured interviews (i.e. : qualitative interview) with pedagogical engineers. In a series of open-ended questions prepared in advance to guide our interview, we collected information to improve the quantitative study. This qualitative method was chosen because we needed the interviewee to answer freely, express a specific point of view, and bring out potential new working hypotheses [START_REF] Magaldi | Semi-structured interviews[END_REF]. We performed then a second quantitative analysis using the same previous method to address the engineer's comments by adding or modifying some variables. In order to design a behav-ior model that can handle both present and future expected usages of the LMS, we merge both results we obtained from this latest analysis and those we obtained from the interviews. Particularly, some of the discussed LMS features are still not used enough to appear in the results of the quantitative analysis. Moreover, the choice of the model axes (i.e.: the structure, how variables are grouped by axis) is also made from the results of the last PCA analysis, and modified thanks to the qualitative interviews.

Finally, this model allowed to design several TA metrics. We applied clustering methods to be able to provide a social awareness-based indicator and then defined interpretable scores to offer more detailed personal awareness. In parallel to that, we created a questionnaire of teachers on LMS habits in order to (i) validate our needs and the interest of our work, and (ii) have directions on the functionalities of the application we will develop for the purpose of exploiting the results of our model and metrics.

Based on the TA metrics and the questionnaire, we eventually designed a tool mainly dedicated to teachers but also to the university's pedagogical engineers. It supports self-assessment and awareness, and can also provide automatic peer recommendations using our model and metrics.

Qualitative study

We chose to conduct interviews with pedagogical engineers because they are always in contact with teachers to help them use the University's LMS, so they have a global insight into the practices used by teachers and the problems they encounter when using the platform. In addition, with the transition to fully online teaching due to COVID19, it was difficult to contact teachers due to their charges unlike PE. Therefore, the interviews were conducted separately with 3 female engineers on the same day and each lasted 40 to 50 minutes. All interviews were tape-recorded, transcribed, and analyzed by 2 researchers who compared the different responses by grouping similar ones and detecting particular cases.

Prior to the interviews, we prepared the interview guide that includes the different questions, classified according to their themes: introduction (mutual presentation, research objectives, PE biographies and competencies); implementation of pedagogical scenarios on LMS (method used to implement teachers' practices); use of the LMS by teachers (PE' perception of the teachers' use, difficulties encountered by teachers for the implementation of their practices, typical teachers' profiles observed, suggested indicators to define and detect these profiles); evaluation of the variables used in the first analysis (opinion about the variables used in the first analysis, discussion about other variables that might be relevant); evaluation of the groups of teachers obtained (consistency of the identified groups, and usability of the model); tool and expectations (the vision PE have of an application for them and expectations for the further development of the research project).

Throughout the interviews, no contradictory statements were detected, and there was a consensus on most of the conclusions. For the implementation of pedagogical scenarios, they mentioned not using any predefined formalism but rather adapt to the teacher's choice. Regarding LMS usage, they indicated the LMS of the University is underutilized to its potential. One engineer specified that its use is mainly in science faculty with people who are "not afraid of computers" and that this use is very variable from one teacher to another. The difficulties experienced by these engineers are considered mainly due to insufficient knowledge of the platform and to the lack of time for learning. Another engineer added that teachers only see the LMS as a computer tool, which prevents them from improving. According to them, the different activities used in the LMS are resource repository, communication, evaluation, and feedback. More recently they have noticed a demand for more fun and attractive activities. Then, they proposed some indicators to assess these profiles which revolve around activities' frequency of consultation by students, the use of links, individual or collective resources and quizzes.

With respect to the first analysis we have done (detailed in the following section), they encouraged us to correct some variables calculation. For instance, while we used the resource "url" proposed in the LMS to compute the number of external references a course may do, PE explained that many reference to external content were directly written in the content of labels or section summaries. Furthermore, they suggested adding some activities that were not collected at the time such as game-type ones. They also emphasized the importance of including feedback that was unfortunately removed during the preprocessing phase due to its low variance. On the other hand, they expressed, once they saw the teacher groups, their interest in getting to know the very active teachers. They were actually eager to invite them to have a discussion and get their feedback. At the end, they described their needs regarding the exploitation of our results. It consists mainly in the necessity to have elements to better support teachers without being drowned in a mass of numbers. Furthermore, they would like to be able to have insights on how good the course spaces are to engage students in learning, un-derstand why and visualize the results by department and by discipline. On the COVID part, they were curious to see the increase in demand for the LMS.

Quantitative study

The LMS adopted by our University is used by most teachers and students. We recovered traces of teachers' activities from June 2016 to July 2018 and from October 2019 to November 2020 (an IT failure on LMS caused the loss of data between the two periods). We present here the study we made following the qualitative study, which takes into account the PE' remarks. Data were preprocessed from the Moodle Database and the LDAP server to store them into a Learning Record Store following the xApi2 convention: each action is represented as a standalone document that provides a view of the related course and activity (if any) and details of the user at the time of the action. From that LRS, 30 variables have been identified to analyse the teachers' behavior. 974 teachers did at least one action related to these features.

We started the preprocessing phase with removing variables with low variances. We tested multiple values and ultimately set a threshold that allowed us to keep 15 variables with variance greater than 0.4. The second step aimed to eliminate "ghost teachers" who are considered as course editors but have only performed very few actions on the course. Therefore, we calculated the number of non-zero variables for each teacher, and we found that most of them (half of the teachers -487 teachers-represented by the red line in figure 1) have at least 9 non-zero variables, hence we eliminated those who have more than 6 null values, and retained the 585 teachers left. Afterwards, we removed variables highly correlated to each other (i.e.: a Pearson correlation coefficient > 0.8 with a p-value < 0.005 after having applied a Bonferroni correction). Two variables appeared to be redundant: label and forum discussion as they were respectively correlated to the overall number of links to external resources (r = 0.89) and the number of forum posts(r = 0.87). The final dataset is eventually composed of 585 instances and 13 variables described in table 1.

We conducted a PCA analysis to detect typologies of LMS uses by teachers. Using the criterion of eigenvalue [START_REF] Tamura | A study on the number of principal components and sensitivity of fault detection using pca[END_REF], the best model includes 5 components that explain 72% of the total inertia (the information contained in the dataset). The rationale for using the eigenvalue criterion is that each component must explain the value of at least one variable, and therefore it indicates that only components with eigenvalues greater than 1 should be retained.

Teacher Questionnaire on LMS habits

In order to validate our needs and evaluate the extent to which teachers would be willing to use our tool, we have developed a web survey intended for them. There were four sections in the survey. The first one was on general questions that were used to capture contextual factors which characterize the teacher (university site, gender, age, department and specialty) as well as the number of courses taught and professional experience. The second section was a system usability scale (SUS), which is a standardized survey with Likert scale questionnaires designed to be both simple and quick. It consists of 10 questions and aims to determine the level of satisfaction experienced by users of a service or system. In our case, we aim to examine the effectiveness, efficiency, and satisfaction of teachers in using the institutional LMS, which will allow us to more easily determine those teachers who use the platform and those who do not. Thereafter, teachers who are satisfied with the platform can provide us with information in the next two sections about the problems they face, how they use the platform and what can motivate them to improve their uses. On the other hand, dissatisfied teachers can give reasons why they do not use the LMS and if there is a way to motivate them to use the platform in the future. The third section is devoted to functionalities and ease of use, so that teachers can explain the difficulties they encounter when setting up their courses on the LMS in order to (i) validate the answers of the pedagogical engineers during the qualitative study, (ii) detect other problems for which solutions might be found. Furthermore, some questions are intended to collect the functionalities of the platform most used by the teachers to validate the quantitative study and others to determine the curiosity of teachers to explore more features of the LMS and whether they would be willing to help each other (my colleagues encourage me to use LMS). The latter allows us to study the subjective norm which is a very important criterion in the TAM (Technology acceptance model) which is the most used model in the studies of user acceptance of different technologies [START_REF] Yuen | Exploring teacher acceptance of e-learning technology[END_REF]. It is defined by the social pressure exerted by directors and other teachers on teachers to use the systems (Cigdem and Topcu, 2015) and it allows, in our case, to enrich the peer recommendations and the TA metrics that we propose. The last section of our survey is about confidentiality and teacher expectations to see if they would be interested in our awareness tool, and if they would provide support to each other, so that we can be prepared and make possible modifications to the tool before the experimentation. At the time of writing this article, we have received 43 answers. Regarding the general information of the respondents, there was almost an equality in terms of gender with 51.2% female and 48.8% male. The majority of the respondents' ages were in the range of 35-50 years old (which was 53.5% of respondents). 69.8% of the respondents were from site 2 while the rest from site 1. The largest number of the respondents belonged to a technical department (32.5% Science and technology department and 30.2% university technical institute departments) while 11.6% were from Letters, Languages, Human Sciences department. Therefore, computer science was the speciality of 16.2% of respondents, 9.2% were acoustician whereas physics, chemistry, biology were the specialty of 12% of teachers with 4% each. For the number of years of experience, 48.8% of teachers had between 10 and 20 years of experience, and 83.6% taught between 1 and 10 courses with only one teacher who taught a single course.

The results of the SUS questionnaire allowed us to construct the satisfaction score. This score is between 1 and 100. A score is generally considered "good" from 75, fair or correct between 50 and 75. A score below 50 indicates major problems in terms of customer satisfaction. So, according to the teachers' answers, we have just 3 teachers who are not satisfied of the University's LMS, 28 who find the platform quite satisfactory and 12 who have shown their high satis-faction of the LMS, as shown in the figure 2 . Regarding the use of the platform, the figure 3 shows that most teachers (36 respondents strongly agreed and 6 agreed) frequently use the LMS resources. Then comes the evaluation features with 11 respondents strongly agreeing and 17 mostly agreeing. On the other hand, gamification and collaboration features are apparently not used very much by teachers who responded with 27 and 24 disagreeing respectively. The use of feedback and features that allow to get students feedback on the courses is also not very explored by the teachers with 17 disagreeing and 6 agreeing. Whereas, the communication features are fairly used with 17 not agreeing and 12 agreeing. Some teachers mentioned the use of other features such as activity completion and group selection that will be considered shortly to improve our model. 58.13% of respondents expressed their intention to discover new features on the LMS in order to improve their teaching. 18.6% mentioned that they do not receive encouragement from their colleagues to use LMS, however 41.8% contradicted this statement.

Regarding the capture of teachers' traces on LMS, at this moment, 17 teachers are against, 18 are for and 8 are neutral. We then asked teachers who they would ask for help if they encountered problems on the LMS. The answers are presented in the table 2, it clearly shows that they prefer pedagogical engineers or a close colleagues, which validates the recommendations that we intend to implement. Other teachers mentioned LMS assistance or internet instead of asking for help. On the other hand, 86% of respondents were ready to help their colleagues if they asked.

Finally, we asked if teachers would be interested in a complementary tool to the LMS to improve their practices, 37.2% wanted one to get recommendations from peers, 20.9% to get feedback on their use of the platform and 16.2% to evaluate themselves. We have left the question open for further proposals, so one teacher mentioned that he prefers training on more times, two other teachers proposed tutorials for certain functionalities or a guide of good practices and what they can do on the LMS. It should be mentioned that 6 respondents did not want any complementary tools to the LMS since they are satisfied with their use of the platform (they all have a SUS score higher than 50). These responses assess the need to provide a support tool as a significant portion of the teachers are interested in having it and a large portion of the teachers would like to have recommendations from close colleagues and pedagogical engineers.

Choice

Number of respondents a close colleague 28 a teacher at the university 7 a pedagogical engineer 32 I do not wish to ask for help 1 other 5

MODEL DEFINITION AND EXPLOITATION 4.1 Model

Through the intersection of the qualitative and quantitative studies, we designed a teacher behavioral model. It describes within six axes the behavior of teachers in a comprehensive way, with respect to previously discovered components from the PCA analysis, and to the results of the semi-structured interviews we had with the engineers. The objective of this model is to offer a self-assessment tool to the teachers on several dimensions, so it allows them to evaluate themselves according to 6 axes and thus they can detect their weaknesses and strengths on the use of the LMS. On the other hand, this model includes features that can be used to represent the current situation, and features that represent a usage currently low or null, but that may be of importance in the future.

A.1 Evaluation: this axis represents the different tools used by the teacher to assess their students. It reflects the second component of the PCA and with respect to the results of the qualitative analysis, and aims at evaluating how the teacher benefits from the digital environment to organise and implement students' assessment. It includes obviously the quiz and assignment variables that provide different ways to assess students and provide them with formative feedback, grade to provide summative feedback and calendar for organization (e.g.: deadlines settings). The last variable used is the attendance (num. of "attendance" activities manipulated by a teacher). Unrevealed with the quantitative analysis, it highlights how the LMS of the University is used to evaluate students' presence in the course.

A.2 Reflection: it concerns the LMS features that can provide teachers with a way to get feedback from students on their teaching and the digital resources they use. Both variables survey and choice (number of respectively "survey" and "choice" activities edited by the teacher) reveal this particular exploitation of the LMS. So far they are used marginally, and do not appear but they must be taken into account as reflection has been considered as an important axis of evaluation in the interviews we had with PE.

A.3 Communication: this axis is devoted to the different means of communication used by teachers to facilitate the transfer of information to the students and to improve the sharing between them. It includes forum and chat related variables (forum, forum discussion, forum posts, chat and chat messages). It also brings together the third and the fourth component of the PCA (comp 3 and comp 4).

A.4 Resources: this axis refers to the diversity of resources the teacher provides to students, and include then the file, book, folder, page, glossary and url variables. Based on the comp5 of the PCA analysis, other variables mentioned were added thanks to interviews with PE.

A.5 Collaboration: this axis concerns the promotion of collaboration between students with different LMS features. It includes the workshop, wiki, via, choice et data variable, identified mostly by the qualitative analysis, that all refer to the teacher's manipulation of these features. The workshop functionality allows for the collection, review and peer evaluation of student work. The wiki allows participants to add and edit a collection of web pages. The Via feature allows the creation of synchronous meetings in a virtual classroom. Lastly, data allows participants to create, maintain and search a collection of entries (i.e. records).

A.6 Interactivity and Gamification: this last axis gathers the interactive or playful activities used by teachers to animate their courses and make them more attractive. Also identified on the basis of qualitative analysis, and not revealed by the quantitative analysis so far, lesson, course format, img, gallery, game, lti, refer all to different activities that raise interactivity or gamification. While lessons introduce personalization of the sequences based on student's inputs, galleries allow to expose collections of pictures interactively with the possibility to comment on them, and lti allows to include external activities using the LTI protocol. Eventually, we perceived the modification of the course format itself as evidence of a reflection a teacher can have on the interactions students will have with the course.

Teaching Analytics indicators

Based on the teachers' behavior model derived from the quantitative and qualitative analysis, we designed three TA metrics for awareness and self-assessment.

a) LMS usage trends

The model we designed allows to describe how teachers master the LMS through different pedagogical axes. In order to determine a TA indicator to support social awareness, we decided first to provide teachers with a current view of their position relative to the others, with respect to the different axes. We propose here and for each axis a clustering model in order to distinguish groups of teachers based on their current behavior. Thus for each of the axes, we applied the same preprocessing steps we used in our quantitative analysis, which consists of filtering variables that would decrease the model performance due to their low variances or their high correlation with each other. Based on that cleaned dataset, we tested several clustering algorithms (K-Means, Dbscan, Agglomerative clustering and Gaussian Mixture).

To set the best number of clusters for each model we relied on the silhouette score S: the mean of all silhouette scores for each sample that range from -1 (worst) to +1 (best), where a high value indicates that the sample (teacher) is well matched to its own cluster and poorly matched to neighboring clusters. We then retained the best model with regard to its mean silhouette score and the consistency of its clusters (and outliers for Dbscan). The results are exposed in Table 3, with S the mean silhouette score, N the number of clusters and O the number of outliers for Dbscan.

For each axis, the models converge towards a detection of particular teachers (active teachers and nonactive teachers), and not towards a regular or homogeneous classification. This result is consistent with other studies in literacy so far [START_REF] Park | Using log variables in a learning management system to evaluate learning activity using the lens of activity theory[END_REF]. The second axis "Reflection", initially characterized by feedback and choice, contains only one feature that is choice, because feedback was removed in the preprocessing phase due to its low variance. This explains the number of clusters obtained by the four algorithms that have classified the use of choices by teachers from most to least active. On the other hand, the models of the remaining axes consistently returned two clusters that separate the most active teachers from those who are not or faintly active. For instance, after the analysis of the evaluation axis, Dbscan gave the best results with a group of teachers that use the evaluation tools minimally and three particular teachers that use most of these tools in a homogeneous and intensive way. The best silhouette scores are obtained by the Dbscan algorithm, except for the second axis which was Kmeans. We therefore chose the DBscan algorithm because it is capable of detecting specific instances of the platform usage (teacher groups and outliers). This first metric "LMS usage trends" enables us to detect groups and special instances (outliers) on the different axes of our model, hence allowing the teacher to identify the axes on which he/she is active and those on which he/she is not.

b) Usage scores

The previous metric gives an insight about the present degree of teachers' mastery with respect to the group. However, the clustering method discards some of the features (due to the required preprocess-ing steps), and only provides an overall view of the skills related to other teachers. Here we propose two complementary indicators for self-awareness to measure how the teacher profits from the LMS, based on the complete model we designed. The following usage scores complete the clustering method limits by refining the teachers' self-assessment and allowing a better exploitation of our model.

Curiosity score: this score indicates the teacher's degree of curiosity according to each axis. Counting the number of non null variables over all the teacher's courses, it aims to encourage to discover other LMS features within the axis. This score is formalized by the next equation:

curiosity a (t) = |{ C t ∑ c=1 x t,i,c , x t,i > 0 ∀i ∈ [1; m a ]}| (1)
With x t,i,c the value of the feature i ∈ [1, m a ] (m a the num. of features for the axis a) for the teacher t in the course c ∈ [1,C t ] and C t the number of courses where the teacher t has at least one non null variable.

Regularity score: this score considers how often a teacher exploit the features related to an axis with respect to their courses. In other terms, it helps validating a skill based on the repetition of practice. It is calculated by the following formula (using the previous symbols):

regularity a (t) = ∑ m a i=1 |{x t,i,c , x t,i,c > 0 ∀c ∈ [1;C t ]}| m a •C t (2)

Application

We started the development of a tool to engage teachers into learning situations regarding the different axes of our model especially since many teachers wanted such a tool to help them use the university's LMS as revealed by the results of the questionnaire on LMS habits. The main dashboard of teachers is represented in Figure 4. Once logged, the teacher can have an overview of his/her situation. Each axis is detailed within a card in section A of the figure, with a different background color and subtitle whether the teacher was clustered as active or inactive, in other words, it represents the trend of LMS use by the teacher. The green color for the axes where the teacher has a great tendency to use the functionalities of the platform represented by the axis in question, and the red color for the reverse case. For each card, the two different scores of curiosity and regularity are included with values in percentage in order to facilitate teachers' self-assessment and comparison with scores from other axes. A description of the axis, the definition of the scores metrics and details about the clustering are also provided in details for each axis. In section B of the figure, we provide a radar visualisation that sums up the two scores for the teacher to have a quick comparative view of the different axes. This allows teachers to easily see which scores have similar values or if there are outliers among each score. Radar charts are also useful for seeing which variables score high or low, making them ideal for displaying performance. Moreover, according to the different metrics, our system can provide several automatic recommendations to improve the teacher's skill (section C in the figure) and following the teachers' answers to our questionnaires, most of them wanted to contact their close colleagues or pedagogical engineers in case of need. Therefore, when the teacher obtains low scores or is clustered as inactive in an axis, if an active peer exists with better metric values for that axis, the system invites the current user to contact this peer, giving a sample of one of his/her courses selected as a relevant example. Proximity between teachers will also be taken into consideration when recommending to ensure that a close colleague is suggested to each teacher. If no peers can be found, the system uses a fallback and recommends to participate in an open course the PE designed in relation to the axis, or to contact them directly. This support tool will be also addressed to PE, to help them detect cases of importance. The figure 5 represents the pedagogical engineers' dashboard. At the top, a data table is provided to visualize the list of teachers with their information (name, first name, specialty and service) and a column to display the results of the evaluation of each teacher according to the axes. This allows pedagogical engineers to have a global view on the use of the platform by each teacher. At the bottom right, a radar visualization shows the average of the two scores (curiosity and regularity) by axis. On the left, a bar chart summarizes the average number of active/inactive teachers by axis as well. The data in these 3 elements (table, radar, bar chart) depends on the filter at the top of the page that allows PEs to select teachers according to their specialties or departments to which they are assigned, which makes it easier for them to interpret the teachers' results. The different TA metrics we propose can thereby be used to detect teachers in particular needs for a certain axis, in order to propose them consistent and precise help. On the other hand, expert teachers in particular domains of competencies can also be identified, a wish PE have as they are also looking for these profiles to obtain precise feedback on their LMS in order to define its functional evolution, and to better organize tutoring for newcomers.

Limitations and potential flaws

Our behavior model is based on the results of both qualitative and quantitative analysis we carried out. While this model allows to describe in an intelligible way teachers' activities on the LMS and appears consistent with both current usage we can observe through traces and human expert knowledge, these analyses still have several limitations. We have integrated all teacher traces on the University's LMS to analyze their behavior, but many teachers use other technologies to manage their teaching, whom we do not have access to. Moreover, our study does not take into account what happens in a class, outside the technological environment, thus two different courses may be represented the same way in our model. Furthermore, our study considers all teachers the same way. Although this has the advantage of identifying context-independent trends, taking the context into account could provide more refined profiles, particularly with the inclusion of the teaching field and the targeted diploma or academic year. However, the scarcity of our dataset did not allow us to apply such differential study with the same methods. Also, part of our dataset concerns the time of the lockdown caused by COVID, where all the courses were performed remotely with other tools (Teams, Zoom ..). While the lockdown itself remains quite short, teachers may have changed their habits afterwards, and our model may not be valid anymore if such changes occurred and will remain durably. A dedicated study on this problematic is then required. For the teacher questionnaire on LMS habits, there is a risk that the population responding to it might not be representative. Indeed, teachers who are not interested in the LMS or those who think to be experts do not answer the questionnaire, and therefore it could be biased. Internally, the model is representative of the teachers' behavior, which is bound to evolve as well as the population itself (new teachers, others leaving the institution). The clusters must therefore be recalculated and the optimum interval is not known. The structure of the model (the axes) also depends on the functionalities proposed by the LMS and partly on their use by the teachers (quantitative analysis): this structure is thus not stable in time. The model needs to be monitored: certain axis functionalities can be added or removed, new axes can be created or recomposed. Such changes render the analysis of teachers' evolution in the long term a delicate task.

Conclusion and perspectives

In this paper, we designed a behavioral model of teachers based on a qualitative and a quantitative analysis. It describes teachers' practices through six major axes of mastery: evaluation, reflection, communication, resource, collaboration as well as interactivity and gamification. From this model, we designed several TA indicators. Eventually, we proposed a first prototype of a web application that exploits this model and these indicators, dedicated to teachers and PE to provide the former with self-assessment and recommendations features, and to allow the latter to detect teachers in specific needs and teachers with expert profiles. We will continue in the short term to refine our model with the inclusion and analysis of new features that would consolidate our axes, such as time related features to express regularity or skill oversight. On the other hand, other features may also provide new axes, as with social related features to explore knowledge diffusion through the LMS when teachers are working together on the same course. Indeed, once the first version of the tool will be operational, we will experiment it at the scale of our University to evaluate its usability, the interest teachers will show in it, and then test whether it allows inducing learning situations and if recommendations are followed and relevant. The recommendation algorithm, implemented with a simple rule based system, requires also to leverage an important issue: how to recommend on the scarce data? Since teachers do not have infinite free time for peer tutoring, and because our model will always have a given latency before any change, we have to avoid recommending the same tutor too many times. Also, we have to take into account the users' proximity, that could be an important factor of success. Our model is agnostic to the learning domain so far, and thus does not capture the difference of practice it may exist from one discipline to another. Doing so may reduce such risk and at the same time, improve the probability both teachers benefit from sharing a professional context. At a longer term, we will complete the development of a tool that will allow teachers to self-assess and get recommendations in order to enrich their practices on the LMS. Also, we intend to conduct an experiment on the impact that this tool can have on teachers' practices.

Figure 1 :

 1 Figure 1: The number of teachers per the number of nonzero variablesThe first component (comp 1) expresses 33.97% of the total variance, and represents the global usage of the platform with a consistent use of most platform tools. In other words, all variables are involved with the same polarity and regardless of the type of activities exploited, which synthesizes the overall use of the LMS. The second component (comp 2), which explains 11.56% of the total variance, highlights the management of evaluation within the LMS. It gathers tools to manage assignments and submissions (grade, quiz, assignment), as well as the use of calendars which is mainly intended to manage deadlines for assignments and assessments. The third component (comp 3) concerns exclusively the use of forums (forum, forum posts) and expresses 10.42% of the total variance. The fourth component (comp 4) represents essentially the use of chat activities and the exploitation of images in course sections, and it explains 8.61% of total variance. Based on the fact we have no theoretical support nor empirical insight to consider such odd association, we consider that this component which relates chats to images is coincidental. The last component (comp 5) expresses 7.18% of total variance and it is based on course structure. It brings together tools used on LMS to organize and personalize lessons like pages and folders.

Figure

  Figure 2: SUS score

Figure 3 :

 3 Figure 3: Use of the LMS by teachers

Figure 4 :

 4 Figure 4: Teacher dashboard for self-assessment and recommendations

Table 1 :

 1 Description of the final dataset variables.

	Variable	Description : The	Mean Std
		average number of
	grade	teacher's	cre-	1.40	3.86
		ations/editions of
		grades.
	quiz	teacher's	cre-	0.34	1.21
		ations/editions of
		quizzes.
	assignment	teacher's	cre-	0.36	0.77
		ations/editions of
		assignments.
	calendar	teacher's use of cal-	1.86	4.91
		endar.
	chat message teacher's chat mes-	0.61	3.84
		sages sent.
	forum post	teacher's publica-	2.41	21.44
		tions of posts in the
		forums.
	forum	teacher's creations	0.32	0.99
		of forums.
	img	teacher's use of im-	0.19	1.00
		ages included in the
		course sections.
	all links	teacher's use of	4.01	10.40
		links in sections,
		labels or URL.
	url	teacher's use of the	1.46	4.44
		URL resource.
	file	teacher's use of the	5.82	10.31
		file resource.
	folder	teacher's use of the	0.99	2.55
		folder resource.
	page	teacher's use of the	0.66	1.90
		file resource.

Table 2 :

 2 Choices for teachers in requesting help

Table 3 :

 3 Results of the clustering analysis

	Axis Kmeans Dbscan Hierarchical	Gaussian
			Clust.	mixt.
	A.1 S=0.81,	S=0.85,	S=0.84,	S=0.2,
	N=2	N=1,	N=2	N=2
		O=3
	A.2 S=0.91,	S=0.68,	S=0.81,	S=0.89,
	N=13	N=1,	N=6	N=15
		O=11
	A.3 S=0.84,	S=0.92,	S=0.77,	S=0.7,
	N=2	N=1,	N=2	N=2
		O=2
	A.4 S=0.83,	S=0.91,	S=0.85,	S=0.59,
	N=2	N=1,	N=2	N=2
		O=1
	A.5 S=0.76,	S=0.81,	S=0.76,	S=0.34,
	N=2	N=1,	N=2	N=2
		O=1
	A.6 S=0.98,	S=0.98,	S=0.98,	S=0.87,
	N=2	N=1,	N=2	N=2
		O=3

https://organisation.univ-pau.fr/fr/labels/le-projethype-13.html

https://xapi.com/developer-overview/