Konstantinos Chatzilygeroudis

Antoine Cully

Jean-Baptiste Mouret
email: jean-baptiste.mouret@inria.fr

Towards semi-episodic learning for robot damage recovery

The recently introduced Intelligent Trial and Error algorithm (IT&E) enables robots to creatively adapt to damage in a matter of minutes by combining an off-line evolutionary algorithm and an on-line learning algorithm based on Bayesian Optimization. We extend the IT&E algorithm to allow for robots to learn to compensate for damages while executing their task(s). This leads to a semi-episodic learning scheme that increases the robot's life-time autonomy and adaptivity. Preliminary experiments on a toy simulation and a 6-legged robot locomotion task show promising results.

I. INTRODUCTION

.

.

A promising approach is the Intelligent Trial and Error algorithm (IT&E), a recently introduced algorithm [8]. The intuition behind IT&E is that, before the mission, an off-line and computationally expensive evolutionary algorithm can be used to create a behavior-performance map that predicts the performance of thousands of different behaviors. While in mission, this map, guides a fast and on-line search, based on

Bayesian Optimization [START_REF] Brochu | A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning[END_REF], to find a compensatory behavior. An important idea is that the behavior-performance map is created using a simulated intact robot, but the algorithm is able to find a working behavior on the damaged real robot because some behaviors from the map perform similarly on the intact and the damaged robot (typically, the behaviors that do not rely on the broken part). The most recent results showed that IT&E can allow various types of robots (a 6legged robot and an 8-DOF manipulator) to compensate for many different types of injuries in a matter of minutes [START_REF] Cully | Robots that can adapt like animals[END_REF], [START_REF] Cully | Creative adaptation through learning[END_REF].

Although the IT&E approach is promising, its main limitation is the pure episodic approach it has adopted: for each trial (episode), the robot has to begin in the same starting state (Figure 1a). This is limiting because learning a compensatory behavior has to be achieved in two steps, first learn a compensatory behavior, and then use it to complete the task. On the contrary, a wounded animal, for example, can perform trial and error "episodes" to learn how to walk again, while going back to its nest for protection.

In this paper, we extend the IT&E algorithm by (1) using a generic reward of the outcome of each atomic behavior of the robot in the adaptation part, and by (2) adding a specialized reward selection layer that selects a specialized reward function at each episode. These additions allow for a semi-episodic learning scheme that improves the robot's long-term autonomy by allowing to recover while attempting to achieve its task(s) (Figure 1b).

II. BACKGROUND

A. Bayesian Optimization with Gaussian Processes

Bayesian Optimization (BO) is a well-established strategy for finding the extrema of functions that are expensive to evaluate [START_REF] Brochu | A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning[END_REF], [START_REF] Mockus | Bayesian approach to global optimization: theory and applications[END_REF]. It is applicable in cases where one does not have a closed-form expression for the objective function (the function is a "black-box"), but where one can obtain observations (possibly noisy) of this function. One of the distinctive features of BO is that it constructs a probabilistic model for the objective function and then exploits this model to make decisions about which point to evaluate next, while taking into account the uncertainty.

There are two major choices that must be made when performing BO. First, one must select a prior over functions that will express assumptions about the function being optimized. Second, one must choose an acquisition function, u(x|D 1:t ), which is used to construct a utility function from the model posterior, allowing us to determine the next point to evaluate.

Many models could be used for the BO prior, but Gaussian Process (GP) priors are the most common choice [START_REF] Brochu | A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning[END_REF]. A GP is an extension of the multivariate Gaussian distribution to an infinite-dimension stochastic process for which any finite combination of dimensions will be a Gaussian distribution [START_REF] Brochu | A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning[END_REF]. A GP is a distribution over functions, completely specified by its mean function, m(•) and covariance function, k(•, •):

f (x) ∼ GP (m(x), k(x, x ′ )) Assuming D 1:t = {(x 1 , f (x 1 )), • • • , (x t , f (x t ))} is a set of observations and σ 2
noise the sampling noise, the GP is computed as follows:

p(f (x)|D 1:t , x) = N (m t (x), σ 2 t (x))
where:

m t (x) = k ⊤ K -1 D 1:t σ 2 t (x) = k(x, x) -k ⊤ K -1 k K =    k(x 1 , x 1 ) • • • k(x 1 , x t ) . . . . . . . . . k(x t , x 1 ) • • • k(x t , x t )    + σ 2 noise I k = k(x, x 1 ) • • • k(x, x t )
We used Upper Confidence Bound (UCB) as the acquisition function. We refer the reader to Brochu et al. [START_REF] Brochu | A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning[END_REF] for a more detailed explanation.

B. Intelligent Trial & Error Algorithm

IT&E proposed a novel approach for robot damage recovery that consists of a 2-step process. An off-line evolutionary algorithm, MAP-Elites [START_REF] Mouret | Illuminating search spaces by mapping elites[END_REF] [START_REF] Cully | Robots that can adapt like animals[END_REF], that generates many thousands of potential good behaviors is followed by a trial and error on-line adaptation part, based on BO (M-BOA), in order to find a compensatory behavior. MAP-Elites is an evolutionary illumination algorithm: instead of searching for a single, best solution, like optimization algorithms, MAP-Elites searches for the highestperforming individual for each point in a user-defined space. This user-defined space is often called the behavior space, because the dimensions of variation (behavior descriptors) usually measure behavioral characteristics.

In IT&E, the authors made a slight modification to the classical BO scheme. Their BO variation, called Map-Based BO Algorithm (M-BOA), models the difference between a mean function and the actual performance, instead of directly modeling the objective function (P(•) is the mean function):

m t (x) = P(x) + k ⊤ K -1 (D 1:t -P(x 1:t ))
In the original work, the mean function was the prediction of the performance in the map generated from MAP-Elites. Algorithm 1 shows the pseudo-code for M-BOA.

Algorithm 1 M-BOA (Map-Based BO Algorithm) 1: procedure M-BOA 2:
∀x ∈ map :

3: p(f (x)|x) = N (P(x), k(x, x)) 4:
while stopping criteria not met do 5:

x t+1 = argmax x u(x|D 1:t ) ⊲ Next test point 6:
Y t+1 = perf ormance(execute behavior(x t+1 )) 7:

D 1:t+1 = {D 1:t , (x t+1 , Y t+1 )} 8:

Update GP III. APPROACH

A. Generic Reward

In the original IT&E paper, the GP modeled the performance of each atomic behavior given a task. In this paper, we suggest learning a mapping from the atomic behaviors to the resulting relative outcomes. We call it a Generic Reward (GR) of the outcome of each atomic behavior of the robot. We use one GP for each dimension of the GR.

For example, imagine we have a robot moving in 2D space using an 1D continuous atomic behavior (direction to move 0.1-step). A GR could be the relative position of the robot after executing a behavior -(x, y). Thus, we need 2 GPs: GP x (θ), GP y (θ). If we query the GPs at the point θ 0 , then we get a position, p 1 = (GP x (θ 0 ), GP y (θ 0 )), as the prediction. In that way, we can now compute specialized rewards for different locomotion tasks, like the distance to different target points.

Put differently, the GR is a description of the outcome of each atomic behavior of the robot that it is generic-enough to be independent from one task to another, but specificenough so that the performance of the outcome of one atomic behavior given a task can be computed.

The changes for M-BOA to work are:

• define a Reward function that takes the GPs' prediction as input and returns the expected task performance; • define an Aggregator function (afun) that takes as input the execution of an atomic behavior and returns the GR.

B. Specialized Reward Selection Layer

We, also, augment the proposed algorithm, by adding a layer responsible for selecting the Reward function, defined above. We call it Specialized Reward Selection Layer (RSL). Since we are modeling a GR of the outcome of each behavior of the robot and not the actual performance (given a task), we can change the Reward function as often as needed. This is true, because only the acquisition function needs an actual reward to select a new test point. We suggest updating or selecting the Reward function at each iteration of M-BOA.

For instance, if we consider the previous mobile robot example, at each iteration a planner algorithm chooses the next best point to reach. This point can then be used by the RSL in order to update the Reward function so that it outputs the Euclidean distance between the point selected by the planner and the prediction of the GPs.

C. Semi-Episodic Learning Algorithm

Using the two proposed additions, we can now have a non purely episodic version of the IT&E algorithm. We call it Semi-Episodic Learning Algorithm (SELA). The pseudocode is shown in Algorithm 2. Before mission (in simulation with intact robot): Y t+1 = af un(execute behavior(x t+1 )) 14:

Algorithm 2 Semi-Episodic Learning Algorithm

D 1:t+1 = {D 1:t , (x t+1 , Y t+1 )} 15:

Update GPs

IV. PRELIMINARY EXPERIMENTS

A. Toy Simulation

As a toy example, we consider the mobile robot example introduced previously. This mobile robot is a point (no dimensions, no orientation) and can take a 0.1-long step in any direction. We represent each atomic behavior by a scalar value, θ: the direction of the corresponding move. This environment was inspired by Engel et al. [START_REF] Engel | Reinforcement learning with Gaussian processes[END_REF]. The task of the robot is to reach a target point despite some damage.

Because the example is too simple, but also to show the effectiveness of our method without relying on simulated data, we did not generate any behavior-performance map. We used the exact model of the intact robot as the mean function. Also, for the GR, we used the (x, y) relative end position of each behavior, for the Reward function the Euclidean distance between the next target and the prediction of the GPs and for the reward selection layer an A* path planner.

To evaluate our technique we used the following two control experiments:

• learn the model of the robot (using GPs) via random babbling and then use it to complete the task; • solve the problem with the classic IT&E approach: we first learn with IT&E how to walk in 4 major directions (up, down, right, left) and then use these behaviors to reach the target. For both of the baseline approaches, we measure the number of iterations required to learn and the number of steps that they take to complete the task. We ran 50 replicates of each approach.

We ran 50 replicates of each approach for the scenario: "Reach the target point (2.0, 2.0) starting from the origin despite a 0.5 radians angle offset in the range direction θ > 0". To make the task a little more realistic we added a small Gaussian noise (µ = 0, σ 2 = 0.01) to the position observations. Figure 2 shows the resulting performance (number of atomic behaviors taken to reach the target) for the different approaches. Our algorithm is able to reach the target with almost the optimal number of steps (i.e. if we perfectly knew the model), that is in much fewer steps than the other approaches.

B. 6-Legged Simulated Robot locomotion task

As a more realistic example, we consider a simulated 6legged (hexapod) robot moving in space with the same task as in the toy simulation. See Figure 1b for the scenario and [START_REF] Cully | Robots that can adapt like animals[END_REF] for more details on the simulated hexapod. We evolved different atomic behaviors using the MAP-Elites algorithm with an 8D behavior descriptor (2 dimensions for space diversity + 6 dimensions for walking diversity), inspired by [START_REF] Cully | Behavioral repertoire learning in robotics[END_REF], [START_REF] Cully | Creative adaptation through learning[END_REF]. The number of atomic behaviors evolved were approximately 1 million. We used this behavior-performance map as the mean function. All the other parameters were the same as in the toy simulation experiment.

To evaluate our technique we used similar control experiments as in the toy simulation experiment:

• IT&E variant #1: we learn the outcome of the atomic behaviors (using GPs) via selecting the most uncertain behavior for N = 15 iterations. This can be considered as a uniform sampling of the behavior space. We then use what we learned to reach the target. • IT&E variant #2: we first learn with IT&E how to walk in 4 major directions (forward, backward, turn cw, turn ccw) and then use these behaviors to reach the target. Comparison between the baseline approaches and SELA for the 6-legged robot simulation. For both of the baseline approaches, we measure the number of the iterations required to learn and the number of steps that they take to complete the task. We ran 50 replicates of each approach.

We ran 50 replicates of each approach for the scenario: "Reach the target point (2.0, 2.0) despite the middle right leg being removed". We, also, added a small Gaussian noise (µ = 0, σ 2 = 0.01) to the position observations. Figure 3 shows the resulting performance (number of atomic behaviors taken to reach the target) for the different approaches. Our algorithm is able to find solutions in fewer steps than the other approaches.

C. 6-Legged Robot locomotion task

We, also, applied our technique on a real 6-legged robot. Preliminary experiments show promising results 1 .

V. CONCLUSION AND FUTURE WORK

We have introduced a semi-episodic learning scheme for robot damage recovery and a novel algorithm in this direction: Semi-Episodic Learning Algorithm. The intuition behind this scheme is that the robot can learn in a data-efficient way how to compensate for damages while completing its task(s). This is achieved by (1) shrinking the search space, using simulated or computed data as prior knowledge, and by (2) using a generic reward of the outcome of the atomic behaviors of the robot instead of their performance given a task.

Fig. 1 :

 1 Fig. 1: Episodic vs Semi-episodic learning for robot damage recovery. (a) Episodic Learning: The robot learns in episodes how to get to a single target starting from the same initial state. (b) Semi-episodic Learning: The robot learns the outcome of its atomic behaviors while executing the task.

  (x)|x) = N (P(x), k(x, x)) 10: while stopping criteria not met do 11: Update Reward function 12:x t+1 = argmax x u(Reward(GP s(x))|D 1:t ) 13:

Fig. 2 :

 2 Fig.2: Toy Simulation Evaluation. Comparison between the baseline approaches and SELA for the toy simulation. For both of the baseline approaches, we measure the number of iterations required to learn and the number of steps that they take to complete the task. We ran 50 replicates of each approach.

Fig. 3 :

 3 Fig.3: 6-legged Robot Simulation Evaluation: Comparison between the baseline approaches and SELA for the 6-legged robot simulation. For both of the baseline approaches, we measure the number of the iterations required to learn and the number of steps that they take to complete the task. We ran 50 replicates of each approach.

1 https://www.youtube.com/watch?v=Gpf5h07pJFA

Future work includes performing more experiments with the real robot as well as experiments with different robots. In addition, BO can be replaced by other techniques that scale better. What is more, we used a naive reward selection layer, but more efficient/sophisticated methods can be used. We are currently investigating in this direction. Additionally, theoretical guarantees and analysis should be investigated in detail. Overall, this work is a first step towards semi-episodic and life-long learning for robot damage recovery.

ACKNOWLEDGMENTS

This work was supported by the ERC project "ResiBots" (grant agreement No 637972), funded by the European Research Council.

APPENDIX

For all experiments the following parameters were used: Error threshold for reaching goal: ǫ goal = 0.1

A. BO with GPs


François Bouchet
email: francois.bouchet@lip6.fr

Jason M Harley
email: jharley1@ualberta.ca

Roger Azevedo
email: razeved@ncsu.edu

Can adaptive pedagogical agents' prompting strategies improve students' learning and self-regulation?

Keywords: adaptivity, pedagogical agents, self-regulated learning, metacognition, user perception

This study examines whether an ITS that fosters the use of metacognitive strategies can benefit from variations in its prompts based on learners' selfregulatory behaviors. We use log files and questionnaire data from 116 participants who interacted with MetaTutor, an advanced multi-agent learning environment that helps learners to develop their self-regulated learning (SRL) skills, in 3 conditions: one without adaptive prompting (NP), one with fading prompts based on learners' deployment SRL processes (FP), and one where prompts can also increase if learners fail to deploy SRL processes adequately (FQP). Results indicated that an initially more frequent but progressively fading prompting strategy is beneficial to learners' deployment of SRL processes once the scaffolding is faded, and has no negative impact on learners' perception of the system's usefulness. We also found that increasing the frequency of prompting was not sufficient to have a positive impact on the use of SRL processes, when compared to FP. These results provide insights on parameters relevant to prompting adaptation strategies to ensure transfer of metacognitive skills beyond the learning session.

Introduction

Designing intelligent tutoring systems (ITSs) that dynamically adapt to learners' emerging understanding of content and to their use of metacognitive processes has been a major objective for the past decade [START_REF] Azevedo | International Handbook of Metacognition and Learning Technologies[END_REF]. Specifically, intelligent systems should provide learners with individualized instruction, feedback and scaffolding during their learning session [START_REF] Bannert | Scaffolding Hypermedia Learning Through Metacognitive Prompts[END_REF], in a way that fosters the transfer of metacognitive skills beyond that session [START_REF] Roll | Improving students' help-seeking skills using metacognitive feedback in an intelligent tutoring system[END_REF]. It is even more challenging in non-linear open-ended learning environments (OELEs) where no optimal way to navigate through the learning material exists and where learners' goals may vary [START_REF] Land | Cognitive requirements for learning with open-ended learning environments[END_REF][START_REF] Kinnebrew | Studying Student Use of Self-Regulated Learning Tools in an Open-Ended Learning Environment[END_REF]. Many critical questions remain unanswered: how often should learners be prompted to perform actions known to foster effective learning? Should prompts vary over time? How can instances where scaffolding should fade be detected?

In this study, we investigated the effect of adaptive prompting on undergraduates' learning and their use of self-regulated learning (SRL) strategies in an OELE with embedded pedagogical agents (PAs). Specifically, we examined how adapting PA prompting impacted learners': (1) use of SRL processes, (2) learning gains, and (3) perception of the system's usefulness. Our associated hypotheses were that: (1) learners should deploy more SRL processes overall, particularly once the scaffolding fades; (2) more efficient SRL should lead to higher learning gains with adaptive prompts; (3) system adaptivity should have a positive effect on learners' evaluation, but the more frequent initial prompting could have a negative effect by making the learners feel overwhelmed.

Method

Participants and experimental conditions

One hundred and sixteen undergraduate students (N = 116, 17-31 years old, M = 20.9 years, SD = 2.4; 64.6% female; 62.9% Caucasian) from two North American Universities, studying different majors and with various levels of prior knowledge participated in this study. Each participant received $50 upon completion of the study and was randomly assigned to one of three experimental conditions: (1) non-adaptive prompt (NP n = 29), (2) frequency-based adaptive prompt (FPn = 29) and ( 3) frequency and quality-based adaptive prompt (FQPn = 58). Participants from adaptive conditions FP and FQP were grouped in some analyses, leading to two samples of identical sizes.

In the NP condition, learners received a moderate but constant amount of prompts from the PAs (on average, 1 per 10 minutes) to engage in various SRL processes. In the FP condition, learners received more prompts at the beginning of the session (on average, 3.5 per 10 minutes), but the probability of prompts being triggered decreased after each new prompt and after each self-initiated enactment of an SRL process. In the FQP condition, the same prompt decreasing rules as in FP apply, but the probability of prompts could also increase if: (1) the learner did not comply with a PA's prompt, or (2) a learner's metacognitive judgment was inaccurate (e.g., marked a page as relevant to their active sub-goal when it was not; cf. Table 1 for the list of conditions of success).

2.2

The testbed system, experimental procedure and data used System overview. MetaTutor [START_REF] Azevedo | The Effectiveness of Pedagogical Agents' Prompting and Feedback in Facilitating Co-Adapted Learning with MetaTutor[END_REF] is an intelligent, hypermedia learning environment in which four embedded PAs help the student learn by prompting them to engage in SRL processes (cf. Table 1). A table of contents gives access to 38 pages (with text and images) on the human circulatory system. The overall learning goal is always visible, as well as two progress bars (cf. C) associated to the sub-goals chosen at the beginning of the session. A timer displays the time remaining in the learning session. One of the four PAs is always visible. Each PA has a specific role: Pam the Planner helps the student to plan their learning sub-goals, Mary the Monitor helps in monitoring the learning, Sam the Strategizer assists with the deployment of learning strategies and Gavin the Guide introduces the system and its questionnaires. The frequency and circumstances under which PAs' prompts are triggered depends on parameters such as the time spent on a page or the relevance of the page to students' current sub-goal. Below the PA, a palette of buttons allows students to self-initiate SRL processes (cf. Table 1), leading to a set of steps very similar to when the prompt comes from a PA: an invitation to perform the process followed by a feedback on its validity (e.g. agreeing the page is relevant to the current learning sub-goal).

Experimental procedure. The experiment involved two different sessions separated by one hour to three days. During the first one (30 to 40 min. long), participants filled and signed a consent form and completed several computer-based self-report questionnaires, a demographics survey and a pre-test on the circulatory system. During the second session (90 min. long), participants used MetaTutor to learn about the circulatory system. Participants had exactly 60 minutes to interact with the content during which they could initiate SRL processes or do so after a PA's prompt. MetaTutor was paused when participants were watching a video, taking a survey, and during an optional 5 minutes break half-way through the session. At the end of the session, participants were given a post-test and filled a questionnaire, the Agent Response Inventory (ARI) [START_REF] Harley | Examining the Potential of Personality Traits and Trait Emotions to Create Emotionally-adaptive Intelligent Tutoring Systems[END_REF], which included questions on their perception of the quality of PAs' prompts. All participants completed their sessions individually on a desktop computer.

Data coding and scoring. Six variables were extracted from the pre-test and posttest questionnaires (two equivalent 25-item multiple choice tests on the human circulatory system), the ARI questionnaire, as well as from the system log files (cf. Table 2).

Table 2. List of the six variables used for analyses.

Variable name Description PropLearnGain

Proportional learning gains (between 0 and 1) using the standard formula: (posttest-pretest)/(1-pretest), for questions relevant to the 2 initial sub-goals and treating negative values as 0 UserAllProc_[Ses-sion|first30|last30] Ratio (per period of 10 minutes) of all SRL processes initiated by the user during: the whole learning session/the first 30 minutes of the session/the last 30 minutes of the session FBQuality [Mary|Sam] Learner's evaluation of the quality of the PA's feedback (1 to 7)

Results

In all of the following statistical analyses, an outlier screening was performed beforehand and outlying scores were replaced by the next most extreme score.

Effects of adaptive prompting on the use of SRL processes

Effect on learner-initiated SRL, overall. A one-way ANOVA with prompt condition as the 3-level independent variable and UserAllProc_Session as the dependent variable revealed a significant main effect of condition on learners' self-initiated SRL behaviors, F(2,113) = 10.17, p < .001, n 2 p = 0.15. The application of a more stringent alpha (p < .01) and the general robustness of ANOVAs to violations of assumptions supports the legitimacy of this finding and rendered a transformation unnecessary, despite equality of variances not being met (Levene's test). Follow-up post hoc comparisons using a Bonferroni correction revealed that the quantity of SRL behaviors that learners self-initiated were significantly different between the NP (M = 1.00; SD = 0.89) and FP (M = 2.04; SD = 1.57), and NP and FQP (M = 2.02; SD = 1.42) conditions, but not between FP and FQP conditions.

Effect on learner-initiated SRL, over time.

A repeated measures ANOVA with prompt condition as the 3-level independent variable, time as an independent 2-level within-subjects variable (first and last 30 minutes) and learners' self-initiated SRL processes as the dependent variable (i.e. UserAllProc_first30 and UserAllProc_last30) revealed a significant main effect of time on learners' self-initiated behaviors F(1,113) = 43.95, p < .001, n 2 p = 0.27 as well as a significant interaction effect of time and condition on learners' self-initiated behaviors F(2,113) = 6.65, p < .001, n 2 p = 0.11; both results remained significant after the application of a stricter alpha (related to results of Box's Test of Equality of Covariance Matrices). A significant main effect of condition on learners' use of SRL behaviors was found, F(2,113) = 7.61, p < .001, n 2 p = 0.12 (even with a more stringent alpha). An examination of Table 3 reveals that participants consistently engaged in more self-initiated SRL behavior during the second thirty minutes than the first, the most striking changes occurring in FP and FQP.

Effects of adaptive prompting on learning gains

Table 4 reveals no difference on average between conditions NP and FP&FQP, counter to our hypothesis that adaptive prompting would help with learning. However, when learning gains from NP and FP are compared, it appears that learners in the FP condition had a small benefit over those in the NP, and that FQP did not help.

Effects of adaptive prompting on perceived system's usefulness

Two one-way ANOVAs with prompt condition as the 3-level independent variable and FBQualitySam (resp. FBQualityMary) as the dependent variable failed to reveal a significant main effect of condition on learners' self-initiated satisfaction regarding the PAs. Descriptive statistics revealed that participants were most satisfied with Sam in the NP condition (M = 3.77, SD = 1.63) in comparison to Sam in the FP (M = 3.13, SD = 1.77) and FQP condition (M = 3.31, SD = 1.79). In contrast, participants were least satisfied with Mary in the FQP condition (M = 4.41, SD = 1.74) in comparison to Mary in the NP (M = 5.00, SD = 1.95) and FP condition (M = 4.95, SD = 1.66).

General Discussion

Adaptive prompting helps learners to self-initiate SRL processes. Learners in (pooled) condition FP&FQP deployed more SRL processes than those in condition NP, as they received more frequent prompting from the system. The number of learnerinitiated processes increased over time despite the decrease of agent-initiated prompts, which can be interpreted as a residual and impactful effect of prompting. Our hypothesis was therefore verified. However, taking into account the quality of SRL processes to reduce PAs' prompts did not help: it may be because inefficient self-regulated learners need more than mere (potentially frustrating) reminders to self-regulate.

Adaptive prompting may not directly help to improve learning. We observed no significant differences in learning between conditions NP and FP&FQP, but the expected trend was there when comparing NP and FP. Therefore, it appears that the adaptiveness in FP was going in the right direction, contrary to the one in FQP. Hence our hypothesis was not supported, which could be partially explained by the fact learners might not have been left without scaffolding for long enough for a difference to appear.

Initially frequent but fading prompting doesn't degrade perceived system's usefulness. We observed that PAs in FP and FQP were not perceived as less helpful than in NP, despite more frequent prompting at the beginning of the session, which could have been detrimental to learners' willingness to follow PAs' recommendations. Conversely, learners who appreciated PAs' interventions could have found them less useful overall as they were less present towards the end.

Limitations and future work. Although this study benefited from a significantly larger sample size than [START_REF] Bouchet | Impact of Different Pedagogical Agents' Adaptive Self-Regulated Prompting Strategies on Learning with MetaTutor[END_REF], a larger sample size (with as many participants in FP as in NP) may have led to more significant results. The limited duration of the learning session (1 hour) might also have prevented observing internalization and integration of the use of SRL processes by learners once agents' scaffolding was fully gone [START_REF] Azevedo | Issues in dealing with sequential and temporal characteristics of self-and socially-regulated learning[END_REF]. Another limitation is the lack of evaluation of the importance of the progressiveness in the scaffolding reduction: another condition with frequent prompting for half a session and no prompting for the second half would be necessary to do so. Finally, we have seen that the adaptation exclusively in terms of frequency of prompting might have been detrimental to learners in condition FQP, and that the quality of the feedback should also be adjusted-confirming its importance [START_REF] Vanlehn | When Are Tutorial Dialogues More Effective Than Reading?[END_REF]. The next steps are to test this approach on other systems, on longer periods of time and to have a finer-grained adaptation.

Table 1 . Condition of successes associated to the different type of SRL prompts. Type Type of PA's prompt Condition of success

 1

		Judgment of Learning (JOL)	Accurate evaluation of what has been learnt
	Monitoring	Feeling of Knowing (FOK) Content Evaluation (CE) Management of Progress Toward Goal (MPTG)	Accurate evaluation of what is already known Accurate evaluation of the relevance of the content relative to the active sub-goal Learner validates their sub-goal in the next 45s
		Summarization (SUMM)	If learner delays, must be performed later on
	Strategy	Coordination of Information Sources (COIS) Draw image already opened	Image is opened in the next 45s Digital notepad in the next 45s
		Draw image not opened yet	Learner accepts to open the image

Table 3 .

 3 Learner-initiated SRL processes by time and condition.

	Variable	NP (n=58) FP (n=29) FQP (n=29) All (n=116)
		M	SD M	SD M	SD	M	SD
	UserAllProc_first30 0.86 0.91 1.16 1.09 1.62 1.17 1.14 1.06
	UserAllProc_last30 1.09 1.13 2.12 1.74 2.35 1.84 1.66 1.60

Table 4 .

 4 Learning-related variables in the 3 conditions considered.

	Variable	NP		FP		FQP	FP&FQP
	M	SD	M	SD	M	SD	M	SD
	Pre-test score (/ 1) 0.69 0.24 0.73 0.19 0.71 0.21 0.72 0.20
	Post-test score (/ 1) 0.83 0.15 0.85 0.12 0.79 0.82 0.16
	PropLearningGains 0.400 0.378 0.448 0.375 0.339 0.299 0.394 0.341