C Edrick

T Bonnet

S Ebastien

Szaffarczyk Scalab

Scalab

The Stationary-Gaze Task Should Not Be Systematically Used as the Control Task in Studies of Postural Control

Keywords: cognitive workload, control task, postural control, precise visual tasks, stationary-gaze task, young adults

In studies of postural control, a control task is often used to understand significant effects obtained with experimental manipulations. This task should be the easiest task and (therefore) engage the lowest behavioral variability and cognitive workload. Since 1983, the stationary-gaze task is considered as the most relevant control task. Instead, the authors expected that free looking at small targets (white paper or images; visual angle: 12 ) could be easier tasks. To verify this assumption, 16 young individuals performed stationary-gaze, white-panel, and free-viewing 12 tasks in steady and relaxed stances. The stationary-gaze task led to significantly higher cognitive workload (mean score in the National Aeronotics and Space Administration Task Load Index questionnaire), higher interindividual body (head, neck, and lower back) linear variability, and higher interindividual body angular variability-not systematically yet-than both other tasks. There was more cognitive workload in steady than relaxed stances. The authors also tested if a free-viewing 24 task could lead to greater angular displacement, and hence greater body sway, than could the other tasks in relaxed stance. Unexpectedly, the participants mostly moved their eyes and not their body in this task. In the discussion, the authors explain why the stationary-gaze task may not be an ideal control task and how to choose this neutral task.

I n the literature on postural control, some experimenters are interested in the effect of visual tasks on postural control and how the central nervous system (CNS) can simultaneously control upright stance and oculomotor behaviors to perform visual tasks. In these studies, investigators usually study how precise (fast and accurate) visual tasks can influence the way individuals sway. Many kinds of precise visual tasks have been used, such as tracking a dot appearing alternatively left and right at a constant angle and frequency [START_REF] Giveans | Postural sway and the amplitude of horizontal eye 720 movements[END_REF]Rodriguez et al., 2013;[START_REF] Rougier | Performing saccadic eye movements or blinking improves postural control[END_REF][START_REF] Stoffregen | Postural sway and the frequency of horizontal eye movements[END_REF][START_REF] Stoffregen | Postural stabilization of visually guided eye movements[END_REF], counting the occurrence of a letter in a text [START_REF] Bonnet | Deterministic and stochastic postural processes: Effects of task, environment, and age[END_REF][START_REF] Prado | Postural sway during dual tasks in young and elderly adults[END_REF][START_REF] Stoffregen | Modulating postural control to facilitate visual performance[END_REF], detecting one or several target(s) in a visual display [START_REF] Shockley | Mutual interpersonal postural constraints are involved in cooperative conversation[END_REF], aligning two crosses [START_REF] Mitra | Divergent effects of cognitive load on quiet stance and task-linked postural coordination[END_REF], detecting a difference between two targets (Stoffregen, Hove, Bardy, Riley, & Bonnet, 2007b), and recognizing a target displayed only several milliseconds [START_REF] Poulain | Age-related changes of visual contribution in posture control[END_REF]. All these precise visual tasks are known to significantly influence postural control (e.g., [START_REF] Bonnet | Deterministic and stochastic postural processes: Effects of task, environment, and age[END_REF][START_REF] Mitra | Divergent effects of cognitive load on quiet stance and task-linked postural coordination[END_REF][START_REF] Poulain | Age-related changes of visual contribution in posture control[END_REF][START_REF] Prado | Postural sway during dual tasks in young and elderly adults[END_REF][START_REF] Rougier | Performing saccadic eye movements or blinking improves postural control[END_REF][START_REF] Stoffregen | Modulating postural control to facilitate visual performance[END_REF][START_REF] Stoffregen | Postural stabilization of visually guided eye movements[END_REF]Stoffregen, 55 Bardy et al., 2007;Stoffregen, Hove et al., 2007). They are the experimental tasks of interest.

When experimenters search to discover the effect of precise visual tasks on postural control, they usually record upright stance in a control task to provide baseline data.

60

Baseline data are required both to show the existence of significant effects caused by the experimental manipulations and to know the direction of these effects. It is definitely important to use an appropriate control task to well understand the effects of the experimental manipulations (Fraizer 65 & [START_REF] Fraizer | Methodological and interpretive issues in posture-cognition dual-tasking in upright stance[END_REF]. The control task should depend on-or be adjusted to-the experimental tasks of interest, both control and experimental tasks being linked to each other.

In 1983, a consortium of specialists discussed the definition and criteria of this control task [START_REF] Kapteyn | Standardization in platform stabil-740 ometry being a part of posturography[END_REF].

70

They argued that this task should lead to the minimum amount of variability of postural behavior. Indeed, a task engaging low variability could be the most reproducible task and thus allow comparisons of results within and between studies [START_REF] Kapteyn | Standardization in platform stabil-740 ometry being a part of posturography[END_REF]. Moreover, analyses 75 between the control and experimental tasks could more easily lead to significant findings if individuals exhibit lower behavioral variability in the control task. The consortium decided that the most relevant control task should be the stationary-gaze task. In this task, participants stare at a sta-80 tionary target (dot, cross, or circular area) throughout the trial [START_REF] Kapteyn | Standardization in platform stabil-740 ometry being a part of posturography[END_REF].

A few years later, researchers acknowledged that postural control can involve higher structures of the CNS [START_REF] Mihara | Role of the prefrontal cortex in human balance control[END_REF] 85 [START_REF] Teasdale | On the cognitive penetrability of posture control[END_REF][START_REF] Woollacott | 830 Aging and posture control: Changes in sensory organization and muscular coordination[END_REF]. Hence, since 1985, it can be assumed that different experimental tasks may alter the cognitive demand. Therefore, and to complete [START_REF] Kapteyn | Standardization in platform stabil-740 ometry being a part of posturography[END_REF] definition, the control task should also require 90 the minimum amount of cognitive workload, it should not induce any interference [START_REF] Swan | Improving balance by performing a secondary cognitive task[END_REF], and it should be a single task [START_REF] Swan | Improving balance by performing a secondary cognitive task[END_REF] or the easiest task [START_REF] Mitra | Postural costs of suprapostural task load[END_REF]. In contrast, the experimental dual tasks should be harder and induce inter-95 ference Q1 (Swan et al., 2003). In the present manuscript, we questioned whether the stationary-gaze task should systematically be considered as the gold control task. We questioned whether this task was the easiest task requiring the lowest variability in posture and the lowest cognitive workload. We had some doubt about it because this stationary-gaze task was already assumed as cognitively demanding (Ajrezzo, Wiener-Vacher & Bucci, 2013;Ben Hamed, Duhamel, Bremmer, & Graf, 2002), as a difficult [START_REF] Wade | Pioneers of eye movement research[END_REF] and tiring task [START_REF] Rougier | La r ealization de mouvements de saccades oculaires affecte les strat egies de maintien de l' equilibre[END_REF].

The white-panel task has also been used as a control task in the literature reports (e.g., [START_REF] Bonnet | Deterministic and stochastic postural processes: Effects of task, environment, and age[END_REF][START_REF] Prado | Postural sway during dual tasks in young and elderly adults[END_REF][START_REF] Stoffregen | Modulating postural control to facilitate visual performance[END_REF]. In this task, participants simply look at a small white panel. In these studies, the white panel was projected within a small visual angle (i.e., on a visual angle lower than 15 delimited by a circle with a black line) to allow participants to look at the target in moving (rotating) only their eyes and not their head or body [START_REF] Hallett | Eye movements[END_REF]. This task may be less cognitively demanding than the stationary-gaze task because it does not impose any constraint on fixation or attention (Ben Hamed et al., 2002). Similarly, when participants simply perform free exploration of a small image (lower than 15 ), the visual task may not be cognitively demanding because fixations and saccades are simply exploratory, individuals can look at the image with no goal. Moreover, if participants do not move any body part to perform the visual task, but only their eyes, they may not engage greater behavioral variability than in the stationary-gaze task. Hence, these free-looking tasks may also be relevant control tasks. In other words, we suggest that free looking could stand as a potentially control, single, tasks and not as precise or experimental tasks.

The study's primary objective was to test whether whitepanel and/or free-viewing 12 tasks could be easier tasks and thus lead to lower cognitive workload and lower variability of postural sway than the stationary-gaze task. We assumed that the stationary-gaze task should lead to higher cognitive workload than these other tasks. As a consequence, we assumed that the stationary-gaze task would lead to higher variability of postural sway. A secondary objective was to test whether a free-viewing 24 task would lead the participants to rotate their head and, therefore, move their body significantly more than in the three other tasks. The general goal of the present manuscript was methodological; we did not test any model of postural control.

METHODS

Participants

Sixteen healthy young students (eight men, eight women) from the Universities of Lille volunteered to participate in this study. The mean age, bodyweight and height of the participants were 20.5 § 0.89 years, 66 § 15.74 kg, and 172.19 m § 10.13, respectively. The study was approved by the local ethical committee of our laboratory and performed in accordance with the tenets of the Declaration of Helsinki. The participants gave their written informed consent to participation.

Apparatus

A dual-top force platform (AMTI, Watertown, MA) was used to record center of pressure (COP) displacement with a sampling frequency of 120 Hz. The platform was placed 2.75 m from the facing wall. A magnetic tracking system (Polhemus Liberty 240/8-8 System, Colchester, VT) was used to record head, neck and lower back marker displacements with a sampling frequency of 120 Hz. The markers were positioned at the occiput (head marker, on the headset), at the seventh cervical vertebra (neck marker), and at the fifth lumbar vertebra (lower back marker, on a chest belt). A head-mounted eye tracker (SensoMotoric Instruments, Teltow, Germany) was used to record eye motions. The iViewX system recorded the pupil position at a sampling rate of 50 Hz. These three apparatuses were synchronized all together with the projection of the experimental images.

A validated French version of the National Aeronotics and Space Administration Task Load Index (NASA-TLX; [START_REF] Cegarra | Etude des propri et es de la ver-705 sion francophone du NASA-TLX[END_REF] was used to quantify the cognitive workload. This multidimensional questionnaire was chosen because it has excellent reliability, sensitivity and utility [START_REF] Hart | NASA-TLX index (NASA-TLX); 20 years later[END_REF] and because it is sensitive to fine variations between tasks [START_REF] Cegarra | Etude des propri et es de la ver-705 sion francophone du NASA-TLX[END_REF]. This questionnaire was used to measure a global variable of the cognitive workload.

During recordings, the participants had their feet on the printed lines that marked the normative stance width and angle recommended by [START_REF] Mcilroy | Preferred placement of the feet during quiet stance: Development of a standardized foot placement for balance testing[END_REF]17 cm and 14 ).

The eight images projected to the participants displayed scenes of real life such as parts of a town (streets, crowds) and various rooms from houses. The content of all images were as neutral as possible to avoid emotional images to potentially impact postural control [START_REF] Stins | Effects of affective picture viewing on postural control[END_REF]. Each trial showed a different image.

Tasks

The study consisted of eight experimental tasks, each of which performed with two trials (each lasting 45 sec). The eight tasks were run successively in a random order but the two trials in each task were performed one after another in each task. The four visual tasks were the stationary-gaze, white-panel, and free-viewing 12 and 24 tasks (Figures 1A,1B,1C,and 1D,respectively). These tasks were performed with the participants told to stand as steadily as possible (steady instruction) or to adopt a relaxed stance. The participants were also told to refrain from making any voluntary movement unrelated to the task performed (e.g., hand movements). Both types of instruction (steady and relaxed) were used to test if the findings could be generalized in both contexts.

In the stationary-gaze task (Figure 1A), the participants had 205 to stare at a black dot of 1 projected at eye height and surrounded by a black circle line of 12 . The white-panel task was similar to the stationary-gaze task for the first 5 s. Then, the central black dot disappeared and the participants could freely look anywhere they like within the circle (Figure 1B). In the 210 free-viewing 12 and 24 tasks, the participants first had to stare at a black dot (1 ) surrounded by an image for 5 s. Then, the dot disappeared and they could freely look at the image within the circle (Figures 1C and1D).

In the free-viewing 12 and 24 tasks, the images were 215 different (e.g., Figures 1C and1D). To control the image effect on postural and oculomotor behaviors, half of the participants looked at images 1-4 in the free-viewing 12 task and images 5-8 in the free-viewing 24 task and the other half looked at images 5-8 in the free-viewing 12 220 task and images 1-4 in the free-viewing 24 task. In both free-viewing tasks, the participants were invited not to search anything in the image but to look at it freely. After the completion of each free-viewing task, the participants were questioned to check that none of them performed any 225 searching task. Indeed, we needed to control whether the participants did not perform an experimental task as we defined it in our introduction.

The participants performed the tasks barefoot. The light was turned off so that the participants could clearly see the 230 image. The eight tasks were run by block of two trials in order to evaluate the cognitive workload after each task (based on two successive trials). The images were projected further than 1.5 m to avoid images to provide any useful information for postural control (Bonnet, Temprado, & Berton, 2010b;[START_REF] Dijkstra | Postural responses to stationary and moving scenes as a func-710 tion of distance to the scene[END_REF]. If images had been projected at a lower distance, the presence of the image-independent of the visual task performed-could have significantly changed the characteristics of postural sway (cf. Bonnet, Temprado, & Berton, 2010b).

Dependent Variables and Analyses

The global measure of workload in the NASA-TLX assessed the subjective cognitive workload in each task [START_REF] Hart | Development of the NASA task load index (TLX): Results of empirical and theoretical research[END_REF].

The range (R), standard deviation (SD), and mean velocity (V) were used to analyze linear displacements of the COP, head, neck, and lower back in both anteroposterior (AP) and mediolateral (ML) axes (e.g., [START_REF] Era | Postural balance in a random sample of 7,979 subjects aged 30 years and over[END_REF][START_REF] Prieto | Measures of postural steadiness: Differences between healthy young and elderly adults[END_REF]. The variables were called R AP , R ML , SD AP , SD ML , V AP , and V ML . The angular displacement (yaw, pitch) of the head, neck and lower back were used to assess if the participants rotated their body segments during the tasks. The variables were called R yaw , SD yaw , V yaw , R pitch , SD pitch , and V pitch . Changes in linear variables were used to discuss the so-called COP and body displacements or sways while changes in angular displacements were used to discuss goal-directed behaviors to perform the tasks. Hence, changes in angular displacement were not used to discuss an increase or a decrease in sway.

Usually in the literature reports, investigators seek to discover in which task the participants exhibit more or less COP and/or body sway. However, [START_REF] Kapteyn | Standardization in platform stabil-740 ometry being a part of posturography[END_REF] suggested that the gold standard control task should lead to the minimum interindividual postural sway variability, not 265 to the minimum amount of postural sway. Therefore, to test our main hypothesis, the question was asked how to obtain interindividual postural sway variability for each participant in each trial?

In usual time-series, there is one value for each subject in 270 each trial. Hence, there is only one SD for each time-series, which one is calculated as the mean distance of data to the mean of the time-series ( SD

D ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P x ¡ x ð Þ 2 n q
). We needed to obtain a time-series in which each single value could design a variability and not a quantity of postural sway. 275 Our insight was to calculate the distance of each datum to the mean of the time-series with the formula

SD D ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi x ¡ x ð Þ 2 q .
In this way, each datum in the time-series showed interindividual variability (to the mean) that could be tested in statistical analyses. To be clear, we used all the 280 dependent variables mentioned earlier (R AP , R ML , SD AP , SD ML , V AP , V ML , R yaw , SD yaw , V yaw , R pitch , SD pitch , and V pitch for the head, neck, and lower back displacement and also R AP , R ML , SD AP , SD ML , V AP , V ML , R yaw , and SD yaw for the COP displacement) but analyzed the distance to the 285 mean for each single variable. For these unconventional dependent variables chosen to test our main hypothesis, we only searched for effects of task between three tasks, that is, the stationary-gaze, white-panel, and free-viewing 12 task. We did not analyze the effects of instruction or the 290 task by instruction interaction effects with these unconventional dependent variables because these results did not interest us (i.e., they could not test any of our hypotheses).

Many investigators in the literature reports assumed that the best control task should lead participants to 295 exhibit the lowest, or minimum, amount of postural sway [START_REF] Glasauer | How the eyes move the body[END_REF][START_REF] Laurens | Visual contribution to postural stability: Interaction between target fixation or tracking and static or dynamic large-field stimulus[END_REF][START_REF] Rougier | La r ealization de mouvements de saccades oculaires affecte les strat egies de maintien de l' equilibre[END_REF][START_REF] Ustinova | Gaze and viewing angle 825 influence visual stabilization of upright posture[END_REF] and not lower interindividual postural sway variability. For this reason, we also ana-300 lyzed in which task participants exhibited the lowest amount of COP and/or body sway. These analyses were only secondary, complementary, but did not test our second hypothesis. For these analyses, not only the effect of task but also the effect of instruction and the task by 305 instruction were detailed to test our last hypothesis (related to our second objective).

The mean duration and range of left/right and up/ down fixations were used to analyze characteristics of oculomotor behavior in the three free-viewing tasks 310 (e.g., Thibault, Delerue, Boucart, & Tran, 2016). We were not only interested in showing that participants performed the task as requested but also in showing differences in oculomotor behaviors in the three free-viewing tasks performed either in steady or relaxed stances.

Hence, oculomotor behaviors were dependent variables and not independent variables. The first five seconds of data from each trial were not analyzed to withdraw initial transient sway (e.g., [START_REF] Bonnet | Deterministic and stochastic postural processes: Effects of task, environment, and age[END_REF][START_REF] Kinsella-Shaw | Effects of visual environment on quiet standing by young and old adults[END_REF]. Preliminary Pearson correlation analyses between the time series and age, height, and weight each showed some significant relationships. Therefore, interindividual differences in age, height, and weight had an effect on the recorded behavioral variability. For this reason, the data were all normalized in terms of age, height, and weight in using the detrending normalization procedure recommended by [START_REF] O'malley | Normalization of temporal-distance parameters in pediatric gait[END_REF]. This detrending procedure was used to eliminate the influence of changes in age, height, and weight on the findings.

For the COP, body, and eye variables, the mean of the two trials per task was calculated. Preliminary analyses of the data showed normal distribution, homogeneity of variance, and no outlier. Hence, two-way analyses of variance (ANOVAs) were performed with task and instruction as independent variables (p < .05). Additionally, post hoc Newman-Keuls tests compared the four visual tasks between each other in relaxed and steady stances separately (p < .05).

RESULTS

NASA-TLX Score

The ANOVA showed significant effects of instruction, F(1, 15) D 8.60, p < .05, and of task, F(3, 45) D 6.32, p < .05. The visual tasks were considered as more difficult in steady stance (10.60 § 3.45) than in relaxed stance (8.88 § 3.00). The cognitive workload was significantly higher in the stationary-gaze task (10.93 § 3.11) than in the three other tasks (white panel: 9.31 § 2.89; free-viewing 12 : 9.41 § 2.85; free-viewing 24 : 9.31 § 2.62, p s < .05) with no difference between these three latter tasks (p s > .33). The instruction by task interaction effect was not significant.

COP and Body Displacements

Analyses of Interindividual Variability

In summary, and for linear variables, there was significantly lower interindividual body linear variability in the free-viewing 12 task than in the stationary-gaze task for four variables (head SD AP ; neck SD AP ; neck V AP ; neck V ML ; Figures 2A, 2B, 2C, and2D). Additionally, the participants exhibited significantly higher interindividual body linear variability in the white-panel task than in the freeviewing 12 task for two variables (neck V AP ; Neck V ML ; Figures 2C and2D). Hence, there was significantly lower interindividual body linear variability in the free-viewing 4

Journal of Motor Behavior 365 12 task than in both stationary-gaze and white-panel tasks. For angular variables, there was significantly lower interindividual variability in the stationary-gaze task than in the free-viewing 12 task for both head SD yaw and lower back V pitch (Table 1). In contrast, there was significantly higher 370 interindividual body angular variability in the stationarygaze task than in the free-viewing 12 task for both neck V pitch and lower back R pitch (Table 1). For all variables, interindividual body variability were not significantly different between both white-panel (0.30 § 0.04 ) and sta-375 tionary-gaze task (0.19 § 0.03 ) but for the head SD yaw .

Analyses of Conventional Linear and Angular Displacements

Linear COP, head, neck, and lower back displacements. The effect of task was significant for two linear variables: COP R AP , F(3, 45) D 3.56, p < .05, and neck SD AP , F(3, 45) D 2.98, p < .05. For COP R AP , post hoc tests were not significant, both in steady and relaxed stances (p s > .18). However, the most important difference between conditions were found between the white-panel task and the free-viewing 24 task. For COP SD AP , individuals exhibited significantly greater neck SD AP displacement in the white-panel task (0.35 § 0.15 cm) than in the free-viewing 24 task (0.30 § 0.15 cm) only in steady stance (p < .05) Hence, classical dependent variables of COP and body sway, representing the amount of sway, did not show any difference between the stationary-gaze task and the three other tasks. Significant effects of instruction are shown in Table 2. No instruction by task interaction effect was significant.

Angular head, neck, and lower back displacements. The effect of task was significant for angular variables at the head level only. It was significant for head R yaw , F(3, 45) D 3.61,

Q2

for the interindividual body variability in linear displacement. The interindividual variability for each subject was calculated in subtracting the mean value of the time-series with the value of each subject, taken in absolute value. The four significant effect of task were found for (A) the standard deviation of the head displacement (in cm) in the anteroposterior axis (head SD AP ). (B) The SD of the neck displacement in the AP axis (neck SD AP ); the mean velocity (V) of the neck displacement in the AP axis (neck V AP ); and (D) V of the neck displacement in the mediolateral axis (neck V ML ). In the stationary-gaze (SG) task, participants looked at a black dot (1 of visual angle) projected on the wall in front of them. In the white-panel (WP) and free-viewing 12 (FV12) tasks, participants could freely explore a white panel projected in a circle of 12 or an image in a circle of 12 , respectively. The error bars represent the standard error of the mean. *Significant difference between two visual tasks (p < .05).

2016, Vol. 0, No. 0 p < .05 (Figure 3A); head SD yaw , F(3, 45) D 4.43, p < .05); head V yaw , F(3, 45) D 4.23, p < .05; head R pitch , F(3, 45) D 4.19, p < .05 (Figure 3B); and head SD pitch , F(3, 45) D 5.99, 400 p < .05. Post hoc tests were only significant in relaxed stance and showed that the participants exhibited significantly lower head R yaw (Figure 3A), lower head SD yaw (0.76 § 0.28 ), and lower head SD pitch (0.26 § 0.12 ) in the stationary-gaze task than in the three other tasks, that is, the white-panel (head 405 R yaw : Figure 3A; head SD yaw : 0.99 § 0.53 ; head SD pitch : 0.36 § 0.21), free-viewing 12 (head R yaw : Figure 3A; head SD yaw : 1.06 § 0.65 ; head SD pitch : 0.37 § 0.24) and free-viewing 24 (head R yaw : Figure 3A; head SD yaw : 1.18 § 0.65 ; head SD pitch : 0.42 § 0.26) tasks (p < .05). Furthermore, the participants exhibited significantly lower head R pitch in the stationarygaze task than in both free-viewing 12 and 24 tasks (Figure 3B; p < .05). Results for the effect of instruction are shown in Table 2. No instruction by task interaction effect was significant.

Oculomotor Behavior

In each task, the participants performed the visual task as requested. In both stationary-gaze tasks (in steady and relaxes stances), they only looked at the stationary dot. Some fixations The values in the table show the distance to the mean for each variable, not the initially calculated range (R), standard deviation (SD) and velocity (V) for each variable (see section 2.4 for more details). In the stationary-gaze task, participants had to stare at a black dot (1 of visual angle) throughout the trial.

In the white-panel and free-viewing 12 tasks, they could freely look at a white panel or at an image projected within 12 of visual angle. The data represent M § SD. Results of post hoc Newman-Keuls are shown by *, £, C. * represents a significant difference between the free-viewing 12 task and the stationary-gaze task; £ represents a significant difference between the white-panel task and the stationary-gaze task; C represents a significant difference between the white-panel task and the free-viewing 12 task. p < .05. Note. In steady stance, participants had to stand as steady as possible. In relaxed stance, they stood in a relax way but were instructed to avoid making any body movement unrelated to the task performed. The data represent M § SD. AP D anteroposterior; COP D center of pressure; ML D mediolateral. p < .05.

were performed away from the black dot (Figure 4A and3B) but only a few (SD of left-right and up-down fixation D 0.72 § 0.35 and 0.66 § 0.23 , respectively). In all other free-looking 440 tasks, the participants freely explored the images without looking outside of the circle (Figure 4A and4B). The larger the circle was, the more extended their exploration of the image was (Figure 4A and4B). The effect of instruction was significant for the range of left-445 right fixations, F(1, 15) D 6.30, p < .05, and for the mean duration of fixation, F(1, 15) D 20.48, p < .05. The participants performed shorter left/right visual exploration in steady stance (9.75 § 1.17 ) than in relaxed stance (10.76 § 1.18 ). The participants performed longer fixations in steady stance (0.86 § 450 0.24 s) than in relaxed stance (0.63 § 0.17 s). Moreover, the instruction by task interaction effect was significant for the mean duration of fixation, F(3, 45) D 4.66, p < .05. The greater the participants explored, the shorter their mean duration of fixation was (stationary gaze: 1.66 § 0.65 ; white panel: 0.54 § 455 0.20 ; free-viewing 12 : 0.44 § 0.08 ; free-viewing 24 : 0.34 § 0.05 ).

DISCUSSION

The present study first tested which of the white-panel, free-viewing 12 , and stationary-gaze tasks was the easiest task (i.e., the task requiring the lowest amount of cognitive workload and of interindividual postural sway variability). The results showed that both white-panel and free-viewing 12 tasks were cognitively less challenging than the stationary-gaze task. Moreover, there was significantly lower interindividual postural sway variability (COP or linear body displacements) in the free-viewing 12 task than in both other tasks. Therefore, the free-viewing 12 task could be assumed as significantly easier, less variable, than the stationary-gaze and white-panel tasks. Our second objective was to test if a free-viewing 24 task could increase head and/or body segment rotation and therefore significantly increase COP and body displacement. Our study did not confirm this assumption because the participants mostly moved their eyes even in this free-viewing 24 task.

Definition and Role of the Control Task

As explained by [START_REF] Fraizer | Methodological and interpretive issues in posture-cognition dual-tasking in upright stance[END_REF], the control task should provide baseline results to understand the effects of other manipulations (i.e., all sorts of precise visual tasks in our manuscript). To be considered as control, the task should be the easiest task [START_REF] Mitra | Postural costs of suprapostural task load[END_REF] and therefore lead to the minimum amount of cognitive workload and interindividual body sway variability. It should lead to 2016, Vol. 0, No. 0 a minimum amount of interference [START_REF] Swan | Improving balance by performing a secondary cognitive task[END_REF]. In 1983, a consortium of researchers decided that the station-485 ary-gaze task should be considered as a gold control task both because all participants perform exactly the same task in which they are expected to engage a minimum of interindividual body sway variability [START_REF] Kapteyn | Standardization in platform stabil-740 ometry being a part of posturography[END_REF].

Does the Stationary-Gaze Task Engage the Minimum 490 Amount of Cognitive Workload and Interindividual

Body Sway Variability?

Our results did not confirm the intuitive consensus [START_REF] Kapteyn | Standardization in platform stabil-740 ometry being a part of posturography[END_REF]) that the stationary-gaze task is the easiest task. Indeed, a first key finding is that the stationary-gaze 495 task was more cognitively demanding than the three other tasks, both in relaxed and steady stances (p < .05). Therefore, the methodological requirement to keep the eyes stationary or free has important consequences on the cognitive demand. Our results confirmed former reports that this sta-500 tionary-gaze task is constraining and may lead to higher cognitive workload (Ajrezzo et al., 2013;Ben Hamed et al., 2002) and higher attention [START_REF] Legrand | Differing effects of prosac-755 cades and antisaccades on postural stability[END_REF]. It is a difficult task [START_REF] Wade | Pioneers of eye movement research[END_REF] and in our study, it was actually the hardest task. Even more interesting, our main analyses 505 showed the stationary-gaze task led to significantly larger interindividual postural sway variability (Figures 2A, 2B, 2C, and 2D) than the free-viewing 12 task. This is a second key finding. Our secondary analyses with many classical dependent variables also showed that the free-viewing 12 510 task did not destabilize our group of healthy young individuals at all levels of the body (COP, head, neck, lower back), in all directions (AP, ML). We assume that the larger interindividual postural sway variability in the stationary-gaze task may be explained by 515 the higher cognitive workload (i.e., higher cognitive interference) in the stationary-gaze task than in the free-viewing 12 task. This higher interindividual postural sway variability in the stationary-gaze task may also be due to a higher variability in cognitive workload in that task. Indeed, some 520 participants could have thought a lot while other participants may not have done so in this stationary-gaze task [START_REF] Fraizer | Methodological and interpretive issues in posture-cognition dual-tasking in upright stance[END_REF]. This high variability in psychological thoughts may be better controlled in the free-viewing 12 task because participants are all engaged to perform an 525 exploratory task. Other criticisms of the stationary-gaze task exist in the literature reports. For example, the stationary-gaze task can quickly cause an attentional fatigue [START_REF] Rougier | La r ealization de mouvements de saccades oculaires affecte les strat egies de maintien de l' equilibre[END_REF] or it can be a boring task (Barlow, 1952).

530 Slight Angular Displacements in the Free-Looking Tasks Did Not Increase Postural Sway

The two free-looking tasks (white-panel and free-viewing 12 tasks) led to significantly greater amount of angular head displacement than the stationary-gaze task (Figures 3A and3B), with almost no difference between these two free-looking tasks. These findings were unexpected because the targets were projected lower than15 of visual angle [START_REF] Hallett | Eye movements[END_REF]. However, it should be noted that the differences with the stationary-gaze task were very small (differences in R pitch and R yaw < 0.90 ; differences in SD pitch and SD yaw < 0.17 ; Figures 3A and3B). Hence, these differences in task performance did not induce greater postural sway (COP, head, neck, lower back linear displacements) in the two free-looking tasks than in the stationary-gaze tasks.

How to Design Future Studies Including Both Control and Experimental Tasks?

Until this part of our manuscript, we only discussed that the control task should be the simplest task. Another important aspect in the definition of the control task is that it should be as similar to the experimental task(s) as possible but on one specific (tested) aspect. Hence, when different kinds of visual task are performed, the visual background should be identical, or equivalent, in both control and experimental tasks. We now discuss the most relevant control task in different paradigms of postural control.

If the experimental and control visual tasks both enables free eye motions on the same pictures, our study definitely shows that the ideal control task is a free-viewing task. For example, a free-viewing task could an ideal control task for another task in which participants are instructed to precisely detect something within the image (searching tasks; [START_REF] Shockley | Mutual interpersonal postural constraints are involved in cooperative conversation[END_REF]. Instead, the stationary-gaze task is definitely not a relevant control task for this experimental searching task because the stationary-gaze task leads to higher cognitive workload (cf. the NASA-TLX Score section) and interindividual postural sway variability (Figures 2A,2B,2C,and 2D) than the best control free-viewing task. Moreover, the visual background is different in both tasks.

If the experimental visual task imposes specific oculomotor behaviors (e.g., gaze shifts on dots projected at a certain amplitude and frequency; [START_REF] Rougier | Performing saccadic eye movements or blinking improves postural control[END_REF] or also in the task of counting the occurrence of letters in a text; [START_REF] Stoffregen | Modulating postural control to facilitate visual performance[END_REF], the most relevant control task may be the task of freely looking at a cubist transformation of these other experimental dots, or letters, or images [START_REF] Kapoula | Pictorial deph increases body sway[END_REF]. Indeed, the visual background would have the same number of pixels in both tasks and participants would be free to move their eyes in this control task. This free-viewing task would be better than the stationary-gaze task as a control task because it would avoid the cognitive workload to be biased.

The stationary-gaze task may be used adequately if participants stare at a black dot in all experimental conditions (e.g., [START_REF] Bonnet | Deterministic and stochastic postural processes: Effects of task, environment, and age[END_REF]. Here, the act of staring at a stationary target is a constant requirement and not an independent variable. However, one disadvantage of using the stationary-gaze task, we recall, is that it significantly increases the cognitive workload of all tasks. If investigators are willing to perform the easiest possible control task, a solution could be to let participants freely look at small images projected in a circle of 12 in all tasks, regardless of other manipulations superimposed on the free-viewing task.

In our study, first the participants exhibited significantly lower COP, head, neck, and lower back displacements when they stood in steady stance than in relaxed stance (Table 2), as classically shown in the literature reports [START_REF] Bonnet | Advantages and disadvantages of stiffness 695 instructions when studying postural control[END_REF][START_REF] Zok | Should the instructions issued to the subject in traditional static postur-835 ography be standardized?[END_REF]. Second, the steadiness requirement had problematic consequences on free oculomotor behaviors of the participants. Indeed, it significantly reduced the amplitude of the visual exploration in the left/right direction and increased the mean times of fixation although the participants were entirely free to look at the image. Moreover, the steadiness requirement had the disadvantage to significantly increase the cognitive workload (cf. the NASA-TLX Score section), as suggested by [START_REF] Zok | Should the instructions issued to the subject in traditional static postur-835 ography be standardized?[END_REF]. Overall, therefore, we do not recommend using this requirement in paradigms allowing free eye motions in one or several tasks.

Additional Findings in the Free-Viewing 24 Task

The free-viewing 24 task did not lead the participants to significantly increase their amount of body linear displacement than the three other tasks both in relaxed and steady stances (p D ns). It also did not lead the participants to significantly increase their amount of body angular displacement compared with the white-panel and free-viewing tasks both in relaxed and steady stances (p D ns). These findings were unexpected in the relaxed stance conditions because gaze shifts greater than 15 were expected to require head motions [START_REF] Hallett | Eye movements[END_REF] and, as a consequence, increase postural sway. A first explanation is that the participants only extended their visual exploration until 19.2 in the free-viewing 24 task, not 24 (Figures 4A and 4B). A second explanation is that eye rotations lower than 20 are almost entirely performed by the eyes [START_REF] Land | Looking and acting. Vision and eye movements in natural behavior[END_REF]. The lack of greater angular body rotations in the free-viewing 24 may explain why this task did not lead to significantly greater COP and/or body linear displacement than the other tasks. Overall, even this free-viewing 24 task was easier than the stationary-gaze task, both in relaxed and steady stances.

Limits and Conclusion

The present study was performed with healthy young adults and the message therefore only concerned this population. In older adults or patients, we could assume that the cognitive workload would be significantly higher in the stationary-gaze task than in the free-viewing 12 task because the second task should still be less constraining, less tiring, and less frustrating. However, we cannot be sure that older adults or patients may exhibit significantly lower interindividual body linear variability or may sway less in a free-view-645 ing 12 task than in a stationary-gaze task. Indeed, some age-related or disease-related physiological disabilities may modify the way upright stance and visual explorations are controlled. For example, some of these individuals may rotate their head significantly more in 650 the free-viewing 12 task than in the stationary-gaze task-thus significantly increasing their postural sway-to better scan the image in the free-viewing task simply because of an age-related reduction in their field of view. For these individuals, a free-viewing 8 655 or even stronger 4 task may be more adapted control tasks to avoid this issue.

In summary, our study discussed the importance of the control task in paradigms of postural control. This task should be well chosen to avoid irrelevant interpretations of 660 the data and to clearly detect and understand the presence and direction of significant findings. Our methodological study showed that the stationary-gaze task may not systematically provide the best baseline data to understand the effects of precise visual tasks on postural control. Instead, 665 free-viewing tasks or cubist transformation of the experimental images (depending on the paradigm) may better provide relevant baseline data, in taking great care that the visual backgrounds are the same, or equivalent, in all visual tasks. At the practical level, the free-viewing task is also 670 interesting as a control task because it is a more common, everyday life, activity than staring at a stationary dot. Hence, the comparison between the free-viewing task and the experimental task seems more relevant than the comparison between the stationary-gaze task and the experimental 675 task.

ACKNOWLEDGMENTS

The authors thank Professor Normand Teasdale for his help to improve the manuscript. FUNDING 680

The Polhemus system was funded by a research grant from the French National Research Agency (programme SHS2 ANR Lowvision).

Ajrezo L., Wiener-Vacher, S., & Bucci, M. P. (2013). Saccades 685 improve postural control: A developmental study in normal children. PLoS ONE, 8, e81066. http://dx.doi.org/10.1371/journal.pone.0081066 Barlow, H. B. (1952). Eye movements during fixation. Journal of Physiology, 116, 290-306. 690 Ben Hamed, S., Duhamel, J. R., Bremmer, F., & Graf, W. (2002).

Visual receptive field modulation in the lateral intraparietal area 2016, Vol. 0, No. 0

FIGURE 1 .

 1 FIGURE 1. (A) Stationary-gaze task: the participants had to stare at the black dot (1 of visual angle) throughout the full trial. (B) White-panel task: the participants first stared at the black dot (as in panel A) 5 s and then looked at the white panel within the circle 40 s (12 of visual angle); (C) Free-viewing 12 : the participants first stared at the black dot 5 s (dot surrounded by the image) and then freely explored the image within the circle 40 s (12 of visual angle). (D) Free-viewing 24 : the participants first stared at the black dot 5 s (dot surrounded by the image) and then freely explored the image within the circle 40 s (24 of visual angle). The images are not on scale. The images were shown in color, not in black and white as shown in Figure 1.

FIGURE 2 .

 2 FIGURE 2. Significant effect of task (p < .05) in the analysis of variance

FIGURE 3 .

 3 FIGURE 3. Significant effects of task (p < .05) in the analysis of variance for the range of the angular displacement of the head (in degrees or ) in (A) the yaw direction (head R yaw ) and (B) the pitch direction (head R pitch ). See Figure 2 for the details of the visual tasks. The error bars represent the standard error of the mean. *Significant difference between two visual tasks (p < .05).

FIGURE 4 .

 4 FIGURE 4. Significant effects of task (p < .05) in the analysis of variance for the eye range of fixations (in degrees or ) in (A) the left-right direction and (B) the up-down direction. See Figure 2 for the details of the visual tasks. The error bars represent the standard error of the mean. *Significant difference between two visual tasks (p < .05).

TABLE 1 .

 1 Significant main effect of task in the analysis of variance for the head, neck, and lower back angular displacements in range, standard deviation, and velocity in the angular yaw and pitch directions.

	Tasks	Main effects
		of task

TABLE 2 .

 2 Significant main effect of instruction in the ANOVA for the COP, head, and neck linear displacements (in cm) both in range and in standard deviation in the AP and ML axes and in the angular yaw and pitch directions.

	Instruction

Journal of Motor Behavior

Received March 14, 2016 Revised June 8, 2016 Accepted July 19, 2016 2016, Vol. 0, No. 0


François Gauthier-Clerc

Ashley Hill

Jean Laneurit

Roland Lenain

Éric Lucet

Online velocity fluctuation of off-road wheeled mobile robots: A reinforcement learning approach

During the off-road path following of a wheeled mobile robot in presence of poor grip conditions, the longitudinal velocity should be limited in order to maintain safe navigation with limited tracking errors, while at the same time being high enough to minimize travel time. Thus, this paper presents a new approach of online speed fluctuation, capable of limiting the lateral error below a given threshold, while maximizing the longitudinal velocity. This is accomplished using a neural network trained with a reinforcement learning method. This speed modulation is done side-by-side with an existing model-based predictive steering control, using a state estimator and dynamic observers. Simulated and experimental results show a decrease in tracking error, while maintaining a consistent travel time when compared to a classical constant speed method and to a kinematic speed fluctuation method.

I. INTRODUCTION

Mobile robotics attracts more and more interest in many applications such as agriculture [START_REF] Duckett | Agricultural robotics: The future of robotic agriculture[END_REF]. In these applications, autonomous systems need to be adaptive to the natural environment with its unpredictable and heterogeneous properties. Path following is necessary for a large number of agricultural tasks such as seed planting, irrigation, or plant treatment [START_REF] Fountas | Decomposition of Agricultural tasks into Robotic Behaviours[END_REF], [START_REF] Blackmore | Robotic agriculturethe future of agricultural mechanisation?[END_REF]. In these applications, accurate tracking is essential to avoid damage to vegetation. Travel time is also a criterion for maximizing robot availability and minimizing the overall cost of these solutions. Both of these requirements must be met by these tracking algorithms to ensure a successful application of robotics in agriculture.

A common approach for path tracking is to consider a constant speed tuned experimentally that is as fast as possible while ensuring a limit on the tracking error [START_REF] Bacha | Autonomous Vehicle Path Tracking Using Nonlinear Steering Control and Input-Output State Feedback Linearization[END_REF], [START_REF] Lenain | Accurate and stable mobile robot path tracking: An integrated solution for off-road and high speed context[END_REF]. This can make the system easier to control, but it has a significant impact on the optimality of the solution, both in terms of tracking accuracy and travel time. And the highly variable grip conditions in off-road context accentuates this issue, as certain portions the trajectory may require a very low speed. Therefore, in order to consider speed variations, the dynamics of the system must be taken into account, in particular for preventing wheel slip.

A planning approach can be used in order to implement an a priory online speed variation. This involves defining in advance either the trajectory and the speed for each key-point, or simply the speed on an already known static trajectory. This solution is often applied to robotic arms [START_REF] Debrouwere | Time-Optimal Path Following for Robots With Convex-Concave Constraints Using Sequential Convex Programming[END_REF], [START_REF] Bobrow | Time-Optimal Control of Robotic Manipulators Along Specified Paths[END_REF] and it can be extended to mobile robots.

A travel time minimization approach applied to a simple dynamic model of the vehicle [START_REF] Lipp | Minimum-time speed optimisation over a fixed path[END_REF] is also possible. This technique can consider other constraints such as energy consumption for an electric car [START_REF] Ebbesen | Timeoptimal Control Strategies for a Hybrid Electric Race Car[END_REF]. It guarantees an optimal travel time and a limited lateral error under the considered model. However, this technique is limited in an agricultural context as it does not take into account the evolution of the dynamics at the wheel ground interface.

Another solution of second order approximation on the acceleration constraints of a four-wheel model leads to an online planning that is able to manage the speed to guarantee a small tracking error [START_REF] Altche | High-speed trajectory planning for autonomous vehicles using a simple dynamic model[END_REF]. Considering the speed management as an upper limit definition makes the problem easier to solve. Thus a simple model can be used, based on curvature and speed limit [11], [START_REF] Park | Development of Steering Control System for Autonomous Vehicle Using Geometry-Based Path Tracking Algorithm[END_REF]. A model of the wheel ground interaction can also be considered [START_REF] Braconnier | Ensuring path tracking stability of mobile robots in harsh conditions: An adaptive and predictive velocity control[END_REF], [START_REF] Braconnier | High speed path tracking application in harsh conditions: Predictive speed control to restrict the lateral deviation to some threshold[END_REF], [START_REF] Hach | Avoiding steering actuator saturation in off-road mobile robot path tracking via predictive velocity control[END_REF]. This estimate is based on the angular velocity which predicts a situation of under-steering or over-steering. However, all these methods are not sufficiently reactive to the online variation of parameters, in particular grip conditions.

Overall, the previously described solutions are limited to simple models that do not integrate the online evolution of wheel ground grip conditions. However, the online estimation of such slip parameters and cornering stiffnesses is essential to guarantee the stability of the system operating in highly variable environments.

Recent advances in machine learning and more specifically in reinforcement learning (RL) provide an efficient approach to derive a model from a set of input data and an appropriate reward function [START_REF] Sutton | Learning to predict by the methods of temporal differences[END_REF]. These methods take advantage of advances in deep learning that allow to use a neural network as a universal function estimator [START_REF] Hornik | Multilayer feedforward networks are universal approximators[END_REF]. Contributions have applied such RL approach to a path following task [START_REF] Baltes | Path Tracking Control of Non-holonomic Car-Like Robot with Reinforcement Learning[END_REF], [START_REF] Zhang | Neural-network-based reinforcement learning control for path following of underactuated ships[END_REF], [START_REF] Kamran | Learning Path Tracking for Real Car-like Mobile Robots From Simulation[END_REF], these examples showing the relevance of a learning process to solve mobile robotics problems.

The contribution proposed here is a neural network based online velocity controller. It is associated to an existing front steering Model-based Predictive Controller (MPC) for path tracking, that was previously implemented with a constant speed [START_REF] Braconnier | High speed path tracking application in harsh conditions: Predictive speed control to restrict the lateral deviation to some threshold[END_REF]. This task was previously addressed with a traditional approach [START_REF] Braconnier | High speed path tracking application in harsh conditions: Predictive speed control to restrict the lateral deviation to some threshold[END_REF], [START_REF] Braconnier | Ensuring path tracking stability of mobile robots in harsh conditions: An adaptive and predictive velocity control[END_REF] which demonstrated the problem's complexity and the limits of this approach. The RL algorithms are used to solve this problem with more flexibility, as they are based on a data derived from a fully dynamic model with varying grip conditions.

The speed control solution is implemented in the following, a mobile robot dynamics simulator being used to train a neural network with reinforcement learning techniques in order to generate a speed modulation algorithm from available inputs. This is achieved by minimizing an objective function of travel time and tracking errors.

This method is evaluated in simulated and real environments to assess its performance and limitations. For this, the methodology and models are described in section II, followed by the simulation and simulated results in section III, then continued with the experimental setup and results in section IV, and then finally concluding and discussing future works in section V.

II. METHOD

In this section, after a brief reminder of the steering control law, the problem of speed fluctuation during path tracking is discussed. First, a deterministic kinematic solution is proposed. Then, a new RL approach is investigated. To this end, an objective function is defined and the problems of actuator response time and neural network outputs are addressed.

A. MPC steering controller

The path following strategy is from previous works on the design of a front steering controller in agricultural context [START_REF] Lenain | Adaptive and Predictive Path Tracking Control for Off-road Mobile Robots[END_REF]. This MPC is based on a bicycle kinematic model in the horizontal plane, extended by taking into account the tyre slip angle. The following variables are expressed in the Frenet frame (see Fig. 1).

δF = tan(βR) + L cos(βR) c(s) cos( θ) k + A cos 3 ( θ) k 2 +βF (1)
with

   k = 1 -c(s)y θ2 = θ + β R A = -K p y -K d k tan( θ) + c(s) k tan 2 ( θ)
The variables used are the following:

• s, c(s) the curvilinear abscissa of the robot projection on the path and the associated curvature, • y, θ the lateral and angular deviations, • β F , β R the front and rear slip angles, • K p , K d the proportional and derivative gains, • L the wheelbase. Both slip angles β F,R and cornering stiffness coefficients C F,R are estimated online during the path following using a dynamic observer [START_REF] Deremetz | Adaptive trajectory control of off-road mobile robots: A multi-model observer approach[END_REF]. This observer formulation is based on a linear approximation of a Pacejka model [START_REF] Bakker | Tyre modelling for use in vehicle dynamics studies[END_REF] for the expression of lateral forces F F,R = C F,R β F,R . This way, cornering stiffnesses reflect lateral grip properties of the encountered terrain.

As a varying speed will be used, the steering controller's K p and K d gains must evolve accordingly in order to maintain a constant reactivity of the controller, with respect to the speed value. Thus, a linear interpolation is performed in real-time between predefined expert gains for different values of static speeds and cornering stiffnesses.

One observation is that the steering controller used does not consider a pure dynamic model. This choice is motivated by the fact that a dynamic model involves too many parameters that are complex to estimate and very variable, especially in the agricultural context [START_REF] Lucet | Dynamic path tracking control of a vehicle on slippery terrain[END_REF]. A deterministic method for speed modulation with a kinematic model is introduced, in order to compare to the RL approach. This technique is derived from a limitation on the steering rate δF max (similar to [START_REF] Wolek | Time-optimal path planning for a kinematic car with variable speed[END_REF] ). The following assumptions are made: y and θ are assumed to be zero, which means that the robot is expected to follow its path correctly. By deriving the angular error with respect to the curvilinear abscissa, the following equations are successively deduced:

∂θ ∂s = tan(δ F ) L (2)
∂ 2 θ ∂s 2 = 1 Lv (1 + tan 2 (δ F )) ∂δ F ∂t (3)
And then, by replacing tan(δ F ) with L.c(s) (as the lateral and angular error as assumed to be zero):

∂δ F ∂t < δF max ⇒ ∂ 2 θ ∂s 2 < 1 + L 2 c(s) 2 Lv δF max (4)
Considering that ∂ 2 θ ∂s 2 = ∂c(s) ∂s and the previous equation ( 4), it is possible to determine a positive maximum speed depending on the desired maximum steering rate:

v max = 1 + L 2 c(s) 2 L ∂c(s) ∂s δF max (5)
As such, given a desired maximum steering rate δF max that limits the dynamic behavior in real world experiments, a reasonable maximal speed v max is obtained. However, since the rate of change of v max can be greater than the maximum acceleration of the robot, a predictive speed modulation is performed to ensure that the robot speed remains below the defined v max . This speed controller does not consider the estimated cornering stiffness but it does provides a relevant comparison, since the speed setpoints generated are optimal in a kinematic context.

C. Description of the selected reinforcement learning approach

The TD3 [START_REF] Fujimoto | Addressing function approximation error in actor-critic methods[END_REF] reinforcement learning method is considered, in order to train a suitable speed controller based on a neural network (NN). It is an off-policy RL method that was chosen based on empirical results compared to alternative RL based methods (neural networks trained with PPO [START_REF] Schulman | Proximal policy optimization algorithms[END_REF] and TRPO [START_REF] Schulman | Trust region policy optimization[END_REF] obtained lower performance than TD3).

A dynamics simulator of the robot along with a sufficient diversity of trajectories, allows for off-line training of the neural network using the TD3 method. Then the trained neural network is used online, in order to generate speed setpoints and ensure a fast and accurate path following performances (see the global control architecture Fig. 2).

During the online uses, there are no changes to the weights and biases of the neural network. This velocity fluctuation management is formulated as a Markov Decision Process (MDP) with a discrete time sampling. The following neural network inputs are considered:

• y, θ the path tracking lateral and angular errors.

• v, θ the measured speed and yaw rate.

• v max the maximum allowed speed. The measured yaw rate is included in the input data it is an important indicator of vehicle stability [START_REF] Braconnier | Ensuring path tracking stability of mobile robots in harsh conditions: An adaptive and predictive velocity control[END_REF]. The curvature is also a crucial variable for speed modulation as it defines the admissible yaw rate and speed (see deterministic alternative section II-B). This curvature variable is extended to a vector over a fixed time horizon, allowing for the anticipation of future curvatures and estimation of its rate of change. The estimated cornering stiffness is a direct indicator of the slip dynamics at wheel ground interface. It can therefore be used by the neural network to estimate the ability of the vehicle to follow its trajectory at a given speed. Whereas the estimation of slip angles alone does not allow the method to anticipate the sliding behavior, but instead allow for the correction in case of undesirable slips.

A path episode starts with a stationary robot at the beginning of a considered path. The episode ends either when the robot reaches the end of the path (successful episode), or when it reaches a state that does not respect a hard constraint (failed episode). The lateral error in this approach is defined as a hard constraint, where exceeding a maximal lateral error causes the episode to end prematurely with a low reward.

1) The reward function: RL algorithms aim to maximize a discounted cumulative reward along an episode [START_REF] Sutton | Learning to predict by the methods of temporal differences[END_REF]. Both traveling time and lateral error limitation must be considered in this reward function in order to correctly train the neural network model.

The reward is defined in accordance with the speed that has to be maximized, while avoiding hard constraints. The reward for the whole trajectory is the sum of the rewards at each step. In this case, a linear speed-dependent reward should be avoided since the number of steps in an episode depends directly on speed, and thus the sum of the rewards reflects the length of the trajectory, which is a desired optimization criterion. However, a square function can be used to solve this issue (see equation ( 6)), and to improve the convergence of the RL algorithm a constant negative reward -C te is added in order to encourage higher speeds.

R t = v 2 /v 2 max if |y|< y lim -C te else (6)
While this addition creates discontinuity, it reinforces the behavior of the policy to avoid exceeding the lateral error limit. Without this penalty, a sub-optimal policy that would rather fail at a higher speed, rather than reach the end of the path at a lower speed may emerge. The learning process on a trajectory continues until the mean episodic reward is no longer increasing.

2) Restoring the Markov hypothesis and observability of the system.: A progressive variation of the speed (limited acceleration) and the frequency of the target controller defeat the convergence of RL algorithms. Indeed, the small discretized time interval (10Hz) leads to the cancellation of the traditional RL exploration using Gaussian noise and the actuator response time affects the Markov property (causal link) [START_REF] Raffin | Generalized state-dependent exploration for deep reinforcement learning in robotics[END_REF], [START_REF] Korenkevych | Autoregressive policies for continuous control deep reinforcement learning[END_REF].

An action skipping is often used to address both issues and significantly improve the learning process (as it was done in previous contributions [START_REF] Mnih | Playing atari with deep reinforcement learning[END_REF]) by artificially increasing the time scale during the exploration and learning process. Rather than sampling the policy at each step, the neural network evaluation is performed every k steps and kept the same action between each evaluation. This allows the exploration to maintain a consistent control setpoint (convergence of the speed towards its desired value) and to reduce the actuation delay.

In addition, the neural network can output an oscillating speed setpoint, with a potentially infinite variation. For this reason, the neural network output is interpreted as an acceleration setpoint instead of a speed setpoint. This reduces the output noise and makes it possible to add a constraint on the acceleration. Then, an integration is performed to retrieve the value of the speed setpoint. And, to preserve the observability of the system, this speed setpoint is added to the list of the following inputs.

Action skipping and acceleration output integration can both be applied in the same process by maintaining acceleration setpoint between each step instead of speed setpoint. In this way, the action skipping has no impact on the final shape of the exploration process, which is very close to an optimal policy. Fig. 3 illustrates the difference in shape between a direct speed control at 10Hz, an applied action skipping (k = 10), and an acceleration output consideration with the same action skipping.

III. SIMULATION

In this section, the training process using a dynamic simulator is described. Then, the constant speed, the kinematic

A. Training setup

The simulator used to calculate the TD3 algorithm is parameterized in accordance with properties of the testing robot presented in section IV. This robotic platform is a car-like robot called Robufast (presented in [START_REF] Lenain | Accurate and stable mobile robot path tracking: An integrated solution for off-road and high speed context[END_REF]). It is an electrically powered 4-wheel drive robot capable of reaching a speed of 8 m/s in off-road conditions. It can perform a maximum acceleration/deceleration of 1.5m s -2 with a maximum steering angle of 20 • . Three maximum speed values are considered for training (4, 6, 8m s -1 ). A deceleration phase is added on the top of the speed modulation to ensure a safe stop when the path end is almost reached.

The dynamic simulator integrates a linear approximation of Pacejka model with cornering stiffnesses of 7000 to 25000N rad -1 . A second-order model of actuator delay is also implemented in order to model the steering behavior.

The training path set is defined in order to represent a large set of possible paths with similar lengths, to avoid sub optimal policy, and to conform to the given specifications. In particular, the range of curvatures (0-0.3m -1 ) and maximum turn number (3) are fixed. Path shape and grip conditions are re-sampled using a uniform law at each episode beginning to ensure changing conditions during training, allowing for the generalization of the neural network for the task.

The training is performed under a Python application linked by a C++ hook to the simulator implemented in C++ for performance and practicality. Stable baseline [START_REF] Hill | Stable baselines[END_REF] is used as a RL library in order to get a reliable TD3 implementation. The actor and the critic neural networks both get the same architecture, which is composed of 2 hidden layers of 64 units, an output layer of 1 units and a state dimension of 32 units. tanh is the activation function used for the whole networks. The TD3 hyperparameters used are :

• number of steps: 2.5M • gamma: 0.

B. Comparative analysis

For the speed output computation with the kinematic speed modulation method, the δF max value is defined empirically in order to obtain a travel time equivalent to that of the neural network based method. This tuning is done for each path.

To compare the performance of each of the methods, three metrics are introduced. The error peak represents the maximum magnitude of the lateral error during the whole path following task. The surface error outside of a corridor is defined by the criterion A off in Eq. [START_REF] Bobrow | Time-Optimal Control of Robotic Manipulators Along Specified Paths[END_REF]. And the total surface error A err is defined as a special case of the A off metric, with a threshold of y lim = 0m.

A off = N n=0 |y(tn)|≥y lim ṡ(t n ) y(t n ) + ẏ(t n ) dt 2 -y lim dt [m 2 ] (7) with: ṡ(t) = v(t) cos( θ(t)) ẏ(t) = v(t) sin( θ(t))
A path set is used to gather an average result over multiple simulations (paths that are not used during the training process). This set is composed of 14 paths with a length of around 200m each. Trajectories are categorized in 3 groups according to the curvature peaks. The first group A gathers all paths with curvature peaks below 0.05m -1 , the second one (group B) with curvature peaks from 0.05 to 0.12m -1 , and the last one (group C) with curvature peaks from 0.12 to 0.3m -1 .

Table I gathers the results obtained with the testing path set under good grip conditions. The neural network based speed controller outperforms the other methods over all the metrics. In accordance with the method design, the neural network method keeps its peak errors very close to the threshold lateral error of 0.2m, and therefore obtains an surface error above the threshold almost equal to zero.

Whereas both the static speed and the kinematic methods struggle to keep the peak errors below the 0.2m threshold. Even though the steering MPC is designed for constant speed, the average speed is too high for proper path following during high curvature parts. This effect is even more pronounced on paths with high peaks in the curvature. The peak error increases by 170% from group A to group C.

The kinematic controller is successful in reducing the total surface error with a modulation based on the curvature. The benefit of this method increase as the peak curvature rises when compared to the static speed method. This behavior is mainly due to a lower average speed in this context, which limits the dynamic effects. Even though the kinematic controller is more efficient than the constant speed method, the neural network based controller achieves a lower surface error in the traj C category.

Fig. 4 shows the results for the category C paths using the three considered methods. The Kinematic and neural network controllers generate similar speed profiles, However two key differences are apparent. First, the neural network controller is reacting to the dynamic phenomena by decreasing the speed near 80m due to the lateral error increase, with the lowest speed value reached at the curvature peak. This speed reduction helps to correct the angular error induced by the curvature and keeps the lateral error below the threshold (0.2m). In contrast, the kinematic method does not adapt to this situation and the lateral error starts increasing near 90m, and continues to increase until 105m.

Secondly, the neural network controller is exploiting the characteristics of the steering MPC, as it has learned to work with it during the learning process. At the beginning of a high-speed turn, the steering MPC is slightly understeering and then an oversteer to minimize lateral error. This behavior is visible around 50m with the static speed and the neural network controllers. Due to the fact that the kinematic controller is based on the curvature rate, it is reducing the speed value before the turn without regard to the MPC behavior, therefore the MPC cannot function optimally. In summary, the NN speed controller has learned the subtleties of dynamics in order to avoid oversteer and understeer due to the large slip angles induced by the difficult conditions. In contrast, the kinematic speed does not take these into account, which justifies the results obtained. Further research could involve a pure dynamic MPC steering controller using an ML-based model to obtain a fairer comparison with the proposed RL speed control.

IV. EXPERIMENTAL RESULTS

Further tests were performed with the Robufast experimental platform (Fig. 5). Those tests aims to proove the transferability of a simulated training applied onto a real application. This comparison with and without the speed modulation, combined with the MPC steering controller, highlights the value of the proposed method.

The neural network speed controller used for this experiment was trained to avoid lateral errors above a threshold of 0.4m. It was then tested along with a constant speed strategy, over the trajectory as shown in Fig. 6 & Fig. 7. Special care was taken for these tests, in order to generate a trajectory different from those of the training process. This was done in order to verify the generalization of the neural network speed controller to different trajectories. Each test was run several times to validate the repeatability of the results, which are presented below.

Initially, the robot was positioned with a lateral error of more than one meter. The constant speed controller was set to 4m s -1 , as this achieves a similar travel time when compared to the neural network speed controller.

Results from Fig. 7 show that the neural network method is capable of maintaining the lateral error below the 0.4m threshold, whereas the constant speed controller which has a similar travel time could not achieve a similar performance. Over the surface error metric (Eq. 7 with y lim = 0), the neural network method has 20.35m 2 of surface error, while constant speed method has 32.98m 2 of surface error. This shows a reduction of 38.3% in the neural network method's surface error when compared to the constant speed method. As such, the neural network method was capable of significantly reducing the overall surface error and preserving the 0.4m lateral error threshold with a similar travel time, by modulating the target speed in real time.

From Fig. 7, the neural network method seems to be decelerating the robot before each turn (red area on the velocity plot). This shows the neural network is anticipating the behavior of the robot during turning, as decelerating prevents strong dynamic effects from occurring. Furthermore, this shows that the neural network is safely adapting the speed value to the new states that were observed, which demonstrates the transferability of the method to real world experiments, as the simulation used during the training process was not a perfectly accurate robotic model.

The proposed neural network method also obtained good reactive behavior to external perturbation. At 60m and 90m a strong lateral error occurs due to the terrain quality in the corners, these lateral errors cause the neural network method to decelerate in order to give more time to the steering controller, allowing the robot to converge while limiting the amplitude of the lateral error.

At 100m, an oversteer occurred at the exit of the second turn, leading to large angular and lateral errors with a With the neural network method however, the speed setpoint was reduced from 3.6m s -1 to 1.9m s -1 at 93m, while the steering controller reduced the steering angle to about 5 • . Both actions in tandem, allowed the system to converge back to a low lateral error with minimal perturbations. After the lateral error returned to acceptable values, the steering controller restored the original steer angle and the neural network method started a new acceleration phase. This modulation of the speed by the neural network minimized the error that was induced from the environment, allowing for a stable path following with minimal error. In order to validate the importance of cornering stiffness inputs, a comparative experiment was done. The results of said experiment are displayed on Fig. 8. The neural network method that was trained without the cornering stiffnesses obtained considerably worse performance over the lateral error with a lower speed. From s = 10 to the end of the trajectory, the neural network with the cornering stiffnesses obtained a surface error of 2.77m 2 , while the neural network without the cornering stiffnesses obtained a surface error of 4.89m 2 , a 43.4% reduction in the surface error. This implies that the cornering stiffnesses are necessary in order to anticipate the future dynamic behavior of the next corner, and to slow down if necessary or to speed up if possible.

While a strict constraint over lateral error can be met in the simulated environment, in real application it is more difficult due to external disturbances such as GPS signal inaccuracies or terrain topology, as they can induce unmodeled lateral errors. Those disturbances cannot be anticipated and can be difficult to accurately model. As such this limits the reliability of the solution, and safety margins must be taken into account when defining the lateral error threshold.

High speed experiments have also shown unmodeled peaks in the lateral error that occurred in a deceleration phase during the corner (e.g. the spike at s = 60m). This implies some load transfer is occurring during the deceleration that is increasing the vertical force on the front wheels. The change in the vertical force causes a drastic change in the steering and the tyre grip, causing the system to react considerably faster, and as such causing oversteer which in turn produces higher error than those experienced in simulation.

V. CONCLUSIONS AND PERSPECTIVES

A reinforcement learning approach was proposed for speed fluctuation over a path following task. This method consists in training a neural network in order to control the acceleration for the speed setpoint to be as high as possible while maintaining the lateral error below a fixed threshold.

The neural network trained with a wide variety of paths, perception qualities and grip conditions was successful in setting the speed for both simulation and during real experiments. The simulator, based on a vehicle dynamic model, was able to train a model that is capable of obtaining favorable performance at high speed, in a real application without the use of transfer learning. This approach using a simple lateral error constraint can easily be extended to other hard constraints, such as a maximum angular deviation. Furthermore, the methodology is independent from the steering system and can be applied to other robot architectures with very little modifications.

However, some drawbacks appeared during the experiments. The longitudinal dynamic interactions may be considered in order to ensure an improved path following with less oversteering. And an improved steering controller designed to take into account speed variations, might improve the performance substantially.

Future works will consider a specific speed control for each wheel motor in order to increase the rotation speed of the system, similar to a previous deterministic approach [START_REF] Lenain | Accurate and stable mobile robot path tracking: An integrated solution for off-road and high speed context[END_REF]. Transfer learning techniques may also be involved to improve results by considering the real world wheel/ground dynamics. Merging this approach with previous work in online gain setting [START_REF] Hill | Online gain setting method for path tracking using cma-es: Application to off-road mobile robot control[END_REF] is also being considered for future contributions.

Fig. 1 .

 1 Fig. 1. Kinematic bicycle model projected on the frenet frame B. A deterministic kinematic speed constraint solution

Fig. 2 .

 2 Fig.2. The new RL approach (red and orange parts) as part of the overall path following control architecture.

  • δ odom F , δ setpoint F the measured and setpoint steer angles. • β F , β R the measured front and rear slip angles. • C F , C R the front and rear estimated cornering stiffnesses. • c(s) the path curvature.

Fig. 3 .

 3 Fig. 3. A comparison of the three exploration techniques and the NN-based methods are compared to each other, using each case the same MPC steering controller.

Fig. 4 .Fig. 5 .

 45 Fig. 4. Speed modulation obtained with the selected methods

Fig. 7 .Fig. 8 .

 78 Fig. 7. Velocity and lateral error in the real world experiment

Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France firstname.lastname@cea.fr

Université Clermont Auvergne, Inrae, UR TSCF, Centre de Clermont-Ferrand, F-63178 Aubière, France firstname.lastname@inrae.fr