Guillaume Mornieux

D Gehring

P Furst

Albert Gollhofer

Dominic Gerhring

Patrick Fürst

Keywords: Anticipatory postural adjustments during cutting manoeuvres in football Anticipative strategies during cutting manoeuvres change of direction, trunk, knee abduction moment, head rotation, step width Introduction through conclusion, 4197

Anticipatory postural adjustments during cutting manoeuvres in football and their consequences for knee injury risk

Introduction

Changing the direction or avoiding obstacles is an integral part of many team sports as for instance football. It is evident that preparatory strategies, so-called anticipatory postural adjustments (APA), are necessary for a successful movement execution [START_REF] Jindrich | Maneuvers during legged locomotion[END_REF][START_REF] Patla | Online steering: coordination and control of body center of mass, head and body reorientation[END_REF][START_REF] Xu | Anticipatory postural adjustments for altering direction during walking[END_REF]. Basically, changing direction requires a reorientation of the body together with the impulse vector to move the centre of mass (CoM) in the direction of the intended movement. This could be achieved by three main strategies involving the whole body.

Firstly, the foot placement via the step width regulation plays an important role [START_REF] Patla | Online steering: coordination and control of body center of mass, head and body reorientation[END_REF][START_REF] Vallis | Locomotor adjustments for circumvention of an obstacle in the travel path. Experimental brain research[END_REF][START_REF] Wheeler | Modification of agility running technique in reaction to a defender in rugby union[END_REF]. Indeed, the placement of the foot determines the position of the centre of pressure and defines in which direction the CoM can be accelerated [START_REF] Patla | Online steering: coordination and control of body center of mass, head and body reorientation[END_REF]. Secondly, head orientation is used to adjust the visual field and to initiate the reorientation of the whole body [START_REF] Hicheur | Head motion in humans alternating between straight and curved walking path: combination of stabilizing and anticipatory orienting mechanisms[END_REF][START_REF] Sreenivasa | Walking along curved paths of different angles: the relationship between head and trunk turning. Experimental brain research[END_REF][START_REF] Vallis | Children use different anticipatory control strategies than adults to circumvent an obstacle in the travel path. Experimental brain research[END_REF]. Finally, the control of trunk motion is crucial [START_REF] Sreenivasa | Walking along curved paths of different angles: the relationship between head and trunk turning. Experimental brain research[END_REF][START_REF] Vallis | Children use different anticipatory control strategies than adults to circumvent an obstacle in the travel path. Experimental brain research[END_REF]. The trunk yaw (rotation about the long axis of the body) orientates the body in the new direction. Moreover, the trunk roll (lateral flexion) is used to move lower limbs and upper body in opposite directions, which allows moving the CoM towards the new direction during the stance phase [START_REF] Patla | Online steering: coordination and control of body center of mass, head and body reorientation[END_REF], especially when the change of direction occurs late. It is worth noting that APAs must already be planned and initiated in the step before changing the direction [START_REF] Patla | Online steering: coordination and control of body center of mass, head and body reorientation[END_REF][START_REF] Vallis | Children use different anticipatory control strategies than adults to circumvent an obstacle in the travel path. Experimental brain research[END_REF].

Changes of direction can be performed differently, e.g. side-stepping, cutting or crossover manoeuvres during various sport disciplines. The way APAs initiate and organize the movement (manoeuvrability) influences the performance [START_REF] Jindrich | Maneuvers during legged locomotion[END_REF]. For instance it has been reported that higher skilled rugby players demonstrate faster changes of direction than less skilled players [START_REF] Gabbett | Reactive agility of rugby league players[END_REF].

But manoeuvrability is also associated with injury risk [START_REF] Jindrich | Maneuvers during legged locomotion[END_REF]. Indeed, lateral movements with changes of direction may lead to misalignments in the lower extremities with the consequence that the knee may be at a relatively high risk of injuries, such as anterior cruciate ligament ruptures [START_REF] Alentorn-Geli | Prevention of non-contact anterior cruciate ligament injuries in soccer players. Part 1: Mechanisms of injury and underlying risk factors. Knee surgery, sports traumatology, arthroscopy[END_REF][START_REF] Olsen | Injury mechanisms for anterior cruciate ligament injuries in team handball: a systematic video analysis[END_REF]. As football is the most commonly practiced sport in the world, the injury concern in this activity is even higher than in other sports [START_REF] Alentorn-Geli | Prevention of non-contact anterior cruciate ligament injuries in soccer players. Part 1: Mechanisms of injury and underlying risk factors. Knee surgery, sports traumatology, arthroscopy[END_REF], which underlines the need for researchers to keep improving the body of knowledge dealing with knee injury mechanisms and risk factors associated with football practice. Moreover, knee injuries are often experienced in an unpredictable setting, as for instance perturbed landing or insufficient time to prepare the movement [START_REF] Olsen | Injury mechanisms for anterior cruciate ligament injuries in team handball: a systematic video analysis[END_REF]. Experimental results revealed that knee joint loading, especially knee valgus moment, was greater during unpredictable conditions and consequently may have increased the risk of knee injuries during these cutting manoeuvres [START_REF] Besier | Anticipatory effects on knee joint loading during running and cutting maneuvers[END_REF][START_REF] Houck | Comparison of frontal plane trunk kinematics and hip and knee moments during anticipated and unanticipated walking and side step cutting tasks[END_REF].

Therefore, the influence of unpredictable conditions on APAs during cutting manoeuvres and their consequences for knee injury risk are of interest. Increasing the body of knowledge dealing with APAs would help to further understand how performance and safety could be improved during such tasks typically encountered during football games. In addition to the unpredictability of the upcoming change of direction, the time available to prepare the movement plays an important role as recently proposed by [START_REF] Brown | Sex and limb differences in hip and knee kinematics and kinetics during anticipated and unanticipated jump landings: implications for anterior cruciate ligament injury[END_REF]. Although anticipatory postural adjustment strategies have been fairly well described during changes of direction, their limitations have not been investigated so far during critical situations, i.e. during a cutting manoeuvre with little time available to prepare.

The aim of the present study was to determine which APAs are the most relevant during cutting manoeuvres in football and which fail when time available to prepare the movement is limited. We hypothesized that football players would increase their lateral trunk flexion when the time available to prepare the cutting manoeuvre was reduced. Finally, we aimed to evaluate the consequences for knee injury risk resulting from these APA failures.

Materials and methods

Participants

Using data from the literature evaluating unanticipated movements [START_REF] Brown | Sex and limb differences in hip and knee kinematics and kinetics during anticipated and unanticipated jump landings: implications for anterior cruciate ligament injury[END_REF], the sample size was estimated to achieve 80% statistical power with an alpha level of 0.05. Accordingly, thirteen male amateur football players (age: 24 ± 3 years old; height: 1.79 ± 0.03m; mass: 73.8 ± 7.4 kg; football experience: > 10 years in the 5 th -7 th German league) participated in the study. Prior to testing, all participants were informed about possible risks and gave written informed consent. Participants did not have a previous history of serious knee injury or any current knee pain. All procedures were approved by the local ethics committee.

Protocol

Football players were asked to perform three different tasks in a randomized order, including a cutting manoeuvre to 45°, a crossover to -20° and a straight run with an approach running speed of 5 ± 0.2 m.s -1 . The tasks were performed on a force plate and were followed by a maximal sprint over a 4m distance.

The direction of the movement was indicated by a light signal occurring at different times before the left foot contacted the force plate. The first condition was anticipated (Ant), meaning that participants already knew which light would turn on before starting to run. For this condition the light stimulus appeared 850ms before ground contact. The three other cutting conditions were performed without this knowledge, while the light stimulus was presented 850ms, 600ms and 500ms before ground contact. The rationale for these delays was determined during pilot measurements.

850ms was considered long enough to prepare the movement in a similar manner to the Ant condition. In this condition the light stimulus appeared approximately 3 steps (during right foot contact) before the change of direction. 600ms was felt by the participants to be a typical time constraint comparable to game situations and the signal occurred during the initial contact of the left foot two steps prior to changing the direction. From this value, we decreased the time available until players started to fail in the task execution, but not in more than 20% of the trials. In this condition (i.e. 500ms condition), the light stimulus occurred during the push off phase of the left foot contact two steps prior to the change of direction.

The order of the four time conditions was randomly assigned over the 13 participants.

Participants performed the movements on an artificial football turf (Ecofill®, Mondo S.p.A., Alba, Italy) while wearing the same neutral football shoe (Predator, Adidas, Herzogenaurach, Germany). All participants were used to such movements due to their football practice and were also familiar with the light stimulus procedure, as they had already participated in studies using this testing setup.

During the present study, participants had to perform about 80 running and cutting movements to ensure eight successful 45° cutting manoeuvres for each time condition.

Participants had at least 30 seconds recovery after each trial but could rest as long as they wanted to avoid the onset of fatigue.

The 45° cutting manoeuvre was the only movement to be further analyzed due to its relevance when investigating performance and injury aspects of changing direction tasks (see introduction). A trial was defined as successful if the approach speed was reached, the left foot hit the force plate and the sprint was conducted in the direction of the light stimulus already one step after the force plate.

Material

The approach running speed was measured over the last 3m before the force plate (Timer S3, Alge-Timing, Palling, Germany) and the sprint velocity over the first 3m after the force plate were recorded by means of light switches (M18 series, Panasonic Electric Works Europe AG, Holzkirchen, Germany).

The visual light stimulus was triggered automatically via a light switch (M18 series, Panasonic Electric Works Europe AG, Holzkirchen, Germany), triggered during the approach run. The three lights were placed at eye level in the directions of the three tasks, 7m away from the forceplate. Three-dimensional head, trunk and leg kinematics were recorded using reflective skin markers ( 14mm) placed with self-adhesive tape on anatomical landmarks of the participants' head, trunk, pelvis and both right and left legs. Specifically, markers were placed around the head (four markers), on the suprasternal notch and the transition between the body of sternum and the xiphoid process as well as on the T6 vertebra, on the anterior and posterior superior iliac spines, on the great trochanter, medial and lateral epicondyle of the knee, tibia (two markers) and the medial and lateral malleolus of the left leg, and on the posterior, medial and lateral aspects of the heel cup as well as the toe of both feet. This marker placement was based on previously published articles for the head [START_REF] Sreenivasa | Walking along curved paths of different angles: the relationship between head and trunk turning. Experimental brain research[END_REF] and the lower limb [START_REF] Fleischmann | Load-dependent movement regulation of lateral stretch shortening cycle jumps[END_REF]. The marker positions for the trunk segment were developed based on the conclusions reported by [START_REF] Leardini | Quantitative comparison of current models for trunk motion in human movement analysis[END_REF]. Markers were captured with a 12-camera motion analysis system (Vicon V-MX, VICON Motion Systems Ltd., Oxford, UK) at a sampling frequency of 200 Hz. A static trial with the subject standing in a pre-defined neutral position was used to define the kinematic model and to calculate segment lengths and joint centres [START_REF] Grood | A joint coordinate system for the clinical description of three-dimensional motions: application to the knee[END_REF][START_REF] Wu | ISB recommendation on definitions of joint coordinate system of various joints for the reporting of human joint motion-part I: ankle, hip and spine[END_REF]. Joint kinematics in three rotational degrees of freedom were determined using an yxz Euler rotation sequence of the respective segment coordinate system. Marker trajectory and ground reaction force (BP600900, AMTI, Watertown, USA) signals were both filtered with a low pass Butterworth filter (4 th order, 20 Hz cut-off frequency) prior to calculating external joint moments with a standard inverse dynamics approach.

Data analysis

Kinematic parameters were first investigated at the end of the right foot contact, i.e. one step prior to the change of direction, as the body posture at this moment would represent the APAs and would also help to understand the organization of the transition phase between APAs and the moment of the change of direction. The right heel strike and toe off were determined based on the peak vertical acceleration signal of the heel marker and toe marker, respectively. Head rotation, trunk rotation and lateral trunk flexion with respect to the global coordinate system as well as step width were calculated (for positive orientation see Figure 1).

Then the kinematic parameters were analyzed at the time of the peak knee abduction moment during the left foot contact during the cutting manoeuvre, as this is a relevant parameter associated with knee injury [START_REF] Hewett | Biomechanical measures of neuromuscular control and valgus loading of the knee predict anterior cruciate ligament injury risk in female athletes: a prospective study[END_REF]. Again, head rotation, trunk rotation and lateral trunk flexion as well as step width were determined. In addition, the foot progression angle was defined as the rotation of the calcaneus in relation to the global coordinate system. Moreover, hip and knee joint moments in the frontal plane were analyzed. **** Figure 1 near here ****

Statistics

The selected biomechanical parameters were averaged across eight trials for each subject in each cutting condition and served as the basis for the statistical analysis (SPSS 20.0, IBM SPSS Statistics, IBM Corporation, Armonk, NY, USA). After having confirmed that the data followed a normal distribution (Kolmogorov-Smirnov test), main effects of cutting condition (Ant vs. 850ms vs. 600ms vs. 500ms) on the dependent biomechanical variables were tested using a one-way analysis of variance (General Linear Model) with repeated measures. Pearson's correlation coefficient was used to determine the relation between lateral trunk flexion and peak knee abduction moment during the cutting manoeuvre [START_REF] Jamison | Knee moments during runto-cut maneuvers are associated with lateral trunk positioning[END_REF]. The level of significance was set at 0.05. If a statistically significant main effect was observed, LSD multiple comparison procedures determined where the differences occurred. All results are presented as group means ± SD.

Results

One step before changing the direction, there was a significant influence of the cutting condition on head rotation (p=0.033), trunk rotation (p=0.002) and lateral trunk flexion (p<0.001), whereas step width was not reduced (p=0.081) (see Table 1).

In the cutting condition with 600ms the head was significantly less rotated to the right compared to both Ant and 850ms (p=0.033 and p=0.028, respectively), while higher trunk rotation to the left was found for both 600ms and 500ms compared to Ant and 850ms (p<0.026) and higher lateral trunk flexion values were reported for both 600ms and 500ms than Ant and 850ms (p<0.001). None of these parameters were significantly different between Ant and 850ms, as well as between 600ms and 500ms.

At the time of the peak knee abduction moment, there was a significant influence of the cutting condition on head rotation (p=0.015), trunk rotation (p=0.028) and lateral trunk flexion (p<0.001), whereas step width and foot progression angle were not altered (Table 2).

During the cutting condition with 600ms the head was significantly less orientated towards the right compared to both Ant and 850ms (p=0.032 and p=0.018, respectively). The 500ms and 600ms cutting conditions yielded larger trunk rotation to the left than 850ms (p=0.036 and p=0.043, respectively). Compared to both Ant and 850ms, lateral trunk flexion values were significantly higher for 600ms (p<0.002) and 500ms (p<0.001). Additionally, the lateral trunk flexion was higher for 500ms than 600ms (p=0.015). None of these parameters were significantly different between Ant and 850ms.

Mean values of the parameters dealing with the loading of the lower limb at the time of the peak knee abduction moment are reported in Table 3. Along with the reduction of the time to prepare the cutting manoeuvre, there was a significant increase of the posterior (p<0.001) and vertical ground reaction forces (p<0.001). The cutting condition significantly influenced both hip (p<0.001) and knee abduction moments (p<0.001). Moreover, the contact time was influenced by the cutting condition (p<0.001).

Again, none of these parameters were significantly different between Ant and 850ms.

Compared to Ant and 850ms, posterior ground reaction force was significantly higher for 600ms (p=0.005 and p=0.024, respectively) and 500ms (p<0.001 for both), while 500ms even yielded a higher posterior ground reaction force than 600ms (p=0.04). The vertical ground reaction force during the cutting condition with 600ms was significantly higher than Ant and 850ms (p=0.003 and p=0.015, respectively). In the same manner, performing the cutting manoeuvre with 500ms resulted in higher vertical ground reaction forces than both Ant (p=0.001) and 850ms (p=0.006). The 500ms cutting condition yielded increased hip abduction moment compared to Ant (p=0.002) and 850ms (p=0.007), as well as increased knee abduction moment compared to Ant (p=0.001) and 850ms (p=0.003). The same observation could be made for 600ms, which, compared to Ant and 850ms, generated higher hip and knee abduction moments (p<0.009 and p<0.011, respectively). Compared to Ant and 850ms, contact time values were significantly higher for 600ms (p=0.007 and p=0.002, respectively) and 500ms (p<0.001 for both). Additionally, the contact time was longer for 500ms than 600ms (p=0.008).

To further understand the influence of lateral trunk flexion on knee abduction moment, variations from the Ant condition baseline in the three unpredictable cutting conditions for these two parameters were plotted together (Figure 2). Lateral trunk flexion significantly correlated with peak knee abduction moment (r=0.41; p=0.009).

Sprint velocity remained comparable over the four cutting conditions, ranging from 5.0±0.4 m.s -1 for Ant to 4.9±0.3 m.s -1 with a delay of 500ms.

Discussion

The primary finding of the present study was that APAs are modulated according to the time available to prepare a cutting manoeuvre. Participants relied on a higher lateral trunk flexion to perform cutting manoeuvres under limited time available. While this was an effective strategy to maintain the sprint performance, higher lateral trunk flexion was associated with increased knee abduction moment. Finally, as soon as the time available was sufficient (850ms), not knowing in advance in which direction one would have to cut did not influence APAs, knee joint loading or sprint performance.

Of the three main APA strategies proposed in the literature, lateral trunk flexion was the most influenced by the cutting condition. Head and trunk rotations were also influenced by the time available to prepare the cutting manoeuvre, whereas the step width regulation was not significantly altered.

With less time available, lateral trunk flexion increased by 10.3° (500ms vs. Ant) already one step prior to the cutting manoeuvre and by 8.4° at the time of the change of direction 150ms later. This finding is in agreement with data from previous studies [START_REF] Houck | Comparison of frontal plane trunk kinematics and hip and knee moments during anticipated and unanticipated walking and side step cutting tasks[END_REF][START_REF] Patla | Online steering: coordination and control of body center of mass, head and body reorientation[END_REF]. Lateral trunk flexion was even more increased in the present study compared to the +3.7° found by [START_REF] Houck | Comparison of frontal plane trunk kinematics and hip and knee moments during anticipated and unanticipated walking and side step cutting tasks[END_REF] when participants performed a 50° cutting task under unpredictable conditions but with a lower approach velocity of 1.9 m.s -1 . The higher lateral trunk flexion can be seen as a strategy to move the CoM in the new direction. A neuromuscular explanation for this increased lateral trunk flexion strategy could be the relative higher strength of the trunk and hip muscles compared to other rather weak muscles (e.g. ankle invertors/evertors) during the stance phase. Therefore, participants could rely more on their trunk than on any foot placement strategy.

Head and trunk rotations were influenced to some degree by the time restriction to prepare the cutting manoeuvre. With less time available, the head was less orientated towards the cutting direction and the trunk was even more rotated in the opposite direction at the time of changing the direction as well as one step before. The of the trunk in the opposite direction has already been reported in the literature during cutting manoeuvers [START_REF] Dempsey | The effect of technique change on knee loads during sidestep cutting[END_REF][START_REF] Dempsey | Changing sidestep cutting technique reduces knee valgus loading[END_REF]. The influence of the cutting condition on head and trunk rotations was limited, as for instance no significant difference was observed between the 500ms and Ant conditions at the time of the peak knee abduction moment or given that standard deviation values for the head (7° to 13°) and the trunk (7° to 11°) were large. The relatively small influence of the time available on these APAs further shows the weak importance of head and trunk rotation strategies during cutting manoeuvres.

The step width strategy depends on the foot placement. One step prior to changing the direction, a medial shift of the foot placement with respect to the CoM would allow the centre of mass to accelerate towards the new direction. However, this strategy tended to be less used under critical time conditions (< 600ms). Moreover, the subsequent foot contact when changing the direction was not modulated, i.e. the foot was not placed more laterally to increase the acceleration of the CoM in the cutting direction. This result is in agreement with [START_REF] Patla | Online steering: coordination and control of body center of mass, head and body reorientation[END_REF] and partially consistent with [START_REF] Wheeler | Modification of agility running technique in reaction to a defender in rugby union[END_REF] and [START_REF] Houck | Comparison of frontal plane trunk kinematics and hip and knee moments during anticipated and unanticipated walking and side step cutting tasks[END_REF]. Indeed, as soon as the change of direction was unpredictable, the first authors reported that the step width strategy was significantly less used one step prior to changing the direction and the latter authors found a reduction of the step width at the time of the cutting manoeuvre.

Moreover, less use of the step width strategy one step before the change of direction was associated with reduced lateral movement speed [START_REF] Wheeler | Modification of agility running technique in reaction to a defender in rugby union[END_REF].

Therefore, the step width strategy does not seem to be the one to increase or to rely on during challenging cutting manoeuvres.

Accordingly, altering lateral trunk flexion is probably the most relevant postural strategy to successfully perform a cutting manoeuvre under unpredictable conditions and to maintain the subsequent sprint performance at the same level.

Despite obvious main effects of the cutting condition on APAs, it is worth noting that further statistical analysis revealed no significant difference between Ant and 850ms.

Accordingly, not knowing in advance which cutting manoeuvre will have to be executed does not influence APA strategies as long as the time available to prepare the movement is long enough. Under the present conditions, i.e. 5m.s -1 approach velocity and 45° cut angle, a time of 850ms (3 steps prior to the change of direction) was enough to use postural strategies comparable to those observed under predictable conditions.

Whereas lateral trunk flexion was increased during successful cutting manoeuvres under limited time to prepare the movement, the overall leg loading as well as the joint loading increased. Indeed, posterior and vertical ground reaction forces at the time of the peak knee abduction moment (roughly 20-25ms after foot strike) were significantly higher for both the 600ms and the 500ms cutting conditions. This higher loading of the leg could increase the difficulty of the task, requiring significantly longer contact time and explaining also why 20% of the cutting manoeuvres with 500ms time available were not successful.

Moreover, hip and knee joint moments increased in the frontal plane as soon as the time available decreased to 600ms. This observation is in agreement with previous results showing increased joint loading under unpredictable conditions [START_REF] Besier | Anticipatory effects on knee joint loading during running and cutting maneuvers[END_REF][START_REF] Brown | Sex and limb differences in hip and knee kinematics and kinetics during anticipated and unanticipated jump landings: implications for anterior cruciate ligament injury[END_REF], although [START_REF] Dempsey | Changing sidestep cutting technique reduces knee valgus loading[END_REF] reported no significant influence of unpredictability on knee joint moments under conditions comparable to the present study (approach velocity, cutting angle and light stimulus timing). The large peak knee abduction moment values (up to 1.24 Nm.kg -1 ) are consistent with recently published data [START_REF] Vanrenterghem | The effect of running speed on knee mechanical loading in females during side cutting[END_REF] and underline the high loading of the knee and the possible risk for knee injury. The substantial new information provided by the present study is that not only cutting angle or approach velocity or even the unpredictability about the cutting condition influence knee joint loading but also the time available to prepare the cutting manoeuvre.

The increased lateral trunk flexion due to the reduction of the time available explained approximately 17% of the variance in peak knee abduction moment and should therefore be considered as an important parameter when investigating knee injury mechanisms (Figure 2). This association confirms previous observations dealing with the mechanical coupling between trunk and knee joint in the frontal plane [START_REF] Dempsey | Changing sidestep cutting technique reduces knee valgus loading[END_REF][START_REF] Hewett | The mechanistic connection between the trunk, hip, knee, and anterior cruciate ligament injury[END_REF][START_REF] Houck | Comparison of frontal plane trunk kinematics and hip and knee moments during anticipated and unanticipated walking and side step cutting tasks[END_REF][START_REF] Jamison | Knee moments during runto-cut maneuvers are associated with lateral trunk positioning[END_REF].

Thus, it can be concluded that altered APA strategies under unpredictable conditions are one factor leading to an increased knee joint injury risk during cutting manoeuvres, along with, for instance, less suitable neuromuscular activity of the knee joint muscles [START_REF] Besier | Muscle activation strategies at the knee during running and cutting maneuvers[END_REF].

Accordingly, the aim of future studies could be to provide solutions for this excessive use of lateral trunk flexion. Reducing the use of this strategy would then probably need to be compensated by other anticipatory postural strategies to maintain the subsequent sprint performance at the same level. For instance it is known that head and trunk rotation strategies tend to be reduced with maturation [START_REF] Vallis | Children use different anticipatory control strategies than adults to circumvent an obstacle in the travel path. Experimental brain research[END_REF]. Therefore, training these strategies again to improve the re-orientation of the whole body based on a head and trunk rotation might be a relevant solution. Another solution could be a modified foot placement. [START_REF] Dempsey | Changing sidestep cutting technique reduces knee valgus loading[END_REF] have already reported a reduction of knee joint loading associated with a specific whole body cutting technique, characterized by a medial shift of the foot together with a more neutral lateral trunk flexion at the time when changing the direction. However, even if a more externally rotated foot placement, i.e. the foot progression angle, might be a strategy to reduce knee joint frontal plane loading [START_REF] Fleischmann | Load-dependent movement regulation of lateral stretch shortening cycle jumps[END_REF][START_REF] Sigward | Loading characteristics of females exhibiting excessive valgus moments during cutting[END_REF], this parameter was not influenced by the cutting conditions in the present study. Therefore, modulation of the foot rotation does not seem to be an appropriate strategy to prepare a cutting manoeuvre under limited To conclude, this study provides a detailed biomechanical and functional analysis of anticipatory postural adjustment strategies during cutting manoeuvres in football.

From the variables analysed in the present study, increasing lateral trunk flexion is the most effective strategy to successfully perform a cutting manoeuvre when less time is available to prepare the movement. But this is also associated with increased knee injury risk. Considering knee injury as a serious issue in football, training programs need to improve anticipatory postural adjustments to reduce injury risk and possibly enhance performance.

Figures captions

  time available. It has to be emphasized that any cutting technique modification through a training intervention would probably be more effective if attention is brought to the step prior to the change of direction, whichever strategy, i.e. head rotation, trunk rotation or step width regulation, is trained. The relevance of the present results can be transferred to daily football training. Firstly, training should put emphasis on decision making situations, where players have to identify ball trajectories, teammate or opponent displacements that are not pre-defined and make the appropriate decision accordingly (relevant timing and direction of the pass or displacement). Secondly, training should focus on situations where players experience one against one situations (either by attacking or defending) and where changes of direction are required. During these situations, players would focus on their trunk rotation and the lateral trunk flexion to operate such cutting manoeuvres. The upper body should not only be kept straight but orientated towards the new direction of the cutting manoeuvre. Moreover, players should pay attention to the alignment of the leg and the upper body already one step prior to the change of direction.

Figure 1 :

 1 Figure 1: Schematic representation of one subject performing the 45° cutting

Figure 2 :

 2 Figure 2: Linear regression between peak knee abduction moment and lateral trunk

Tables

 Table 1Ant 850ms 600ms 500ms

Head rotation (°) 11.1 ± 9.1 10.9 ± 9.9 6.5 ± 10.5 * a 6.9 ± 7.1

Trunk rotation (°) -7.1 ± 8.4 -7.0 ± 8.5 -10.8 ± 9.3 * a -11.1 ± 7.3 * a Lateral trunk flexion (°) 1.9 ± 5.2 3.6 ± 4.7 9.8 ± 5.4 * a 12.2 ± 5.7 * a

Step width (m) -0.24 ± 0.06 -0.27 ± 0.08 -0.23 ± 0.09 -0.21 ± 0.11 * a significant differences compared to Ant and 850ms cutting conditions (p<0.05) Lateral trunk flexion (°) 5.3 ± 6.1 6.1 ± 6.4 10.4 ± 6.4 * a 13.7 ± 6.9 * a,c

Step width (m) 0.36 ± 0.03 0.35 ± 0.04 0.36 ± 0.03 0.35 ± 0.03

Foot progression angle (°) 7.8 ± 7.8 9.2 ± 7.9 6.9 ± 9.7 6.3 ± 9.8


Xiaohong Chen

C Matin

Music assisted language learning

come

1 Introduction Language learning system is very complex. In an oral language acquisition process, the facial expressions may act, the body movement may intervene, the emotions may influence. Even to learn to produce a single word, many related subsystems may come to play a role. Although the quantitative studies in the research laboratories attract more and more attention, the complex learning mechanisms are still far from being understood. Practical, more qualitative complementary approaches are still necessary for providing research materials, view angles and verification. In a previous work we have described a Binomial Learning methodology [START_REF] Perdereau | A binomial learning methodology[END_REF] which involved mainly social interaction and has shown efficiency for almost all the participants. We expected a similar homogeneity in the music assisted language learning method, to our surprise this method has its limit.

Practical approach

We are interested in the oral production of comprehensive phrases. The learners are students, people in working activities and retired people. Their mother tongue is French. The target language is Chinese Mandarin in which the learners are beginners. When music was introduced in our lessons for the first time, we were surprised by the inhomogeneity: a number of people are not interested in singing or even do not like music, so that the music related method was not suitable for them, especially for some of them who are interested only for written language learning. For this reason the participants have chosen to be separated into one of the two groups, a singer group in which the music assistance is welcomed, and a speaker group in which the music assistance is absent. The number of participants in each group was inferior to 25.

It is well known that the speech and music materials must be consistently structured [START_REF] Schön | [END_REF]. Our learning material were inspired from the concepts developed in the anterior theoretical works on the music and language structures hierarchies [START_REF] Lerdahl | A Generative theory of tonal music[END_REF]. As a concrete example, we take two short phrases from Soirée Zou [START_REF] Rouge | Soirée Zou[END_REF]:

Sequence 1 : Very glad to meet you. Sequence 2 : Me too, very glad to meet you. The phonetic transcription (pinyin without tones) is the following :

Sequence 1 : hen gao xing jian dao nin Sequence 2 : wo ye hen gao xing jian dao nin The corresponding music associated to Sequence1 and Sequence2 is vocal melody, extracted from the same source.

In the singer group, the sequence learning takes at least two lessons.

Lesson 1 contains successively a speech sequence listening, a singing sequence familiarization, a singing sequence production, a speech sequence production, with correction or consolidation and a validation sequence. Its duration depends on the participants number and the circumstance.

Lesson 2 contains a revision sequence, a consolidation sequence, some exercises with variations and extensions. Usually Lesson 2 lasts longer than Lesson 1. An interval of a week separates 2 lessons, which is constrained by the academic schedule. In the speaker group, the sequence learning is similar without singing sequences.

Observations and discussions

In a short time interval of a week, beyond the individual oral production performance, we observed a general tendency : people in the speaker group pay more attention to the passage from syllables to words and from words to sentences. Their production is less fluid but they get better meaning of the individual words. On the contrary, in the singer group, people tend to produce a sentence as a whole. They have better fluency and pronunciation, but there are more people in this group who do not associate the oral production with the word meaning. A common trait for both groups exists: the mastering of the Sequence2 after that of Sequence1 was straightforward with or without music.

Subsystem considerations

To discuss something in foreign language learning process, we usually have to simplify. In this section we oversimplify. We consider only Sequence1 and Sequence2 and the corresponding speech and song cognitive subsystems involved in the music and speech interactions in the oral production. We denote Subsing(Music, Sequence1) the subsystem involved in the music production in Sequence1 by singing. We denote Subsing(Parole, Sequence1) the subsystem involved in the parole production in Sequence1 by singing. We denote SubSpeak(Parole, Sequence1) the subsystem involved in the parole production in Sequence1 by speaking. We denote SubSpeak(Parole, Sequence2) the subsystem involved in the parole production in Sequence2 by speaking.

Music and speech interactions

The interaction between the speech and the music subsystems may occur at the following stages. i. At the beginning : The speech sequence is perceived together with the music, the relevant rhythm, pitch and melody are important for familiarization [START_REF] Perdereau | Speech rhythm processing[END_REF][START_REF] Perdereau | Decrease of pitch perception ambiguity in tone language processing[END_REF], the latter allows successive music notes to be anticipated by a singer. We postulate that the elements associated with the successive music notes in the subsystem Subsing(Music, sequence1) are activated before the corresponding music note is orally produced. The corresponding successive syllables in the subsystem Subsing(Parole, sequence 1) are expected to act as "followers" to the music notes. However this process is not automatic. Although individually one may search internal factors for the speech syllables to follow the music notes, we call the external cues for help. The most efficient way we found is the group oral production. Often the singer group claims to sing again and again. Upon the members appreciation, when they sing in a group the wrong pronunciations are perceived quickly by themselves, so that quickly self-corrected. This is probably the main reason for which the singers get self-confidence.

ii. During the interval week without lesson : the singer group members says that some times after the lesson they repeat by singing alone for pleasure. Without music they did not like to repeat. We were among researchers interested in the relationship between cognitive subsystems involving music and reward [START_REF] Perdereau | Mirror Binomial Language Learning[END_REF]. Recent studies on neuroscience confirmed that music can act as a reward for learning [START_REF] Gold | [END_REF]. Our results support the idea that music may assist song parole repetition through the reward system, the subsequent consolidation in turn improves the pronunciation and the phrases fluency. As the music structure properties were designed to be highly consistent with the speech structure properties, it is not surprising that the training of Subsing(Parole, Sequence1) and Subspeak(Parole, Sequence1) leads to fluid phrases mastering in the singer group.

iii. After a week, at the moment of the speech reproduction, a part of the singer group members recalled firstly the song of Sequence1 before to reproduce the speech of Sequence1. Here the music acts as an additional cue for the speech subsystem memory retrieval. This role played by the music seems in line with psycho-cognitive research results in the field of health science. [START_REF] Perdereau | Decrease of pitch perception ambiguity in tone language processing[END_REF].

Conclusions

We described a practical approach using quality structured material to include music in the beginner's learning of a foreign language. By splitting participants into two groups, we have observed that learners in the singer group were on average more fluent and at ease with the oral speech phrases production. Based on the observations and the learners comments analysis we conclude that the music assisted learning offers benefits on getting self-confidence throughout the learning process, in three stages music aids language learning: 1. in the perception stage through collective singing stimulation, 2. in the consolidation stage through rewarded repetition and 3. in the memory retrieval stage as an additional cue. We also observed the inhomogeneity in music sensitivity among learners. This limits the music assisted language learning to be learner specific.

Acknowledgments

The author thanks Sciences and Techniques UFR and Physics department of Burgundy University for providing working conditions.


Jajwalya R Karajgikar
email: jajwalyak@gmail.com

Amira Al-Khulaidy

Anamaria Berea

Computational Pattern Recognition in Linear A

Keywords: Linear A, Information Theory, NLP, Topic modeling, Clustering, Knowledge Mining, Humanities and Social Sciences/Literature, Humanities

Linear A is an ancient Mycenaean Greek language that is as of yet undeciphered. Computational analysis of the symbols using natural language processing and data mining techniques can aid to better understand this logogram language, and to help us uncover interesting statistical and information-theoretic patterns even without deciphering the language. Using information theory and data science techniques, we show an exploratory analysis of these symbols. We use n-gram analysis to summarize and predict the possible symbols on lost stone tablets, while symbols are clustered by topic modeling as well as k-means, for comparison. The results show that there are universal features in this language that we can explore further with computational methods.

INTRODUCTION

The historical context of the ancient languages may be lost to time but some of the archaeological evidence found in the form of stone tablets (as in our case), contain information that can be considered for the possibility of computational study. In the case of undeciphered languages, such as Linear A, many researchers turned to new computational techniques such as machine learning and natural language processing with the aim to decode them. But fewer authors have approached these languages from the data science and information theory perspective, with the aim to discover statistical and informational patterns which, even in the absence of a decipherment, would help us understand better the universality of human language, both ancient and modern. From the data science perspective, languages are repositories of information that hold quantitative value in addition to the meaning that we are assigning them from the human cultural perspective. Undeciphered languages represent a type of agnostic data that can help us design better, unbiased algorithms, particularly for natural language processing methods. Therefore, in this paper, our main goal is to extract meaningful emergent patterns of vocabulary from an independent primary source, Linear A. British Archeologist Arthur Evans discovered the hieroglyphic seal-stones with successive types of script around 1893 along the eastern Mediterrennean basin, which culminated in the dramatic excavation of the palace hill of Mycenaean Knossos on the Greek island of Crete [START_REF] Evans | Scripta Minoa: The written documents of Minoan Crete, with special reference to the archives of Knossos[END_REF]. Dated from around 1400 BCE, the ancient Minoan civilization inscriptions are one of the earliest forms of writing discovered. Evans was the first to define the pictographic Cretan hieroglyphics, and the subsequent linear of classes A and B. The unique structure of the ancient palace site was such that Evans believed the possibility of tracing the whole evolution of the art of writing all the way up to the Neolithic times, thereby establishing its importance in historical linguistics and art history. The Linear A writing system was prevalent in the Bronze Age Minoan civilization from 1800 to 1450 BC. It was the primary script used in palace and religious writings of the Minoan civilization. The etchings on the clay tablets found in the excavation are called logogram symbols 2 and form the basis of our dataset that will be run through several analysis techniques before they are available for subject matter expertise' discernment and approval. This research work is an attempt at taking a step on the identification, simulation, and study of language families origins and evolution with constellations of computational methodology.

I RELATED WORKS

Literature Survey

While Linear A is as yet undeciphered, the Linear B syllabary has been computationally considered (Papavassiliou et al., 2020 ) via smoothed n-gram analysis, Hidden Markov Models, Bayesian classifiers, and Conditional Random Fields (CRF). This paper proposes to conduct an exploratory approach to conducting data mining on an undeciphered language. Linear B has been challenging to decipher due to reasons similar to Linear A's. There is a limited amount of data that can be used for processing since there are only so many archaeological remains from the historical evidence of the Linear A language, which makes it a greater challenge to decipher than Linear B. Furthermore, there is no real way of knowing if the output will be accurate without validation data or material. A machine-learning system capable of deciphering lost languages [START_REF] Luo | Deciphering Undersegmented Ancient Scripts Using Phonetic Prior[END_REF] was demonstrated by having it decipher Linear B-the first time this has been done automatically. The approach used was very different from the standard machine translation techniques by using a novel neural decipherment approach for automatic decipherment. The outcome accuracy can be verified and validated against existing Linear B literary decipherment. In the below Linear B tablet facsimile 3 (see figure 1, right), the following transliterations 4 are known: men (ideogram5 VIR), women (ideogram MUL), girls (ko-wa) and boys (ko-wo). This is available because the women and children of Knossos were identified by the occupational, geographical, or ethnic names. The DĀMOS or Database of Mycenaean at Oslo (Aurora, et al., 2013) is a corpus of complex architecture just for Linear B syllabary that is standardized by the TEI epigraphic standard and Leiden convention that contains 5900 documents of metadata, textual markup, annotation, and modularity in a relational database MySQL workbench. The purpose is to translate each lemma, disseminate knowledge of the language, and its irregularities. (Chadwick, et al., 1987). Right, the transliterated text of the KN As 1516(Chadwick, et al., 1987) There are rich interdependencies between various unknown languages and their applicability in unraveling mysteries of modern evolutionary biology studies [START_REF] Daggumati | Data mining ancient scripts to investigate their relationships and origins[END_REF]. Genomic data availability lies in the gray area of abundance and sparseness depending on the kind of algorithm at work. The construction of phylogenetic trees uses algorithms like Neighbor joining method, Maximum Likelihood, Common Mutations Similarity Matrix, which are comparatively objectively free of human bias [START_REF] Daggumati | Data mining ancient scripts to investigate their relationships and origins[END_REF]. The ClustalW2 cladogram with 2 phylogenetic tree generation algorithms using both Neighbour Joining method and UPGMA method (unweighted pair group method with arithmetic mean) give the similarity index between various comparable language models.

Figure 2: Evolutionary Tree of ancient scripts reconstructed using Neighbor Joining algorithm in

ClustalW2 [START_REF] Daggumati | Data mining ancient scripts to investigate their relationships and origins[END_REF] As such, languages with the same proto-language structures having similar distribution profiles is not unknown to linguists. The sequence-to-sequence model capturing character-level correspondence between cognates using minimum cost flow problem, that is Long Short Term Memory algorithm. There are many more research papers to consider going forward but these provide a guideline to previous methodology and are indicative of the expected results.

Research Question(s)

Can we use natural language processing and data mining methods to create a meaningful taxonomy of vocabulary from a sparse dataset showing patterns of a language that is yet to be deciphered? To specify, the question is: What are the frequently occurring patterns in the Linear A corpus that are also similar in form?

Objectives

• Computational linguistic preprocessing and analysis via vector embeddings and similarity analysis via K-Means Clustering. • Methods for the detection of intertexts and text patterns using Zipf Law and Entropy possibilities.

Motivation

We are interested in this particular problem for its applications in historical Computational Linguistics research. The multidisciplinary nature of the project is exciting for gaining knowledge in several domains in Digital Humanities. Furthermore, algorithms used with Natural Language Processing are applicable in a variety of fields like Evolutionary Biology Genomics. This is part of the wider project on studying the origins of socio-linguistic complexity combining natural language processing, information theory, data science techniques, and eventually even agent-based modeling.

II Model

Dataset

Preparing the Data

The Linear A data was available in JSON format in the Github repository (mwenge / lineara.xyz, 2019). The original .ods file was converted into a series of .csv extension files for easy machine readability, based on each attribute in the ODS extension file 6 . To read the inscriptions in Linear A and Linear B, special unicode font character installations are required: NotoEmojiRegular.ttf 7 , NotoMono-Regular.ttf 8 , NotoSansLinearA-LinearB.ttf 9 . When text is 9 https://github.com/googlefonts/noto-fonts 8 https://github.com/googlefonts/japanese/tree/main/build/fonts/noto 7 https://github.com/googlefonts/noto-emoji rendered by a computer, the characters in the text can not all be displayed, because no font that supports them is available to the computer. When this occurs, small boxes called "tofu" are shown to represent the characters. The Python package Matplotlib also does not render the font as of the current version 3.3.4.

Dataset Schema

Now the files can be opened to show the attribute features: Name, Parsed Inscription, Transcription, Translated Words

Preprocessing the Data

The first step in the preprocessing pipeline is the fragmentation of the text blocks into sentences. However, in the case of our dataset, the values are in dataframe format and not in larger blocks of text. It is possible to consider the possible sentence boundaries depending on the transliterated pieces. During the initial steps of the pre-processing pipeline, the content is fragmented into sentences. In numerous dialects and languages --for example, English --accentuation, particularly the full stop or period character, outcry ,and question marks can be utilized to distinguish the finish of the sentence. However in the case of our dataset, we are unsure of the actual sentence boundaries since tablets are not indicative of word tokenization although it has been considered within the probable "words" column of the dataset. This is a major challenge in both processing the data and deciphering the language in general.

System Framework

Exploratory N-Gram Analysis

The purpose of this module is to explore the Linear A dataset for better understanding of the distribution of symbols in the corpus. We implemented different NLP techniques such as generating bigrams, trigrams, predicting symbols based on weighted frequency and generating a summary of the dataset. The diagram (see Figure 4) shows the process flow of this module. The process starts with importing required Python libraries nltk. The bigrams and trigrams collections are collocations that help us learn of the summary and predicted symbols in the missing or lost stone tablets. Calculating the word weighted frequency generates a summary of the dataset and predicting symbols. The frequency distribution of the symbols gave us an interesting Lexical Dispersion plot10 (see Figure 5). The frequency of a word across the corresponding parts of the corpus highlights the word offsets.

Figure 5: Lexical Dispersion plot for the most frequently occurring symbols

The plot of frequency of the terms is a short tail distribution, that is universal with Zipf's Law 11 parameters (see Figure 6). The terms considered for summary using n-gram analysis where (n-1) terms give us the nth terms, are thus the more highly frequent symbols through the relative frequency table. We find that, interestingly, the predicted symbols are also the ones found at the beginning of other tablets.

11

Zipf's law is an empirical law formulated using mathematical statistics with related discrete power law probability distributions. The underlying principle in Zipf's law accepting models is that short sequences have a high probability of occurring while long sequences have a low probability and hence a high energy (Zipf, 1936).

Figure 6: Short-tailed distribution plot of the most frequently occurring symbols

Then we test the Zipf law by plotting first the frequency of the symbol ranked in descending order, then taking the logarithm of these values. Given that this dataset is allegedly 90% administrative records with billing records, the summary of the "goat" symbol along with fractions is an apt summary, we believe, for the dataset in question.

Figure 7: Plotting Zipf Law short-tailed distribution of probable words in the data set

For Entropy analysis, we expected the output value to be comparable to other known languages. The stone tablets containing only 96 unique symbols or transliterations transfers to the entropy12 analysis where we gain the high entropy level of slightly above 9.4. The short tail distribution of the log of frequency and rank versus the transliterations is also very evident at each iteration.

In this case, our analysis of entropy reveals a larger level of disorder in the Linear A language. If the disorder of the symbols is high, that could mean that it is not a language as we understand human languages to be, since most human languages have a similar entropy value [START_REF] Shannon | A mathematical theory of communication[END_REF]. The higher the Shannon entropy 13 value, the greater the disorder and, subsequently, the less likely we are to think of that dataset as a language.

In this case, we obtained two results. The first result was obtained using only the "probable words" from the corpus, instead of the entire corpus. The output revealed a greater entropy of 9.404, than that of languages we know, which was inconclusive in terms of defining whether what we were looking at was a human language. However, when including more of the symbols and ideograms, the entropy decreased to 3, showing that the disorder had been greatly reduced [START_REF] Matricciani | Shannon's entropy as a measure of the "life" of the literature of a discipline[END_REF]. One possible conclusion is that the introduction of more symbols in the dataset allowed the algorithm to recognize more patterns, which reinforces that we are looking at a human language from the information theory perspective (see Figure 8). 13 Shannon Entropy formula where it is defined as "In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent in the variable's possible outcomes." [START_REF] Shannon | A mathematical theory of communication[END_REF] As seen in the output, the first set of symbols are the same, so their similarity comes to 1.0, while the similar but not same symbols values are 0.99. The list of symbols with similar positive values as the first set of symbols and then the negative associations of the second symbol in the second set is also apparent. The within-cluster sum of squares is a measure of the different types of transliterations within each cluster. The average distance from observations in this graph is seemingly random which can be attributed to the high entropy given that it is a measure for the variability of the transliterations within each cluster. Centroid helps interpret each cluster as a focus for the cluster.

As a generative statistical model, Latent Dirichlet Allocation (LDA)14 uses known observations to group similar unobserved groups of terms, which is extremely useful for undeciphered languages. A word cloud gives the visual representation of most common stone tablets in the text corpus (see Figure 15) The terms are ranked by their relevance in the corpus data with the λ slider. While the terms are ranked by the probability of the topic, since the pyLDAvis is an interactive tool, the topics can be selected and adjusted accordingly. The λ slider adjusts the critical value of the relevant term rank for each specific topic. The value that is usually used is 0.6. The Topic Modeling results go on to show that there are five major groupings of the transcriptions in Linear A language as the Intertopic Distance Map (via multidimensional scaling) shows. While group 2, 3, and 4 are overlapping, which seems to indicate closer similarity, 1 and 5 are on the opposite side of the graph in the modeling graph.

The LDA proved to be an improvement in similarity analysis by topics. While this is one methodology to understand the patterns in the undeciphered language, and the visual gave a better idea of token relationships, by itself LDA was an improvement over the base NLTk entropy function.

Exploratory Knowledge Mining

In addition to NLP, we also used some machine learning methods to detect most common symbols. In order to use machine learning, the data needed to be pre-processed so that it could work best within the constraints of the two following methods we tried: Naïve Bayes 15 (NB) and K-nearest-neighbor16 (kNN). NB uses a probabilistic method for breaking down the components of the symbols. From a data pre-processing perspective, we needed to understand which symbols and representation to use [START_REF] Bunescu | Collective information extraction with relational Markov networks[END_REF]. We opted for using the symbols which were likely words, as well as symbols that were likely ideograms. The difference between symbols and words were great enough so that assigning a true or false to whether the symbol represented a word or ideograms, could allow for NB to make a determination on future symbols or ideograms, or if a symbol fell outside of these categories.

Figure 18: Determination of symbols commonality by their representation

In order to accomplish this, we ran NB on the "word" symbols, as well as the "ideograms" symbols. We used different percentages for training versus testing data, with the results below using 1/3 of the data for training and the remaining data for testing. The accuracy of NB for word detection was 73% with a low precision rate of around 0.45, while the accuracy for ideogram detection was lower with 65% accuracy with a higher accuracy of 0.71. In order to read the symbols, we had to make an assumption on what consisted of a word. Each instance of a symbol represented a word, and ideograms were found in words, which may explain the lower accuracy for ideograms detection. Another point to note is that the dataset included fewer ideograms than words.

Figure 19: Restructuring data to identify probable words as a word or a logographic idiogram

Another machine learning application we used was kNN. In order to use kNN, we needed to understand what we wanted to use as a measure of nearness. The "Gibbs" method, which utilizes a Markov chain in order to establish the next nearest point by generating a chain of samples, seemed like the most flexible use of kNN for a dataset that had not been deciphered, and therefore did not allow decision based on the knowledge of the language. As the "Gibbs" method has been used for words in sentences, the use for this dataset seemed to work theoretically [START_REF] Malouf | Markov models for language-independent named entity recognition[END_REF]. While "Gibbs" can provide a helpful way to determine nearest when ambiguity of the text is present, it does so with greater error. With the "Gibbs" method, along with 10-fold cross validation, the accuracy of prediction was around 64%, with a kNN of 7 being used for the final model.

III Conclusion

Discussion

The NLP analysis gave us the frequency distribution of popular symbols in the Linear A stone tablets. n-gram summary of the tablets' contents and predicted the possible subsequent symbols in other tablets. With the inclusion of symbols beyond probable words, the entropy reduced from 9.4 to 3 thereby verifying the consistent unstructured nature of the language via word embedding process using Word2Vec and Principal Component Analysis. Three methods of grouping the symbols were explored from Cosine Similarity Analysis, K-Means Clustering, Topic Modeling, and K-Nearest Neighbor verified by 10-fold Cross Validation, all of which were successful in various measures. Majority of the symbols were grouped in similar buckets except a few interesting outliers. Attempting to classify the symbols was an interesting exercise for ascertaining the accuracy of ideograms particularly.

Deciphering Linear A, the ancient Mycenaean language has been an ongoing effort for many decades now. Through our exploratory computational analysis, we hope to add to the discourse via introducing various new methodologies to this study of this language. To the best of our knowledge, this is the first study to discuss and show computational analysis of Linear A. The purpose of the current study was to examine and apply the possibilities of a series of Natural Language Processing and Machine Learning techniques to undeciphered languages. The paper also proposes to broaden the toolkit for these languages and bridging the gap between a linguistic approach and a data-mining approach. Utilizing computational techniques on the undeciphered scripts in Linear A, we yield some preliminary results. Our future work includes a comparison and analysis of Linear A with other ancient languages, both deciphered (Linear B) and undeciphered (Rongorongo).

Figure 1 :

 1 Figure 1: Left, the Mycenaean Tablet KN As 1516(Chadwick, et al., 1987). Right, the transliterated text of the KN As 1516(Chadwick, et al., 1987)

Figure 3 :

 3 Figure 3: Linear A Dataset from Github repository mwenge / lineara.xyz

Figure 4 :

 4 Figure 4: Natural Language Processing N-gram analysis process flow

Figure 8 :

 8 Figure 8: Entropy of natural languages and other sequences.(Rao, 2018)

Figure 9 :

 9 Figure 9: Word Embedding Process Flow

Figure 10 :

 10 Figure 10: Word Embedding and Principal Component Analysis of Linear A data set

Figure 11 :

 11 Figure 11: Cosine Similarity analysis of the symbols

Figure 12 :

 12 Figure 12: K-means clustering scores

Figure 14 :

 14 Figure 14: Topic Modeling process flow

Figure 15 :

 15 Figure 15: Word cloud of locations the Linear A tablets were found by the number of symbols gathered from each

Figure 16 :

 16 Figure 16: Interactive Topic Modeling graph

Figure 17 :

 17 Figure 17: Knowledge Mining process flow

An ideogram is a character or symbol representing an idea or a thing without expressing the pronunciation of a particular word or words for it, as in the traffic sign4 A transliteration is a transcription from one alphabet to another transcription, written text -something written, especially copied from one medium to another, as a typewritten version of dictation.3 A facsimile is a copy or reproduction of an old book, manuscript, map, art print, or other item of historical value that is as true to the original source as possible.2 In a written language, a logogram or logograph is a written character that represents a word or morpheme.

A file with the .ODS file extension is most likely an OpenDocument Spreadsheet file that contains spreadsheet information like text, charts, pictures, formulas and numbers, all placed within the confines of a sheet full of cells.

Lexical dispersion is a measure of how frequently a word appears across the parts of a corpus. This plot notes the occurrences of a word and how many words from the beginning of the corpus it appears (word offsets).

Entropy is the measure of uncertainty, randomness, or a disorder of a system(Clausius, 1865)

The Latent Dirichlet Allocation (LDA) is used to generate a probabilistic model of a corpus with a variety of topics using Jensen-Shannon divergence between topics computation and term frequency relevance. LDA is a useful technique within the machine learning toolbox and thus, the artificial intelligence toolkit.

In instance-based classification, each new instance (of symbol, in our case) is compared with existing ones using a distance metric, and the closest existing instance is used to assign the class to the new one. This is called the nearest-neighbor classification method.[START_REF] Witten | Data mining: Practical machine learning tools and techniques[END_REF] 15 In statistics, Naive Bayes classifiers are a family of simple "probabilistic classifiers" based on applying Bayes' theorem with strong independence assumptions between the features.

Acknowledgments

We are incredibly grateful for Dr. William Kennedy, and Dr. Duoduo Liao's support and guidance in this endeavour. Further specialized inferences will be gleaned from the results by Dr. Frederico Aurora, in conjunction with our team.

Zipf, G. K. (1965). The psycho-biology of language: An introduction to dynamic philology. Cambridge, Mass. : M. I. T.

Press. http://archive.org/details/psychobiologyofl0000zipf "mwenge/lineara.xyz," GitHub. https://github.com/mwenge/lineara.xyz (accessed Oct. 18, 2020). The complete Linear A corpus is relatively small, consisting of only a few hundred documents. The total corpus would fill only 5 or 6 pages if all the documents were placed end to end. A Linear A scholar George Douros has produced a spreadsheet that tabulates each document and helpfully provides word breaks, ideograms, and numerals broken out for each line of each document. When converted to csv format this spreadsheet allows me to create the core of the app, which is a structured representation of each document in JSON format. Github user mwenge has a python script that takes in the csv file and writes a javascript array of JSON objects to a javascript file called LinearAInscriptions.js.. Here is a snippet of the file, showing how the document HT1 is represented in as a JSON object:


La Bourdonnaye

Céline Teulière

Thierry Chateau

Jochen Triesch

Within Reach? Learning to touch objects without prior models

Keywords: Sensori-motor learning, reinforcement learning, object reachability

Human infants learn to manipulate objects in a largely autonomous fashion, starting without precise models of their bodies' kinematics and dynamics. Replicating such learning abilities in robots would make them more flexible and robust and is considered a grand challenge of Developmental Robotics. In this paper, we propose a developmental method that allows a robot to learn to touch an object, while also learning to predict if the object is within reach or not. Importantly, our method does not rely on any forward or inverse kinematics models. Instead it uses a stage-wise learning approach combining deep reinforcement learning and a form of self-supervised learning. In this approach, complex skills such as touching an object or predicting if it is within reach are learned on top of more basic skills such as object fixation and eye-hand-coordination.

I. INTRODUCTION

For any manipulation robot, estimating the reachability of objects is a very important ability. Traditionally, the problem has been approached by using inverse kinematics, i.e. a mapping from 3D world coordinates to robot joint angles that would bring the robot's gripper to the desired 3D location. For example, [START_REF] Guan | Reachable Space Generation of A Humanoid Robot Using The Monte Carlo Method[END_REF] represents the reachable workspace of a humanoid robot as a set of 3D Cartesian points. Their method samples random 3D positions and simply checks if inverse kinematics provide a solution. Goal babbling [START_REF] Rolf | Goal babbling with unknown ranges: A direction-sampling approach[END_REF] can be considered as a way to learn reachable 3D points while also learning inverse kinematics, provided that the forward kinematics are known. Finally, [START_REF] Yang | Exploiting hierarchical probabilistic motion planning for robot reachable workspace estimation[END_REF] estimates the workspace reachability of a mobile manipulator using a probabilistic motion planner. Importantly, all of these methods assume knowledge of the forward and/or inverse kinematics, which is undesirable in the context of Developmental Robotics. In contrast, [START_REF] Jamone | Autonomous online generation of a motor representation of the workspace for intelligent whole-body reaching[END_REF] can dispense with it, because the reachability estimation is based directly on the robot's gaze. Specifically, the pan, tilt, and vergence angles of the binocular vision system implicitly encode the 3D position of an object. Thus, they can be used as a surrogate of a 3D Cartesian point, This work was funded by the French government research program "Investissements d'avenir" through the IMobS3 Laboratory of Excellence (ANR-10-LABX-16-01), by the European Union through the Horizon 2020 Research and Innovation Programme under grant agreement no. 713010 (GOAL-Robots Goal-based Open-ended Autonomous Learning Robots) and through the program Regional competitiveness and employment (ERDF Auvergne region), and by the Auvergne region. J. Triesch acknowledges support from the Quandt foundation.

which may be more biologically plausible [START_REF] Medendorp | Gazecentered updating of visual space in human parietal cortex[END_REF]. Here, we take inspiration from this idea and combine it with our own prior work on the learning of an object touching skill using deep reinforcement learning [START_REF] De La Bourdonnaye | Stage-Wise Learning of Reaching Using Little Prior Knowledge[END_REF].

Our learning approach comprises three stages during which successively more sophisticated skills are acquired. Importantly, the entire learning process does not require any forward or inverse kinematics models, but relies exclusively on (deep) reinforcement learning and a form of self-supervised learning. In this sense, the complex behavior is essentially learned from scratch. The three learning stages are as follows. First, the robot learns to fixate an object to localise it. Second, it jointly learns to fixate on its own end-effector while learning a hand-eye coordination mapping. In the third stage, the robot uses the skills acquired in the two previous stages to jointly learn to reach for and touch objects and predict if an object is within reach or not. The eye-hand-coordination mapping learned during the second stage essentially substitutes for the forward kinematics during the final stage.

The fixation tasks as well as the reaching behaviours are learned using deep reinforcement learning. The hand-eye coordination and the reachability prediction are learned using a form of self-supervised learning which uses supervised learning with self-generated data. The final reachability prediction outputs a probability of the target object being within reach given the joint angles of the robot's cameras when fixating the target object.

II. METHODS

A. Background 1) Reinforcement learning: Reinforcement learning (RL) is a class of algorithms used to solve sequential decision making problems through learning. Most RL algorithms are based on Markov decision processes < S, A, R, T > where S is the set of states, A the set of actions, T the transition model (T : S × A → S) and R the reward function (R : S × A → R).

The agent learns from interaction with the environment and gathers transitions < s, a, r, s >. s represents a state and a the action performed at state s. After the execution of a, the agent receives a reward r and reaches a new state s .

The goal of an RL agent is to adapt its behaviour to maximise the amount of future rewards. In this paper, we consider the sum of discounted future rewards: J = ∞ k=0 r k γ k , where γ ∈ [0, 1] is a discount factor and r k the reward value at step k. To optimise this criterion, we train a deterministic policy π : S → A jointly with the state-action value function Q in an actor-critic set-up:

Q π (s, a) = E π ∞ k=0 r k γ k s, a , (s, a) ∈ S × A. (1)
2) Deep reinforcement learning: RL suffers from the curse of dimensionality. The common approach for dealing with this problem is to use function approximation, in particular using neural networks. This approach has become very popular with the arrival of GPUs and the wide-spread application of deep learning. In our work, we use the DDPG algorithm [START_REF] Lillicrap | Continuous control with deep reinforcement learning[END_REF], which can solve RL problems with a high-dimensional state space and a continuous action space. This algorithm combines the deterministic policy gradient algorithm [START_REF] Silver | Deterministic Policy Gradient Algorithms[END_REF] and the deep Q network [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF].

DDPG is an "actor-critic" algorithm updating the critic Q φ with parameters φ and the deterministic policy π θ with parameters θ as follows. At each time-step, we sample (uniformly) a mini-batch of N b transitions from a large memory buffer T buf of size N trans :

< s i , a i , r i , s i > i∈{1,...,N b } ∈ S × A × R × S.
The targets of the Q φ neural network are computed using a TD(0) update with a learning rate equal to 1:

∀i ∈ {1, ..., N b }, y i = r i + γQ φ (s i , π θ (s i )) . (2)
φ and θ are the parameters of the target networks updated using a rate parameter τ (t denotes a time-step):

φ t+1 = τ φ t + (1 -τ )φ t , θ t+1 = τ θ t + (1 -τ )θ t , (3)
The Q φ network updates its weights by minimizing the squared error

1 2N b N b i=1 (y i -Q φ (s i , a i ))
2 . Using target networks greatly contributes to the learning stability of the neural networks and using a memory buffer helps to satisfy the constraint of i.i.d samples for learning with neural networks. Using the Q φ network and the fact that the policy is deterministic, the following policy gradient is derived:

∂Q φ ∂θ 1 N b N b i=1 ∂Q φ (s i , π θ (s i )) ∂a ∂π θ (s i ) ∂θ . (4)
This update makes the policy select the actions that maximise the Q function at the batch states. In addition to this algorithm, we use the inverting gradient procedure of [START_REF] Hausknecht | Deep Reinforcement Learning in Parameterized Action Space[END_REF] to bound the actions.

B. Task definition

The robot's goal is to learn to reach for and touch an object on a table in front of the robot and to predict if a successful reach is possible. We use a 7 DOF arm with a pair of cameras (see Fig. 1). Successful reaching means that the robot touches the object with the "palm" of its end-effector. For reachability prediction the robot maps its gaze angles onto a prediction whether the object is within reach or not. Learning occurs in a stage-wise approach, where the robot first learns simpler skills We use the following notations:

• I = (I left , I right ) represents the images from the left and right cameras. • q = (q camera , q robot ) represents the 3 camera joint angles (one common tilt angle and two independent pan angles) and 7 robot arm joint angles. • c b represents the touch sensation of the robot. It is an 8element binary vector representing 8 areas of the robot's fingers. They correspond to the proximal, medial, and distal areas of the three fingers, with the exception of the proximal area of one finger which is linked to the palm. When in contact, the corresponding entry in c b becomes 1 and 0 otherwise.

The reachability prediction takes the form of a function mapping camera joint angles q camera to a probability P ∈ [0, 1] that the robot will be able to touch the object. This mapping is approximated by a feed-forward neural network trained with example pairs (q camera , P T ) collected while the robot learns to reach. More precisely, after each episode, we record the pair (q camera , P T ), where P T is defined as:

P T = 1, if reach was successful, 0, otherwise. (5)
Our stage-wise learning framework involves three successive tasks that are described in the following.

C. Learning object fixation

First, the robot learns from raw pixels to fixate on the object with its two cameras [START_REF] De La Bourdonnaye | Learning of binocular fixations using anomaly detection with deep reinforcement learning[END_REF]. This consists of moving the cameras such that the object comes to the image centers of both cameras. This task is learned using the DDPG algorithm with a weakly supervised reward function. The learned object fixation policy is noted π fix ψ . The reward signal is computed using an anomaly object localisation technique and requires neither calibration parameters nor hand-crafted visual modules (see [START_REF] De La Bourdonnaye | Learning of binocular fixations using anomaly detection with deep reinforcement learning[END_REF] for details). At the end of an object fixation, the camera joint angles q camera fix implicitly encode the object position in 3D space. This object localisation technique has been proven efficient to learn to touch an object [START_REF] De La Bourdonnaye | Stage-Wise Learning of Reaching Using Little Prior Knowledge[END_REF].

D. Learning hand-eye coordination

In the second step, the robot learns a hand-eye coordination function f η , which maps arm joint angles to "virtual" camera joint angles:

q camera virt = f η (q robot ). (6)
These "virtual" camera joint angles correspond to the ones which would make the camera system fixate on the endeffector. The function f η is approximated by a feed-forward neural network and the input-output pairs (q robot , q camera virt ) are generated via the parallel learning of an end-effector fixation task. The latter is learned from raw pixels using the DDPG algorithm and a reward function requiring little prior knowledge. The reward signal is computed based on the localisation of the end-effector in the left and right images using a pre-defined motion of an end-effector finger (see [START_REF] De La Bourdonnaye | Stage-Wise Learning of Reaching Using Little Prior Knowledge[END_REF] for details).

E. Learning reaching skills

1) Learning to reach: In the final learning step, the object fixation policy π fix ψ and the hand-eye coordination mapping f η are utilised to learn reaching and reachability prediction. The reaching behaviour is learned using the DDPG algorithm. The state space S is composed of the arm and camera joint angles q as well as eight binary tactile sensors c b attached to the fingers of the hand. Images are not required here because we use a single object and consider that the camera joint angles give sufficient information about the 3D object position. However, they would be necessary if objects with different shapes were used in the experiments. The actions are changes of the robot joint angles: a = ∆q robot , which are seven realvalued scalars. The reward signal for the reaching task is the sum of three components:

r = r touch + r contact + r shaping . (7)
r touch defines the objective of the task:

r touch = 1, if success, 0, otherwise, (8)
where success means that the robot has reached the object with its hand's palm, ending the episode. r contact penalises contacts between the end-effector and the table. This term is negative when contact occurs and prevents the robot from exploring too much areas in which one of the fingers (but not the palm) touches the table:

r contact = -0.01, if contact, 0, otherwise. (9)
The constant contact value has been empirically found and its absolute value must be small compared to the sparse reward value. If it is too high, the arm is repelled by the table and this slows down learning. r shaping is an additional shaping reward which favors actions that bring the robot end-effector closer to the point the robot is fixating with its cameras:

r shaping = 0, if success, -1 30 ||q camera fix -q camera virt || 2 , otherwise. (10)
Here, q camera fix is the vector of camera joint angles which makes the camera system fixate on the object. To obtain it, the object fixation policy π fix ψ is used. q camera virt is computed using [START_REF] De La Bourdonnaye | Stage-Wise Learning of Reaching Using Little Prior Knowledge[END_REF] and represents the camera joint angles which make the camera system fixate on the end-effector. This requires instantaneous robot joint angles q robot and the learned handeye coordination mapping f η . As for r contact , the shaping term should be small with respect to the sparse reward value. The coefficient of -1 30 has been found empirically. The sum of these three reward terms has been proven efficient to learn the object reaching task for various initial conditions and requires little prior knowledge [START_REF] De La Bourdonnaye | Stage-Wise Learning of Reaching Using Little Prior Knowledge[END_REF].

The learning of the reaching policy uses an episodic set-up. For each episode, the robot uses π fix ψ to fixate on the object and the arm joint angles are set to their initial values q robot 0 . The robot learns to reach during N tot episodes of at most N max iterations. An episode ends if the robot successfully reaches the object or after N max iterations. For exploration, we use the Ornstein-Uhlenbeck process. A noise term j (t) is added to the motor command ∆q robot in every time-step and is computed as follows:

j (t) = θ j µ j + (1 -θ j ) j (t -1) + ξ j (t). ( 11
)
θ j is called the mean reversion speed, µ j is the equilibrium value, j (t -1) is the noise computed at time-step t -1 and ξ j (t) is a centered Gaussian noise of standard deviation σ j . This exploration strategy is particularly helpful for discovering behaviours that imply a succession of actions which have the same direction.

To accelerate learning, we use a strategy to avoid situations where the robot's movement is blocked because of unwanted contact of the fingers with the table or the object. When such a contact occurs, the robot takes a backward action to go to a previous contact-free position.

2) Learning reachability: The reachability mapping Re α is learned using supervised learning. The input-output pairs are collected during the learning of reaching movements. Specifically, the data (q camera , P T ) are added to a database when the reaching policy starts to perform well (from N start episodes) and reachability learning starts when this database contains a sufficient number of elements N upd .

The database of the reachability mapping D is a circular buffer of size N D . This architecture is chosen because it allows to forget episodes in which the reaching policy is not performing well. If, e.g., the robot does not reach a reachable object position, the algorithm should be able to forget this corrupted datum. Algorithm 1 summarises the whole joint learning procedure.

III. EXPERIMENTS

The experiments are carried out in a simulated environment using the Gazebo simulator jointly with the ROS middleware.

A. Implementation details

For all the neural network algorithms, we use the caffe library [START_REF] Jia | Caffe: Convolutional Architecture for Fast Feature Embedding[END_REF]. A GPU (nvidia GeForce GTX Titan X) is used for the experiments. For the reaching task, the Q network has 3 fully connected layers with 250, 200 and 1 neural units. The policy network involves 3 fully connected layers with 200, 150 and 7 neural units. The hand-eye coordination function is a neural network with 2 fully connected hidden layers of 10 and 5 neurons. The neural network representing the reachability prediction has 2 fully connected hidden layers of 5 units. The neural network structures approximating the policy and the Q function for the fixation tasks are described in [START_REF] De La Bourdonnaye | Learning of binocular fixations using anomaly detection with deep reinforcement learning[END_REF]. The Adam solver [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF]

B. Ground-truth reachability estimate

To evaluate the learned reachability network, we need to estimate the ground-truth workspace reachability. For this estimation, we build on the knowledge of the robot structure. First, we empirically know that the left arm can reach an object if it is put at the bottom left of the table (from the robot's point of view). Second, we know from the geometry that the furthest points that the robot can reach follow two Algorithm 1 Joint learning of reaching and reachability Parameters:

1: N trans : size of circular transition buffer Reset arm joint angles and randomly place the object 6:

Fixate on the object using π fix Go to the position q robot 0 8:

while !cond do 9:

if blocked then 10:

Apply a backward action a t 11:

else 12:
Update Γ (t) using equation [START_REF] De La Bourdonnaye | Learning of binocular fixations using anomaly detection with deep reinforcement learning[END_REF] 13:

Apply a t ← π θ (s t ) + Γ (t)

14:

Compute the reward signal r t and sense s t+1 15:

Add < s t , a t , r t , s t+1 > to T buf 16:

Pick N b random transitions from T buf 17:

Update Q φ and π θ using DDPG n eps ← n eps + 1 circular arcs of different radii. Indeed, in Fig. 3, we can see the ways to reach the furthest points. In the first configuration, the arm is outstretched and the second revolute joint can make the arm move while keeping the arm outstretched. The second configuration corresponds to the position where the robot cannot move around its second revolute joint, because it would be hitting its structure. Thus, the arm can reach the furthest points only by moving around its fourth revolute joint and forcing the end-effector to keep a given orientation. Using these facts, the workspace limits are some borders of the table and the mentioned circular arcs. Then, we fit the two circular arcs on measured points using least mean squares.

C. Training evaluation

Our experiments start by the analysis of the training data, which has two objectives. First, we want to evaluate how accurate the learned reachability mapping is compared to the estimated ground-truth reachability. For this, we plot the reachability prediction error, which is the difference between the prediction of the learned mapping and the ground-truth estimate prediction, as a function of episode number. The prediction of the learned mapping is obtained by thresholding the output of the reachability prediction network at 1/2. Second, we wish to evaluate how often the robot matches the ground-truth prediction (through the learned reaching policy). To this end, we compare the reaching success with the groundtruth object reachability. We call this measure the reaching performance, because we measure whether the policy reaches reachable positions. For each of these measures, we provide 95 percent confidence intervals [µ -1.96σ √ Nrun , µ + 1.96σ √ Nrun ]. N run (equal to 3) is the number of run experiments, µ and σ are the average and the standard deviation of the measure at a given episode.

In Figure 4, we notice first that the reaching performance converges to a decent performance of around 95%. This shows that the reaching learning framework allows to learn to reach in most areas where the object is reachable. Besides, we notice that the reachability prediction error falls below 10% within the 40,000 episodes of training.

D. Resulting reachability

After analysing the training data, we wish to evaluate the learned reachability network. For a regular grid of object positions on the table, the object fixation policy π fix ψ is applied. Then, the resulting camera joint angles q camera fix feed the reachability prediction network. Figure 5 represents a heat map of the predicted reaching probabilities. The white curve represents the border of the estimated ground-truth reachability.

We observe that the overall results are very good. The border of the learned reachability prediction mostly matches the one of the estimated ground-truth reachability. The small errors close to the border may occur because of object localisation errors implied by the object fixation policy. Furthermore, we observe that the robot can still efficiently learn both to reach and reachability in most of the reachable and unvisited (during hand-eye coordination training) area. In Figure 5, the latter is localised outside the space surrounded by a dashed curve and inside the one surrounded by the full curve. However, it has difficulties with a specific area. To explain this phenomenon, the most probable hypothesis is that the hand-eye coordination outputs aberrant values for some arm joint angles required to reach these object positions. Then, this slows down the learning of reaching and reachability. We believe that these object positions could be mastered if we make experiments run longer or if the hand-eye coordination is trained on a larger domain.

IV. DISCUSSION

Building robots that can autonomously learn to interact with their environment, manipulate objects, and communicate with people remains a grand challenge for Developmental Robotics.

Here, we have proposed a learning approach through which a robot can learn to reach for and touch objects while also learning if an object is within reach or not. Importantly and in contrast to prior work, our approach works completely without the use of any forward or inverse kinematics models. Our approach also does not require any calibration parameters, visual markers, or hand-tuned vision modules. Instead, it relies on a combination of deep reinforcement learning and a form of self-supervised learning, where input-output training samples are generated through the robot's own behavior. Thus, our learning is more autonomous, as it requires less prior knowledge or structures than previous approaches.

Several studies used also fixation and/or hand-eye coordination to learn reaching behaviours [START_REF] Ghadirzadeh | A sensorimotor approach for self-learning of hand-eye coordination[END_REF], [START_REF] Chinellato | Implicit sensorimotor mapping of the peripersonal space by gazing and reaching[END_REF]. However, they do not learn an explicit model of reachability. In contrast, [START_REF] Jamone | Autonomous online generation of a motor representation of the workspace for intelligent whole-body reaching[END_REF] jointly learned reaching and reachability, but their approach differs from ours in some crucial aspects. First, in our method the binocular fixation and the reaching control are learned using reinforcement learning whereas in their case the binocular fixation task is pre-wired and the reaching task is achieved through the inversion of a learned hand-eye coordination mapping. Second, unlike their approach, we do not assume any visual marker or hand-crafted segmentation to detect the pixellic end-effector and object localisations. Third, both our method and theirs learn the reachability as a function mapping the robot's gaze to the reachability prediction. However, we differ in the ways the targets (for supervised learning) are generated. [START_REF] Jamone | Autonomous online generation of a motor representation of the workspace for intelligent whole-body reaching[END_REF] detects if an object has been reached from the distance between the end-effector and the object in terms of camera joint angles. Then, the targets are computed based on this distance if the object has not been reached and on the optimality of the resulting arm configuration otherwise. In our strategy, we assume that the robot has reached the object when the palm of the end-effector touches the object. Then, we generate a binary target according to the reaching success. This means that the orientation control of the end-effector is involved in our learning of reaching and implies a more complex reaching behaviour. The experiments in simulation show that our framework is efficient to learn both reaching and reachability.

Despite the benefits of our method, the learning procedure still has a number of limitations. First, in our approach we strictly enforce a specific ordered sequence in which the different skills are learned. This makes the approach somewhat inflexible. If, e.g., the hand-eye coordination mapping needs to be re-learned because of some damage to the robot or problem with the joint encoders, a human has to decide this. A system that can autonomously recalibrate "on the job" would be superior and more biologically plausible, see, e.g., [START_REF] Lonini | Robust active binocular vision through intrinsically motivated learning[END_REF].

In the future, it will also be interesting to investigate to what extent such a stage-wise learning could emerge autonomously as a result of a curiosity mechanism [START_REF] Oudeyer | Intrinsic motivation systems for autonomous mental development[END_REF].

A second shortcoming is that once the robot fixates the target object, the subsequent reaching does not make use of any visual information anymore, but only relies on the learned eye-hand-coordination model. Thus the proposed approach would not be suitable for reaching towards moving objects. Note that using images for reaching is not incompatible with our framework since they can be added to the state space.

Third, our approach has been proven efficient only in a simulated environment and may need substantial efforts to be validated using a physical robot.

Despite these limitations, an advantage of our framework is its generality. While we have focused on learning to touch an object with the palm of the hand, it seems likely that a whole repertoire of skills could be learned in the future by providing different reward functions during additional learning phases.

Fig. 1 .

 1 Fig. 1. Illustration of the reachability prediction skill. In our experiments, the robot's left arm (from the robot's view) is used.

Fig. 2 .

 2 Fig. 2. Overall scheme of the stage-wise learning procedure

Fig. 3 .

 3 Fig. 3. Top views of the left arm illustrating the limits of reachability. The left figure illustrates construction of the first circular arc by rotating the shoulder joint. The right figure illustrates the construction of the second circular arc by rotating the elbow joint.

Fig. 4 .Fig. 5 .

 45 Fig. 4. Reaching performance and reachability prediction error as a function of episode number. Lines are averages from three independent experiments and dashed regions show 95 percent confidence intervals.

  is used to update the weights. Parameter values from Algorithm 1 are listed in TableI, II and III.

	Parameters	γ	Nmax	N b	Ntrans	Ntot
	Values	0.99	100		256	60 000	40 000
			TABLE I
		PARAMETER VALUES (1/2)
	Parameters N upd	N D	Nstart
	Values	200		20 000	20 000
			TABLE II
		PARAMETER VALUES (2/2)
	Parameters	θ j	µ j	σ 1 , σ 2 , σ 3 , σ 4	σ 5 , σ 6 , σ 7
	Values	0.8	0		0.01		0.04
			TABLE III
	ORNSTEIN-UHLENBECK PROCESS PARAMETERS

  2: γ: discount factor 3: Γ: parameters of the Ornstein-Uhlenbeck process 4: N tot : total number of episodes 5: N max : maximum number of iterations per episode 6: N b : number of transitions per batch 7: N upd : number of samples in the reachability database required to start the learning of reachability 8: N D : size of the reachability database 9: N start : number of episodes required to start the learning of reachability : hand-eye coordination mapping Outputs: Q φ , π θ , Re α Steps:

	Inputs:
	10: q robot 0 11: π fix ψ : fixation policy : initial arm joint angles
	12: f

η 1: Initialise D, T buf , Q φ , π θ and Re α 2: n eps ← 0 3: while n eps < N tot do 4: t ← 0 and cond ← False

5: