Ashesh Vasalya
email: asheshvasalya@gmail.com

Gowrishankar Ganesh

Abderrahmane Kheddar

More than just co-workers: presence of humanoid robot co-worker influences human performance

Does the presence of a robot co-worker influence the performance of humans around it? Studies of motor contagions during human-robot interactions have examined either how the observation of a robot affects a human's movement velocity, or how it affects the human's movement variance, but never both together. Performance however, has to be measured considering both task speed (or frequency) as well as task accuracy. Here we examine an empirical repetitive industrial task in which a human participant and a humanoid robot work near each other. We systematically varied the robot behavior, and observed whether and how the performance of a human participant is affected by the presence of the robot. To investigate the effect of physical form, we added conditions where the robot co-worker torso and head were covered, and only the moving arm was visible to the human participants. Finally, we compared these behaviors with a human co-worker, and examined how the observed behavioral affects scale with experience of robots. Our results show that human task frequency, but not task accuracy, is affected by the observation of a humanoid robot co-worker, provided the robot's head and torso are visible.

Introduction

Robotics is now increasingly shifting to service and application fields, where robots need to collaborate with, and work in close proximity to, human co-workers. In these scenarios, it is of prime importance to understand how the presence of a robot co-worker influences the performance of humans around them. This understanding is essential not just in regard to productivity, but also in order to monitor and control any emotional and motor effects the presence of robot co-workers may have on the humans.

Observation of actions performed by others is known to induce implicit effects on an individual's action. These effects, that are referred to as motor contagions, have been extensively studied in psychology and sports science [START_REF] Heyes | Automatic imitation[END_REF][START_REF] Blakemore | The role of motor contagion in the prediction of action[END_REF][START_REF] Becchio | Motor contagion from gaze: The case of autism[END_REF][START_REF] Ganesh | Beyond Watching: Action Understanding by Humans and Implications for Motion Planning by Interacting Robots[END_REF][START_REF] Ikegami | Watching novice action degrades expert motor performance: causation between action production and outcome prediction of observed actions by humans[END_REF][START_REF] Hillebrandt | Effective connectivity during animacy perception-dynamic causal modelling of Human Connectome Project data[END_REF][START_REF] Chaminade | From self-observation to imitation: Visuomotor association on a robotic hand[END_REF][START_REF] Oztop | Human-humanoid interaction: is a humanoid robot perceived as a human[END_REF][START_REF] Kilner | An interference effect of observed biological movement on action[END_REF][START_REF] Sciutti | Measuring human-robot interaction through motor resonance[END_REF]. In comparison, studies of motor contagions during human-robot interactions [START_REF] Vasalya | Distinct motor contagions during and after observation of action by a humanoid co-worker[END_REF] are sparse, and have examined either how the observation of robots affect a human's movement velocity [START_REF] Noy | Simple movement imitation: Are kinematic features sufficient to map perceptions into actions?[END_REF][START_REF] Kilner | Interference effect of observed human movement on action is due to velocity profile of biological motion[END_REF][START_REF] Bisio | Automatic versus voluntary motor imitation: effect of visual context and stimulus velocity[END_REF][START_REF] Bisio | Motor contagion during human-human and human-robot interaction[END_REF], or how it affects a human's movement variance [START_REF] Kupferberg | Video observation of humanoid robot movements elicits motor interference[END_REF][START_REF] Kupferberg | Moving just like you: motor interference depends on similar motility of agent and observer[END_REF][START_REF] Brass | Movement observation affects movement execution in a simple response task[END_REF][START_REF] Press | Robotic movement elicits automatic imitation[END_REF]. However, the studies that reported changes in movement variance utilize arguably abstract tasks, and the studies reporting changes in movement speed do not analyze at how the participant movement variance October 30, 2018 1/15 changed with the speed. On the other hand, most industrial tasks require specific precisions in the movements and therefore, the performance in these tasks needs to be defined by considering both task speed (or frequency) and task accuracy. Here primarily, we analyzed how looking at a robot affects both, the speed and variance of the observing human's movement, to see whether we can quantify how human performance is affected by motor contagions. Furthermore, while there is contradictory evidence to suggest that the physical form of a robot co-worker (specifically whether it is humanoid or not) does [START_REF] Chaminade | Social cognitive neuroscience and humanoid robotics[END_REF] or does not [START_REF] Kupferberg | Moving just like you: motor interference depends on similar motility of agent and observer[END_REF] affect the variance of movements by human's, it is unclear whether this is also true for the case of movements speeds, and hence performance. Finally, it is unclear whether and how the performance effects due to a robot co-worker are modulated by a human co-worker's prior experience with robots, an issue that is crucial to understand how the human performance will change with continued exposure to a robot co-worker.

To address these issues, we examined an empirical repetitive industrial task in which a human participant and a humanoid robot work near each other, see Fig. 1. We systematically varied the behavior, specifically frequency of robot movements and examined whether and how the frequency of movements by the human participants, and their task accuracy, is affected by the presence of the robot. To investigate the effect of physical form, we added conditions where the robot co-worker torso and head were covered, and only the moving arm was visible to the human participants. Finally, in order to compare the humanoid co-worker to a human co-worker, we also checked how the effects on the participants changed with a human co-worker, with and without his/her torso and head visible. To anticipate our results, we found that the presence of a humanoid co-worker can affect human performance, but only when it's humanoid form is visible. Furthermore, the effect was observed to increase with prior robot experience by the humans. Experimental setup. The participants in our experiment worked in six conditions; with a robot performing biological movements in A) robot co-worker condition; B) human co-worker condition; to check relevance of human form in C) robot covered co-worker condition; and D) human covered co-worker condition; E) a robot co-worker performing non-biological movements in robot non-biol co-worker condition; F) a robot co-worker performing industrial movements. The coordinate axis defining the movement setup is indicated in white (A). what aspect of their behavior we were analyzing in the experiment) of the experiments to avoid bias in the results as we are interested in the implicit effect of motor contagion. Each Participant received 2021 Japanese Yen (JPY) to participate. Participants for our study were recruited through an advertisement via a local event forum, Facebook page of our experiment and via word of mouth in the Tsukuba University, Tsukuba, Japan.

Materials and methods

Participants

Participants needed to be at least 18 years old to participate in this experiment, apart from that, there were no restrictions.

Setup

The participant and co-worker (a human or a humanoid robot) were seated on a chair with tables facing each other as shown in the experiment setup (Fig. 1). On a horizontally placed touch-screen (23-inch HD DELL, P2314T) on the table, participants were presented with two red circles of diameter 5 cm at distance of 50 cm from each other. The co-worker was similarly presented with two red circles on black cardboard 9 cm at distance of 50 cm. The whole setup was enclosed by movable panels and the panel behind the co-worker was covered with a dark grey curtain. A motion tracking system (Motion Analysis Co.) with six infrared cameras (kestrel) and ten passive markers were used to record the arm motions of the participant and co-worker at 200Hz. A bipedal HRP-2Kai [START_REF] Kaneko | Humanoid robot HRP-2Kai Improvement of HRP-2 towards disaster response tasks[END_REF] humanoid robot (154 cm tall, 58 kg, 32 degrees of freedom) was used as the robot co-worker (Fig. 1). A well-trained experimenter (M, 37) acted as the human co-worker. Both co-workers used their right arm throughout the experiment.

Experimental task and conditions

Motivated by the hand movements during an industrial pick-and-place, or parts assembly task, our task required participants to repeatedly touch two static red circles on the touch-screen with a stylus in their right hand during the task (Fig. 1). While the participants were asked to touch the circle in each trial, they were free to touch anywhere on the circle. In preliminary experiments, we found that the standard deviation of the participant's touches were less than 1 cm (both in the x and y directions), and we purposely chose the radius of the touched circles to be more than 2 times larger than their standard deviation (targets were of 5 cm in diameter). This large size was crucial because, while the participants were asked to touch the circle in each trial, they were free to touch anywhere on the circle. The large target size therefore enabled us to observe any change in participant's touch location (that may accompany contagions in their speed), in terms of position and standard deviation, across our experiment.

A co-worker (human or HRP-2Kai) worked on the same task in front of the participants. The participants were asked to perform their task at their own chosen 'comfortable' frequency, and ignore the co-worker. The participants worked in a series of 50 second trials with the co-worker. In a trial, participants initially performed alone for 10 seconds (participant-alone period), performed with the co-worker for next 20 seconds (together period) and then relaxed while watching the co-worker performs the task for the last 20 seconds (co-worker-alone period) (Fig. 2). All participants wore ear buds and headphones (through which we sent white noise) and had no external audio feedback (confirmed in the post experiment questionnaire, Q6). They were instructed to "always hold the stylus like a stamp and touch alternatively inside each red circle on the touch-screen with continuous and smooth hand movements at a comfortable speed". They were specifically told to "focus on your own task and ignore the co-worker when he/it starts after them". No instructions were given regarding the speed and movement trajectory.

We studied six experimental conditions. The participants worked with a HRP-2Kai humanoid robot co-worker in four conditions, specifically, a) robot co-worker in which the whole robot was visible to the participant, and the robot played back biological movements, b) robot covered co-worker, in which the robot played back biological movements, but its head and torso were covered, such that the participant could only see the robot's moving arm. c) robot non-biol co-worker, in which a fully visible robot performed non-biological arm movements, d) robot indus co-worker, in which a fully visible robot performed industrial arm movements and they worked with a trained human experimenter in the remaining e) human co-worker and f) human covered co-worker (where the head and torso of the human experimenter were covered) conditions, (see subsection HRP-2Kai movement trajectories).

The experiment for each participant consisted of working in 3 conditions. Each participant was assigned to one of 6 condition combination groups, see Table 1, each with the robot co-worker condition (main), in addition to two out of five other conditions. The order of the conditions was balanced across the combination groups. This allowed us to compare the behavior of the same participants in each condition in a combination group, with their behavior in the robot co-worker condition.

Table 1. condition combination groups (G) HRP-2Kai in robot co-worker (RV), robot covered co-worker (RC), robot non-biol coworker (RN), robot indus co-worker (RI) conditions and Human experimenter in human co-worker (HV), human covered co-worker (HC) conditions. The order of conditions in a combination group were randomized across participants.

Sessions/Groups G1 G2 G3 G4 G5 G6

Session 1 RI RC HV RC RN RC Session 2

RV RI RV HC HV RN Session 3

RN RV HC RV RV RV Each condition had 10 trials. The co-worker performed at a constant, pseudo-randomly selected frequency (in the range of 0.16 to 1.1 Hz) in each trial. The pseudo-random nature of the co-worker performance was critical to avoid behavioral drift contamination across trials. The human co-worker was provided with a metronome using earphones like in [START_REF] Bisio | Motor contagion during human-human and human-robot interaction[END_REF], to cue and help maintaining the movement frequency in each trial.

The robot movements in the robot co-worker conditions were a playback of the movements recorded from a previous human volunteer (see subsection HRP-2Kai movement trajectories for details). We quantified the participant performance in the trials by their half time periods or htp (the average time between two consecutive alternate touches, measured using motion tracking), and the variance of their press location (measured as a change of mean and standard deviation of their touch-screen presses in the X-Y plane).

October 30, 2018 4/15

HRP-2Kai movement trajectories

The biological movements played on HRP-2Kai in robot co-worker and robot covered co-worker conditions were a playback of the human arm movements (Fig. 3, blue plot) recorded in a preliminary experiment with three volunteers using the same (Motion Analysis Co.) motion tracking system, while the human movements were cued by an audio metronome. Movements were collected at several frequencies between 0.16 to 1.1Hz. We found the movements of the three volunteers to be statistically similar in the Cartesian velocity profiles (p>0.05), and showing similar trend in movement height with movement frequency -trajectory height consistently decreased with increase of movement frequency. We therefore chose to use the movements recorded from one volunteer (a male) in this experiment. Well learnt human movements are characterized by a bell-shaped velocity profile.

The peak of the bell-shaped profile may be shifted forward in time when precision is required at the reach end (like in our task when the participants required to touch inside a given target region), but the velocity profile is normally characterized by a single peak. Therefore, to develop a 'non-biological' movement profile for the robot non-biol co-worker condition, we developed a movement profile with multiple velocity peaks. This profile was developed in position-time (cyan plots in Figs. 3 and4) profile using fifth and third order polynomial segments (lift-off, carry, set-down) [START_REF] Biagiotti | Trajectory planning for automatic machines and robots[END_REF]. We observed that human volunteers' movements to predominantly be in the Y-Z plane. The piece-wise polynomial trajectory for the robot non-biol co-worker condition was designed over the y (horizontal) and z (vertical) dimensions, while x was always kept constant zero. The industrial trajectory was characterized by a constant velocity phase. Inspired from the industrial manipulators, and keeping in mind our HRP-2Kai joint constraints during fast movements, we improvised the traditionally used industrial trapezoidal velocity profile, with a third order velocity sections in the acceleration and deceleration phase (magenta plots in Figs. 3 and4). Again, since our movements were restricted in the Y-Z plane, we designed our smooth trapezoidal trajectory over the y (horizontal) and z (vertical) dimensions, while x is kept constant and zero. The Z elevation (z max ) in this trajectory was set to 13 cm when the robot moved from left to right, and 8 cm during the return.

Variables

Our analysis is based on the position data from both the participant's and co-worker's stylus markers. To extract out possible behavioral differences between the movements forward and backward, between the touch points, we analyzed behavioral variables across each movement between the red circles on the touch-screen, which we call as iterations (such that two iterations make a movement cycle). As participants and co-workers were required to make non-stop continuous movements between touches, we could extract individual iterations of participant's and co-worker's by looking for October 30, 2018 5/15 changes in the direction of y-velocity in the recorded motion capture data. In this study, we were interested in the task performance of participants, and therefore we primarily concentrated on the time between the alternate touches in each iteration, which we refer to as the half-time period (htp) or τ , and the location of their touches on the touch-screen (in the X-Y plane). In addition, we also analyzed various measures of position, velocity and acceleration along the Y (horizontal) and Z (vertical) axes over each iteration. However, these results are out of scope of this study.

Data analysis

We quantified the motor contagion in a participant' htp (the average time between two consecutive alternate touches) by analyzing the change of participant's htp between the together period and alone-period (see Fig. 2) in a trial (τ t p (i) -τ a p (i)), relative to the htp of the co-worker behavior in the same trial (τ c (i) -Av(τ a p )), where Av(τ a p ) represents the average undisturbed htp by a participants across his/her participant-alone periods. This data was regressed with either a first or second order regression model that was chosen based on the Akaike Information Criteria (AIC) [START_REF] Akaike | Information theory and an extension of the maximum likelihood principle[END_REF] and by using MATLAB's The change of participant's htp (the average time between two consecutive alternate touches) between the together period and alone-period (τ t p (i) -τ a p (i)), relative to the htp of the co-worker behavior in the same trial (τ c (i) -Av(τ a p )), where Av(τ a p ) represents the average undisturbed htp by a participant across all his/her participant-alone periods. Note that the (robot or human) co-worker htp was random across trials, and the data in plots here are the ensemble of the participant behaviors arranged in increasing co-worker's htp on the abscissa. Each plot represent a condition, A) robot co-worker (blue); B) human co-worker (orange); C) robot covered co-worker (dark blue); D) human covered co-worker (dark orange); E) robot non-biol co-worker (cyan); F) robot indus co-worker (magenta) conditions. We used the AIC to choose either a first or second order model to fit the data for each participant. The lines represent the tangent slopes at the minimal data abscissa value. Av(τ a p )]) was collected across participants (as shown in Fig. 5), checked for normality using the Shapiro-Wilk test and then analyzed for difference from zero using a one sample T-test (in case the distribution was normal) or a Signed Rank test.

fitlm
Next, to check the relevance of the human form, we conducted the robot covered co-worker and human covered co-worker conditions, in which the head and torso of the October 30, 2018 6/15 co-worker was covered and only the moving arm was visible to the human (see Fig. 1C, D or inset photos in Fig. 6). All other experimental settings and analysis were same as in the robot co-worker and human co-worker conditions.

Participant sample size

As the effects of the robot co-worker condition was the focus of our experiments, each of our participant worked in the robot co-worker condition, and two of the remaining 5 conditions (human co-worker, robot covered co-worker, human covered co-worker, robot non-biol co-worker and robot indus co-worker). Note that due to the fact that each of our conditions lasted over 20 minutes, resulting in more than 1 hour of total experiment time for the three conditions, we could not have every participant participating in all the conditions. We initially recruited 35 participants to have 14 participants in each of the five conditions (giving five participant groups each of whom participated in one of the five conditions in addition to the robot co-worker condition), so as to enable a intra participant one sample T-test between the robot co-worker and each of the remaining conditions. The number '14' was chosen as it corresponds to participant numbers in similar previous studies [START_REF] Bisio | Automatic versus voluntary motor imitation: effect of visual context and stimulus velocity[END_REF][START_REF] Bisio | Motor contagion during human-human and human-robot interaction[END_REF] and corresponds to a power analysis using G* power for a 2-tailed, one sample T-test (α=0.05, β=0.85, d=0.9) [START_REF] Erdfelder | GPOWER: A general power analysis program[END_REF][START_REF] Verma | Determination of Sample Size and Power Analysis with G*Power Software: Step-wise Illustrated Manual for Research Scholars[END_REF]. However, we found that with these participant numbers, the slopes in the same robot co-worker condition were not similar among the participant groups (p<0.05, one-way ANOVA). The robot co-worker condition slopes were significantly different from zero with two participant groups (p=0.022, and p=0.038), tended to be significant in two (p=0.07, and p=0.08) and not significant in another (p=0.36). As a majority of the values tended to be significant, we decided to increase the participant numbers by 50% (7 participants) across the participant group that were tending or not significant (these included participants who participated also in the robot covered co-worker, robot non-biol co-worker, and robot indus co-worker conditions), making a total of 42 participants.

With this participant number, the robot co-worker htp slopes were observed to be similar across the participant groups (H(4), P=0.99; one-way Kruskal-Wallis H-test).

After removal of three outliers, this gave us participants numbers of 13 (human co-worker condition), 13 (human covered co-worker), 17 (robot covered co-worker), 17

(robot non-biol co-worker), 18 (robot indus co-worker), and 39 in total for the robot co-worker condition, see Table 2.

Questionnaire

Perception and fatigue

Each of the participant in our experiment answered a short post experiment questionnaire consisting of 6 questions. The participants were asked to choose a score on a scale of 0 to 7, where 0 (Not at all), 7 (very strongly), for each of these questions, individually for every session they participated in:

Q1. My movements were affected when the agent was working with me.

Q2. My movement speed was changed when the agent was working with me.

Q3. I was tired during the experiment.

Q4. I could maintain the movement speed that I wanted even when the robot was performing its task.

Q5. I found it difficult to do my task when the agent was working with me.

Q6. I could hear noises from the co-worker during the experiment.

Q1, Q2, Q4 and Q5 were designed to access whether the participants cognitively realized the affects on their behavior due to the co-worker. A score close to one in Q1, Q2 and Q5 (and a score close to 7 in Q4) indicates that they did not consciously realize the effects. Therefore we considered the Q4 scores by subtracting the reported values from 7.

Robot exposure questionnaire

Following the end of our data collection, we also noted the need to measure the participant's robot experienced and exposure to robots. We therefore sent them a questionnaire of four questions:

RQ1. How many hours do you see and/or read about robots on average per week (include robots on TV)?

RQ2. If you work with robots currently, how many hours do you work with robots (or on robotics related topics) per week?

RQ3. If you have worked with robots, but do not work anymore, how many hours have you worked on them?

RQ4. How will you rate your knowledge of robots?

For each question, the participant had to answer in hours and chose between '0', 'less than 5', '5-10' '10-15', '15-20', '20-25', '25-30', 'more than 30'.

Statistical correction

As reported earlier, every participant in our study participated in three conditions: the robot co-worker condition, and two of the remaining conditions. We thus compare the behavior of the participant in any condition with the robot co-worker. For each participant, there were thus two comparisons made. Correspondingly, in our comparisons in Fig. 6, we use a Bonferroni correction of (3 conditions -1) 2, and all p values below 0.05 were multiplied by 2. Therefore, note that all the comparisons between conditions in Fig. 6 are between equal number of participants (and we use a one sample T-test).

October 30, 2018 8/15

Results

Robot behavior influences human movement frequency Fig. 5 shows the change of participant's htp (the average time between two consecutive alternate touches) between the together period and alone-period (τ t p (i) -τ a p (i)), relative to the htp of the co-worker behavior in the same trial (τ c (i) -Av(τ a p )), where Av(τ a p )

represents the average undisturbed htp by a participants across all his/her participant-alone periods. Note that the (robot or human) co-worker htp was random across trials, and the data in Fig. 5 is an ensemble of the participant behaviors arranged in increasing co-worker's htp on the abscissa. We then collected the slope of the polynomial at the lowest data abscissa as a measure of how the participant htp was affected by the co-worker htp. In the robot co-worker condition the slope distribution was not normal across the participants (p<0.05, Shapiro-Wilk test, median=0.017) and was significantly positive across participants (median=0.017, Z(38)=3.70, p=0.0002, Signed Rank test). The positive slopes (light blue data in Fig. 6) show that the robot performance htp (hence frequency) influenced the human participants. First, the human participant's htp increased when the robot htp was longer (see first quadrants of Fig. 5A), but for several participants, this increase had a threshold after which the participant's htp decreased. This behavior is the reason why we found a second order fit to explain the data better with many participants using AIC. Second, the participants htp also decreased when the robot htp was shorter (3rd quadrants of Fig. 5A, only across the participants in robot co-worker condition) indicating that a faster robot made the participants frequency higher. The htp results were similar with the human co-worker.

The positive slopes (orange data in Fig. 6) show that the human co-worker's performance htp (hence frequency) influenced the human participants (median=0.012, p=0.0017, Signed Rank test).

Press accuracy in the human not affected by robot co-worker

Studies in motor control have exhibited that human movements are constrained by motor noise, which increases with the magnitude of motor commands in the muscles [START_REF] Harris | Signal-dependent noise determines motor planning[END_REF]. In the case of 'regular' and automatic movements in daily life, this leads to a trade-off between the speed and accuracy of the movement [START_REF] Fitts | The information capacity of the human motor system in controlling the amplitude of movement[END_REF]. However, the accuracy of movements is also modulated by the regulation of arm impedance by muscle co-contraction [START_REF] Burdet | The central nervous system stabilizes unstable dynamics by learning optimal impedance[END_REF][START_REF] Franklin | CNS learns stable, accurate, and efficient movements using a simple algorithm[END_REF][START_REF] Ganesh | Motor planning explains human behaviour in tasks with multiple solutions[END_REF]. As mentioned earlier, to comment on the task performance of the human co-worker, we next analyzed whether and how the touch accuracy of the participants changed alongside the contagions in their htp.

Again, note that the target circles provided to the participants were large (5 cm diameter), and there were no constraint on where inside the target they touched.

Therefore the participant's touches can change (in position and/or variance) with their movement speed, without violating the task. However, interestingly we found that while the participants htp (hence movement frequency) changed, there were no such trend in the participants press (task) accuracy.

The mean touch positions were (X=0.95 cm; Y =1.17 cm) from the circle center across the participants, and showed a mean standard deviations of (std(X)=0.23 cm; std(Y)=0.79 cm) (across participants) when they worked alone (in the participant-alone period). And crucially, the participants maintained the same touch positions (there was no change of mean touch positions X: p=0.64; Y : p=0.86) and mean standard deviations (change of std(X): p=0.56; std(Y): p=0.41) between when they worked alone and when they worked with the robot co-worker (Fig. 7A), showing that the robot did not affect their task accuracies.

The press accuracy was similarly constant in the human co-worker condition in October 30, 2018 9/15 which the participants worked with another (unfamiliar) human, with no observed changes in the mean touch positions (X: p=0.69; Y : p=0.83; Fig. 7B) and mean standard deviations (std(X): p=0.56; std(Y): p=0.39; Fig. 7B). Here note again, that the touched circles were relatively large (5 cm) and the participants could have changed their touch position and variance while still satisfying the required task, but we do not observe this trend. Together, the change in movement frequency, and the lack of change in task accuracy shows that robot as well as human co-workers influenced participant task performance in our experiment.

Human form matters

Interestingly, covering the head and torso extinguished the contagions in the participant's htp -the participant's htps were no longer affected in the robot covered co-worker (T(16)=-0.3, p=0.78; dark blue data in Fig. 6) and the human covered co-worker (T(12)=0.24, p=0.82; dark orange data in Fig. 6), and these effects were significantly lower than the effects induced in the same participants in the robot co-worker condition (T( 16 This result show that the human form is crucial for induction of the performance changes.

In the robot co-worker and robot covered co-worker conditions, the robot played back the biological arm movements of a previous human volunteers. Finally, corresponding to previous studies that have shown that motor contagions are attenuated when a robot makes non-biological movements [START_REF] Kilner | An interference effect of observed biological movement on action[END_REF][START_REF] Bisio | Motor contagion during human-human and human-robot interaction[END_REF], we added two control conditions (robot non-biol and robot indus) in which the participants could see the robot co-worker's whole upper body, but the robot made non-biologically inspired arms movements to perform the task (see subsection HRP-2Kai movement trajectories).

Consistent with previous studies we did not find any significant change in htps in this condition (Z(16)=-1.07, p=0.29 (cyan data); Z(17)=-0.28, p=0.77 (magenta data) in The performance effect were implicit

An average of the scores from Q1, Q2, Q5 and Q4 (value subtracted from 7) was found to be equal to (mean±SD, 1.90±0.18) for the robot co-worker condition, and

(mean±SD, 1.65±0.24) for the human co-worker condition respectively. These low scores suggested that the participants did not consciously realize the effects on their behavior. Q3 was used to confirm that the participants were not tired in our task. We obtained scores of (mean±SD, 0.96±0.18) across the participants in the robot co-worker condition, and (mean±SD, 0.75±0.22) in the human co-worker condition. Q6 was used to confirm that the participants did not hear any external audio cues from either the robot's joints in the robot co-worker conditions (mean±SD, 0.5±1.25), nor the human co-worker's touches in the human co-worker conditions (mean±SD, 0.58±1.36).

Contagion increases with robot exposure

We received answers from 23 participants on our robot exposure questionnaire. Out of these participants, one participant who scored '0' for all questions was removed. We averaged the scores (taking either one from RQ2 and RQ3, as they were complementary) October 30, 2018 10/15

for the others and plotted the average against their htp slope in the robot co-worker condition in Fig. 8. Interestingly, we observed a significant positive correlation (Pearson's R=0.44, p=0.039), such that the effect on the participants were larger if they had more exposure and experience with robots. This result is conform to a recent report where participants with more experience with robots show higher adaptation to it, see [START_REF] Vannucci | Adaptation to a humanoid robot in a collaborative joint task[END_REF].

Discussion

In summary, primarily, we observed that the performance frequencies of human participants were influenced by the presence of a humanoid robot co-worker (or a human co-worker). We observed that participants not only become slower with a slower co-worker, but also faster with faster co-workers. Here we were interested to see the change of participant behavior 'relative' to the robot behavior. Hence we looked at a ratio, hence to quantify the motor contagion, we analyzed change in participant's htp between the together period and alone-period in a trial and relative to the htp of the co-worker behavior in the same trial. When the numerator term (τ t p (i) -τ a p (i)) is negative that means participants get faster from their initial participant-alone period htp (movement speed) and vice versa. Note that the subtraction in the denominator (of Av(τ a p ) ) is a constant that only shifts the curve and does not effect the slope.

In this study we wanted to choose a task that is simple, yet rich, and is representative of many industrial co-worker scenarios. We found that (repetitive) pick and place tasks to be the most common industrial tasks in which robots are employed. We therefore chose to start with a cyclic touch task in this experiment. The results we obtain here therefore, are specific to repetitive tasks. On the other hand, it has been shown that cyclic and discrete tasks may be very different in terms of neural processes [START_REF] Schaal | Rhythmic arm movement is not discrete[END_REF], and further studies are required to verify whether the effects that we observe here are also valid for discrete movements. Further studies are also required to understand whether and how the contagions we observed here are related to Motor entrainment, which is a phenomena predominantly defined for rhythmic auditory stimuli [START_REF] Tierney | Auditory-motor entrainment and phonological skills: precise auditory timing hypothesis (PATH)[END_REF][START_REF] Schachner | Spontaneous motor entrainment to music in multiple vocal mimicking species[END_REF]. In our case we used white noise feedback to stop the participants from hearing noise from the moving robot. However, having said that, it is possible that the effect we observe here may be a form of visual Motor entrainment.

The performance frequencies of the participants were affected by the human and humanoid robot co-workers, their (press) task accuracy remained undisturbed (Fig. 7). The effect on the human co-worker's frequency and the absence of an effect on his accuracy, suggests that the performance of the human participants is affected by the presence of a robot co-worker; a slower robot co-worker reduces human performance (in terms of speed and accuracy), while a faster robot co-worker improves it. Previous studies have shown that specialized robots can influence both human performance and motivation during physical [START_REF] Takagi | Physically interacting individuals estimate the partner's goal to enhance their movements[END_REF] and cognitive [START_REF] Fasola | Robot motivator: Increasing user enjoyment and performance on a physical/cognitive task[END_REF] interactions, our results here show that the mere presence of humanoid robots can induce effects in human performance.

Interestingly, the effect on the movement frequency was observed only when the head and torso of the co-worker was visible to the participants (Fig. 6), indicating the crucial importance of the human form for these effects. Note that in order to investigate the effect of the visibility of the co-worker torso, we chose to cover the torso of the humanoid robot instead of using a different manipulator as a co-worker due to two main October 30, 2018 11/15 reasons. Primarily, this enabled us to create a condition where the physical appearance of the robot arm and its movement were identical between the robot co-worker and robot covered co-worker conditions. Furthermore, this helped us clarify that the contagions are not influenced by the presence of a humanoid co-worker (and the participant's knowledge of it), but rather by the torso visibility. Both these issues would have been unclear with the use of a manipulator as a co-worker. On the other hand, our results open several new questions for future research perspectives. First, we observed that the visibility of the robot co-worker's torso modulates contagions in a human co-worker, but the reasons behind this are still unclear. The effect is probably related to aspects of saliency as the torso not only occupies a larger visual field, but (especially the head and the eyes) also probably attracts participant attention when present. Second, in our task we examined the case where the robot co-worker made predominantly arm movements, while the torso remained static. While we believe the effect of the torso's visibility should increase in tasks in which the torso moves, it remains to clarify how the torso movements affect contagions. Finally, while here we analyze a task where both co-workers (human and humanoid robot HRP-2Kai) and participants perform the same task, it would be interesting to analyze whether and how the contagions manifest in settings where the co-workers and participants work on different tasks, including non-industrial task that are explicitly collaborative, or competitive.

Quantitatively, the trends we observed were significant but not that substantial.

However, within our participants, these trends were observed to increase with their participant's robot experience (Fig. 8), suggesting that they can prevail over a long time and are thus may important in scenarios involving long time robot-human interactions.

Note that the questions used to quantify robot exposure represent the self perceived robot exposure by the participants rather than the actual robot exposure. A standard questionnaire to access actual robot exposure is absent and the development of one can be useful to understand how the effects, such as the one we highlight here, vary over time.

Overall, the results of our exploratory study highlight several new features of motor contagions, but also opens new questions for future research. These results can be useful for customizing the design of robot co-workers in industries and sports in order to moderate or exploit the contagions induced by them; contagions such as those related to body postures and undesirable competitions and that may affect worker health and psychology in long run, may be reduced by controlling the physical appearance and/or kinematics of robot co-workers, while where ethically valid, contagions may also be used to improve worker performance speed and hence productivity.

Fig 1 .

 1 Fig 1. Experimental setup. The participants in our experiment worked in six conditions; with a robot performing biological movements in A) robot co-worker condition; B) human co-worker condition; to check relevance of human form in C) robot covered co-worker condition; and D) human covered co-worker condition; E) a robot co-worker performing non-biological movements in robot non-biol co-worker condition; F) a robot co-worker performing industrial movements. The coordinate axis defining the movement setup is indicated in white (A).

A

  total of 45 healthy adults participated in our study. 3 participants (2 males and a female of 3 nationalities, 29.6±5, mean±SD, aged[START_REF] Verma | Determination of Sample Size and Power Analysis with G*Power Software: Step-wise Illustrated Manual for Research Scholars[END_REF][START_REF] Harris | Signal-dependent noise determines motor planning[END_REF][START_REF] Fitts | The information capacity of the human motor system in controlling the amplitude of movement[END_REF][START_REF] Burdet | The central nervous system stabilizes unstable dynamics by learning optimal impedance[END_REF][START_REF] Franklin | CNS learns stable, accurate, and efficient movements using a simple algorithm[END_REF][START_REF] Ganesh | Motor planning explains human behaviour in tasks with multiple solutions[END_REF][START_REF] Vannucci | Adaptation to a humanoid robot in a collaborative joint task[END_REF][START_REF] Schaal | Rhythmic arm movement is not discrete[END_REF][START_REF] Tierney | Auditory-motor entrainment and phonological skills: precise auditory timing hypothesis (PATH)[END_REF][START_REF] Schachner | Spontaneous motor entrainment to music in multiple vocal mimicking species[END_REF][START_REF] Takagi | Physically interacting individuals estimate the partner's goal to enhance their movements[END_REF] worked as volunteer models for the capture of human arm motion data. 42 participants (20 Males and 22 females of 12 nationalities, 25.9±4.35, mean±SD, min. age 20, max. age 39), were participated as 'co-workers' in our main experiment. 3 out of 45 participants were left-handed according to the Edinburgh Handedness Inventory, and all participants had normal or corrected to normal vision. The experiments were approved by the local ethics committee at the National Institute of Advanced Industrial Science and Technology (AIST) in Tsukuba, Japan, and all participants read and signed an informed consent form along with the PLOS consent form for the usage of their images in the paper before taking part in the experiments. Participants were well instructed and informed with the experiment and task procedure, however they were naïve to the motives (participants were not told October 30, 2018 2/15

Fig 2 .

 2 Fig 2. Trial protocol. The participants worked in repeated trials with either a robot or human co-worker (the figure shows the trial with a robot co-worker). Each trial consisted of period when the participant worked alone and co-worker relaxed (participant-alone period), both worked together (together period), and the co-worker worked alone (co-worker-alone period). The notation of the time variable (represented in general by τ ) in each period are shown.

Fig 3 .

 3 Fig 3. HRP-2Kai movement trajectories. The trajectories played by the robot in robot co-worker, robot covered co-worker, robot non-biol co-worker and robot indus co-worker conditions.

Fig 4 .

 4 Fig 4. HRP-2Kai trajectory generation. The time trajectories in the Y and Z axis by the HRP-2Kai in the robot non-biol co-worker and robot indus co-worker condition, and the via-points (blue circles) used to generate both trajectories.

  function for each participant. The tangent slope at the minimum data abscissa value (min[τ c (i) -Av(τ a p )]) was collected across participants, checked for normality using the Shapiro-Wilk test and then analyzed for difference from zero using a one sample T-test (in case the distribution was normal) or a Signed Rank test. The fitting of htp in one sample participant from each of the six reported conditions are shown in Fig. 5, and the plot of the collection of slopes are in shown in Fig. 6. A similar procedure was used to analyze the change in a participant's average X press location, average Y press location, standard deviation of X press location, and standard deviation of Y press locations relative to the htp of the co-worker behavior in the same trial (τ c (i) -Av(τ a p )). The slopes from these analysis are shown in Fig. 7.

Fig 5 .

 5 Fig 5. Examples of linear regression fits. The change of participant's htp (the average time between two consecutive alternate touches) between the together period and alone-period (τ t p (i) -τ a p (i)), relative to the htp of the co-worker behavior in the same trial (τ c (i) -Av(τ a p )), where Av(τ a p ) represents the average undisturbed htp by a participant across all his/her participant-alone periods. Note that the (robot or human) co-worker htp was random across trials, and the data in plots here are the ensemble of the participant behaviors arranged in increasing co-worker's htp on the abscissa. Each plot represent a condition, A) robot co-worker (blue); B) human co-worker (orange); C) robot covered co-worker (dark blue); D) human covered co-worker (dark orange); E) robot non-biol co-worker (cyan); F) robot indus co-worker (magenta) conditions. We used the AIC to choose either a first or second order model to fit the data for each participant. The lines represent the tangent slopes at the minimal data abscissa value.

Fig 6 .

 6 Fig 6. All six conditions htp comparision. The plot of the collection of slopes which is obtained in Fig. 5 and S1 Fig to S6 Fig supplementary figures. The condition-wise comparison of the change of participants htp with co-worker htp. P-values are Bonferroni corrected where required. The tangent slope at the minimum data abscissa value (min[τ c (i) -Av(τ ap )]) was collected across participants (as shown in Fig.5), checked for normality using the Shapiro-Wilk test and then analyzed for difference from zero using a one sample T-test (in case the distribution was normal) or a Signed Rank test.

Fig 7 .

 7 Fig 7. Participants touch variance. Change of participant touch position with A) robot co-worker htp; B) human co-worker htp. A similar procedure which was used to quantify htp was also used here (see subsection Data analysis) to analyze the change in a participant's average X press location, average Y press location, standard deviation of X press location, and standard deviation of Y press locations relative to the htp of the co-worker behavior in the same trial.

Fig. 6 )

 6 Fig. 6), and these values were different (tending to significance) compared to the robot co-worker condition of the same participants (robot non-biol condition: T(16)=2.32, p=0.066, Bonferroni corrected, one sample T-test) and (robot indus condition: T(17)=2.53, p=0.043, Bonferroni corrected, one sample T-test).

Fig 8 .

 8 Fig 8. Robot experience exposure. The plot of the change of participant htp, with respect to their prior robot exposure and experience (self-scored by participants) showed a significant correlation between the two.

Table 2 .

 2 Participant sample size

	condition	sample size
	robot co-worker	39
	human co-worker	13
	robot covered co-worker	17
	human covered co-worker	13
	robot non-biol co-worker	17
	robot indus co-worker	18

  )=2.74, p=0.028, Bonferroni corrected, one sample T-test between robot co-worker and robot covered co-worker; T(12)=2.50, p=0.054, Bonferroni corrected, one sample T-test between robot co-worker and human covered co-worker).

Acknowledgement

We much thank Dr. Pierre Gergondet and Dr. Hervé Audren for their constant support to operate HRP-2Kai during the experiments and Dr. Eiichi Yoshida for handling the AIST Ethical procedures.

Supporting information S1 Video. The video of our experiment explaining task protocol and data analysis is available here. https://goo.gl/9rqv4G S1 Fig. All participants regression fits in the robot co-worker condition.

Examples of linear regression fits obtained between the participant's htp change between the together and alone conditions (ordinates), as a function of co-worker's htps (abscissa). Note that most participant plots show a positive slope indicating that the robot co-worker's performance htp (hence frequency) influenced the human participants. change between the together and alone conditions (ordinates), as a function of co-worker's htps (abscissa). Note that there is no trend in slopes across participant -the slopes were in fact observed to be zero across participants (Fig. 6), indicating that the participant's htps were not affected in the robot covered co-worker condition.


Keywords: Image retrieval, Grasping user's intention, Positive examples, Negative examples, Ontology

Understanding user's intention is at the core of an effective images retrieval systems. It still a significant challenge for current systems, especially in situations where user's needs are ambiguous. It is in this perspective that fits our study. In this paper, we address the challenge of grasping user's intention in semantic based images retrieval. We propose an algorithm that performs a thorough analysis of the semantic concepts presented in user's query. The proposed algorithm is based on an ontology and takes into account the combination of positive and negative examples. The positive examples are used to perform generalization and the negative examples are used to perform specialization which considerably decrease the two famous problems of image retrieval: noise and miss. Our algorithm processed in two steps: in the first step, we deal only with the positive examples where we will generalize the query from the explicit concepts to infer the others hidden concepts desired by the user. whereas the second step deal with the negative examples to refine results obtained in the first step. We created an image retrieval system based on the proposed algorithm. Experimental results show that our algorithm could well capture user's intention and improve significantly precision and recall.

Understanding user's intention in semantic based image retrieval: combining positive and negative examples

Meriem KORICHI 1 ,Mohamed Lamine KHERFI 2 , Mohamed BATOUCHE 1 ,Zineb KAOUDJA 2 , Hadjer BENCHEIKH 2 1 Introduction

Image retrieval is a growing field. Recently, it has witnessed an explosion of personal and professional image collections especially with the development of new technologies that allow users share their images via internet. There was an urgent need for automatic tools that organizing these large quantities of images.

Such tools can help users to find the desired images within a reasonable time. These tools are called image retrieval engines. Image retrieval systems can be classified into two main categories: the first category called Content-Based Image Retrieval (CBIR). In this category, the user is generally asked to formulate a visual query (image for example) that correspond to what he is looking for. The research is performed by measuring similarities between the low levels features of the query and the images of the collection. Various visual features, including both global features [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF][START_REF] Torralba | Context-Based Vision System for Place and Object Recognition[END_REF][START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF] (e.g., color, texture, and shape) and local features (e.g., SIFT keypoints [START_REF] Lowe | Distinctive Image Features from Scale-Invariant Keypoints[END_REF][START_REF] Celik | Content based image retrieval with sparse representations and local feature descriptors: A comparative study[END_REF][START_REF] Guillaumin | Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation[END_REF]) have been studied for CBIR.

The second category which exploits the semantic concepts associated with images is called Semantic-Based Image Retrieval : SBIR. The user formulates his/her query with textual terms that express his/her needs. In this case, the search is performed by comparing between these terms and textual annotations that represent the images in the collection by the use of semantic tools. Among the tools that are often used to represent the semantics, there are "ontologies" [START_REF] Liaqat | Image retrieval based on fuzzy ontology[END_REF].

There is another secondary category where the user formulates his/ her query with visual example like CBIR but the research is carried out on the basis of the semantics associated with those images instead of the low levels features [START_REF] Rasiwasia | Bridging the gap: Query by semantic example[END_REF]. This category try to overcome the drawbacks of the two previous categories by avoiding user to identify keywords and letting the images speak for themselves. Whatever the category of the image retrieval system, Their main objective is to understand user's needs and answer him/her accurately. However, these needs may vary from one person to another, and even from a given user at different times; therefore, the success of an image retrieval system depends on how accurate the system understands the intention of the user when he/she formulates a query [START_REF] Lamine | Review of Human-Computer Interaction Issues in Image Retrieval[END_REF]. Creating the query is a difficult problem. It arises two important challenges for both the user and the retrieval system [START_REF] Lamine | Review of Human-Computer Interaction Issues in Image Retrieval[END_REF]: Firstly, for the user the challenge is how to express his/her needs accurately. Secondly, for the system the challenge is how to understand what the user wants based on the query that he/she formulated.

The difficulty in dealing with these two challenges lead to many problems in image retrieval, such as noise and miss. Noise can be defined as the set of retrieved images which do not correspond to what the user wants [START_REF] Kherfi | Combining positive and negative examples in relevance feedback for content-based image retrieval[END_REF] whereas, miss is the set of images corresponding to what the user wants which have not been retrieved.

In this paper, our objective is to improve the efficiency of image retrieval by decrease noise and miss via understanding user's intention. We will be interested by the systems of the last category where the user formulates a visual query using images and the retrieval system exploits the semantic concepts associated with these images .

We propose an algorithm that performs a thorough analysis of the semantic concepts presented in the user's query. Our algorithm is processed in two steps: in the first step, we will perform a generalization from the explicit concepts to infer the other hidden concepts that the user can't express them which decrease the miss problem. In the second step, we ask user to select negative examples to refine results obtained in the first step. In this case the images similar to those of the negative examples will be rejected, which reduces the noise. At the same time, the rejected images are replaced by others correspond to what the user wants, so miss will be decrease also. The rest of the paper is organized as follows: section 2 briefly reviews previous related work. section 3 present the principle of our algorithm , then, we will present our data and our working hypotheses in section 4. Section 5 details our algorithm. An experimental evaluation of the performance of our algorithm is presented in section 6. Finally, section 7 presents a conclusion and suggests some possibilities for future research.

Related works

User is at the centre of any image retrieval system. One of the primary challenges for these systems is to grasp user intent given the limited amount of data from the input query [START_REF] Tang | In-tentSearch:Capturing user intention for one-click internet image search[END_REF]. In order to solve this challenge, several methods has been proposed. One way is keyword expansion where the users original query is augmented by new concepts with a similar meaning to decrease the miss problem. Keyword expansion was implemented by different techniques. Some of them [12][START_REF] Navigli | Word sense disambiguation: A survey[END_REF][START_REF] Azizan | Analysis of retrieval result on ontologybased query reformulation[END_REF] are typically based on dictionaries, thesauri, or other similar knowledge models such as ontology. It is noted by many researchers that the use of an ontology for query expansion is advantageous especially if the query words are disambiguated [11][15]. Note that query expansion is very effective, provided that initial query perfectly matches the user's need [START_REF] Fergus | A Visual Category Filter for Google Images[END_REF], However , this is not always the case, especially in situation where the user has a difficulty to express them . In this case, the extensions of the query can lead to more inappropriate results than the initial query because of irrelevant concepts which leads to significantly increase noise [START_REF] Park | Majority Based Ranking Approach in Web Image Retrieval[END_REF] .

Another way to grasp user's intention is relevance feedback [START_REF] Kherfi | Combining positive and negative examples in relevance feedback for content-based image retrieval[END_REF][START_REF] Deng | Hierarchical Semantic Indexing for Large Scale Image Retrieval[END_REF][START_REF] Glowacka | Image Retrieval with a Bayesian Model of Relevance Feedback[END_REF][START_REF] Tao | Asymmetric Bagging and Random Subspace for Support Vector Machines-Based Relevance[END_REF] and pseudo relevance feedback. In Relevance feedback technique, the user choose from the returned images those that he/she finds relevants (positive examples) and those that he/she finds not relevants (negative examples) then a query-specific similarity metric was learned from the selected examples.

In pseudo relevance feedback [START_REF] Torjmen | Using Pseudo-relevance feedback to improve image retrieval results[END_REF][START_REF] Yan | Multimedia Search with Pseudo-Relevance Feedback[END_REF], the query is expanding by taking the top N images visually most similar to the query image as positive examples.

For many researchers [START_REF] Kang | Query type classification for web document retrieval[END_REF][START_REF] Lee | Automatic identification of user goals in web search[END_REF][START_REF] Figueroa | Ensembling Classifiers for Detecting User Intentions behind Web Queries[END_REF], The task of learning concepts from user's query to understand his/her intent has seen as a supervised learning problem in which the classifiers search through a hypothesis space to find an adequate hypothesis that will make good predictions. Several algorithms of supervised learning were proposed [START_REF] Krammer | On the algorithmic implementation of multi-class svms[END_REF][START_REF] Hsu | A comparison of methods for multiclass support vector machines[END_REF][START_REF] Nigam | Using Maximum Entropy for Text Classification[END_REF][START_REF] Foody | Hard and soft classifications by a neural network with a non-exhaustively defined set of classes[END_REF] such us: Multi-Class Support Vector Machines (SVM), Nave Bayes Classifier, Maximum Entropy (MaxEnt) classifier, Multi-Layer Perceptron (MLP).

Recent search [START_REF] Figueroa | Ensembling Classifiers for Detecting User Intentions behind Web Queries[END_REF], proposed an ensembling multiple classifiers for detecting users intention instead of focusing on single classifiers. This problem has an analogue in the cognitive sciences which called human generalization behavior and inductive inference. Bayesian models of generalization [START_REF] Tenenbaum | Generalization, similarity,and Bayesian inference[END_REF][START_REF] Tenenbaum | Word learning as Bayesian inference[END_REF][START_REF] Abbott | Constructing a hypothesis space from the Web for large-scale Bayesian word learning[END_REF][START_REF] Jia | Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies[END_REF] are models from cognitive science that focus on understanding the phenomena of how to generalize from few positive examples. They has been remarkably successful at explaining human generalization behavior in a wide range of domains [START_REF] Jia | Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies[END_REF], however their success is largely depend on the reliability of the initial query.

The main contribution of our work is to propose an algorithm to perform the challenge of grasping user's intention and consequently decrease miss and noise.

Our algorithm refines the initial query formulated by the user to infer the relevant concepts. It uses an ontology and exploits its semantic richness which allowed us to extract the hidden concepts that the user could not express them explicitly.

Then we can improve the obtained results through the relevance feedback with negative examples.

Principle of our algorithm

In medicine, doctors apply a characteristic rule to summarize the symptoms of a specific disease, whereas, to distinguish one disease from others, they apply a discrimination rule to summarize the symptoms that discriminate this disease from others. This principle is exactly what we will use in our algorithm to understand user's intention from a given query. We perform generalization using positive examples and specialization using negative examples. This principle is applied in content-based image retrieval [START_REF] Kherfi | Combining positive and negative examples in relevance feedback for content-based image retrieval[END_REF] as follows:

-For positive examples, a concentrated characteristic is important and a scattered characteristic is not important. -For positive and negative examples, a discriminated characteristic is important and a non-discriminated characteristic is not important.

We carry out a similar work with semantic concepts as follows:

1. For positive examples: with a query that contains several images: (a) A concept which is salient in the majority of query's images is important. (b) A concept that appears in some images and does not appear in the others, in this case, we cannot judge immediately that it is not important. However, it can be classified into three types:

i. Either, it has concepts that reinforces it in the other images via direct or indirect relationships (for example: apple -fruit -banana), and so it is important, or at least the common concept (here it is the ancestor Fruit) is important.

DATA AND HYPOTHESIS

ii. Either it has concepts that contradicts it in the other images, directly or indirectly, and therefore it is not important. iii. Either there is no concept that reinforces it or a concept that contradicts it in the other images, in this case, it can either be considered with moderate importance, or neglected.

For positive and negative examples:

(a) The common concepts between the two sets (positive and negative) are to be neglected. common concept mean either the same concept or concepts that have a positive relations between them as it is illustrated in the case of positive example only. (b) Discriminant concepts (directly or indirectly through relationships) between the two sets are important. If the concept is present in the positive set, it must be sought; and if it is present in the negative set, it must be dismissed.

4 Data and hypothesis

Query formulation

We ask user to select from displayed images the maximum number of images that correspond to what he/she is looking for. It is the set of concepts that annotate the selected images that will constitute our initial query.

Ontology creation

Our algorithm use an ontology to exploit its semantic richness and apply the principle describe above. We have chosen the domain of "nature" for our ontology. The first step of the ontology creation is to choose a collection of images representative of "Nature" domain. The ontology will then be used to annotate the images in this collection. Relationships in our ontology are classified into three categories:

1. Positive relationships (reinforcement relationship): means that the presence of such concept implies the presence of such other concept, for example: presence of concept "Dune" implies the presence of concept "Desert". 2. Negative relationships (contradiction relationships): means that the presence of such concept generally implies the absence of such other concept, for example: presence of the concept "Snow" implies the absence of concept "Desert". 3. Neutral relationships: means that the relations between concepts are null (ie neither positive nor negative)

The other steps of the ontology creation are the following:

1. Identify the different concepts presents in the images collection: for example: desert, camel, dune, snow,..etc 2. Calculate the implications weights between concepts that illustrate the degree of semantic correlation between these concepts. For each concept, the probability of having the other concepts was calculated as follows:

P (A|B) = P (A ∩ B) P (B)
Where P (A | B) means the probability of having a concept A knowing that we are in the presence of the concept B. Because our challenge is not the ontology creation, we have calculated this probability manually. Firstly, we started the search with the concept B using the retrieval engine "Google". The obtained images represent P(B). In this collection, we calculate the number of images that represents the concept A which gives us P (A ∩ B). 3. Represent the ontology by a value oriented graph where concepts represent the nodes and the arcs are the semantic relationships. Our graph can be seen as a probabilistic graph. 4. Represent the relationships between concepts with a relevance matrix. This latter is a square matrix whose lines and columns are concepts and the value in box M [i, j] represents the probability of the relation between the concept i and the concept j.

A weight between two concepts models a type of semantic relationship between them.

* The weights with the value 0: mean that the two concepts can never be found in the same image so this weight models a negative relationship; * [0,01 0, 49]: meant that there are times when the two concepts can be found in the same image and sometimes it cannot be found. Which models a neutral relationship; * [0. 5 1]: means that the two concepts are often found in the same image so this weight models a positive relationship.

Our algorithm

Our algorithm is processed in two steps. We called the first step: "Generalization algorithm" (see Algorithm 1)and the second step "Specialization algorithm(see Algorithm 2))

6 Experimental evaluation

Images collection

To be able to apply our algorithm, we need an ontology and an images collection annotated with the concepts of this ontology. We have chosen the domain of "nature" for our ontology. Then, we have used Google images as an image retrieval engine to collect images for each concept in the ontology. The top 200 returned images of each query are crawled and manually labeled to construct the experimental data set.

Experiments

We implemented an image retrieval system that integrate our algorithm. We invited thirty users to participate in our experiments. We displayed them a set of images randomly selected from the images collection. Users can ask to see more images if the presented images do not reflect to what they wants.

To evaluate the performance of our algorithm, we carried out a comparison between: our algorithm, the Bayesian model of generalization used in [START_REF] Jia | Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies[END_REF] and the standard matching method (without generalization). As performance criteria, we use the precision and the recall measures.

First experiment The aim of this experiment is to evaluate the effectiveness of the first step of our algorithm (Generalization algorithm) to increase recall compared to the Bayesian model of generalization used in [START_REF] Jia | Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies[END_REF] and the standard matching method (without generalization). We asked users to make different queries, and then we calculated the recall of the three models. Fig. 2 illustrate the obtained results from 16 queries. We can see that the recall is considerably increased with our approach and then consequently the miss problem is decreased. Compared to our algorithm, the success of the Bayesian model of generalization is largely depend on the reliability of the examples present in the query to good reflect the user's intention which is not always the case because sometimes the images of the query may composed of many objects, however the user is interest only with some of them.

Our algorithm overcomes this problem by refining the initial query of the user.

Second experiment The aim of this experiment is to illustrate the effect of the negative examples to decease noise by improving the accuracy. We asked users to formulate queries using the Generalization algorithm and we calculated the accuracy then we asked him to select negative images and restart the search. the obtained results are illustrated in Fig. 3.

It is clear that the use of negative examples increase the accuracy. Indeed, after obtaining the results of a given query, the user can keep the positive images and enrich the query by including some undesired images as a negative examples. This implies that images similar to those of the negative example will be rejected, which reduces the noise. At the same time, the rejected images are replaced by others corresponding to what the user wants, so silence will be diminished.

Conclusion

In this paper, we proposed an algorithm that process in two steps: we called the first step "Generalization algorithm" and the second step "specialization algorithm". The objective of this algorithm is to improve the semantic image retrieval quality by reducing the famous problems of image retrieval: noise and miss via understanding user's intention. Our algorithm is based on the use of an ontology which allows us to exploits it's semantic richness. We implemented an image retrieval system that integrate the proposed algorithm. Experimental evaluation verify the efficiency of our approach in comparison to the others proposed in the state-of-the-art. We have shown that the combination of positive and negative examples improve significantly precision and recall. The perspective of our work is to automate the ontology creation.

Fig. 1 .

 1 Fig. 1. The recall of the three models

Fig. 2 .

 2 Fig. 2. The effect of negative examples to increase accuracy.

Algorithm 1 Generalization

Background: a set of an annotated images D Preprocess: Extract the list of semantic concepts (LCpt)


Henry Chiu
email: h.chiu16@imperial.ac.uk

Rama Cont
email: rama.cont@math.cnrs.fr

On pathwise quadratic variation for càdlàg functions

Keywords: Quadratic variation, semimartingale, pathwise calculus, Ito formula, pathwise integration, cadlag functions, Skorokhod topology

We revisit Föllmer's concept of quadratic variation of a càdlàg function along a sequence of time partitions and discuss its relation with the Skorokhod topology. We show that in order to obtain a robust notion of pathwise quadratic variation applicable to sample paths of càdlàg processes, one must reformulate the definition of pathwise quadratic variation as a limit in Skorokhod topology of discrete approximations along the partition. One then obtains a simpler definition which implies the Lebesgue decomposition of the pathwise quadratic variation as a result, rather than requiring it as an extra condition.

1 Quadratic variation along a sequence of partitions

In his seminal paper Calcul d'Itô sans probabilités [START_REF] Föllmer | Calcul d'Itô sans probabilités[END_REF], Hans Föllmer introduced a pathwise concept of quadratic variation and used it to provide a pathwise proof of the Itô formula. Föllmer showed that if a function x ∈ D([0, T ], R) has 1 quadratic variation along a sequence π n = (t n 0 = 0 < .. < t n j < ... < t n m(n) = T ) of time partitions of [0, T ] in the sense that for each t ∈ [0, T ] the limit

µ n := [t n j ,t n j+1 ]∈πn δ(• -t j ) |x(t n j+1 ) -x(t n j )| 2 (1)
converges weakly to a Radon measure µ π such that

[x] c π : t → µ π ([0, t]) -s≤t |∆x(s)| 2 is a continuous increasing function, (L)
then a pathwise Itô formula may be obtained for functions of x [START_REF] Föllmer | Calcul d'Itô sans probabilités[END_REF]: for any

f ∈ C 2 (R, R), f (x(t)) -f (x(0)) = t 0 f (x)d π x + 1 2 t 0 f (x(s))d[x] c π (s) + 0≤s≤t ( f (x(s)) -f (x(s-)) -f (x(s-))∆x(s) ) (2)
where t 0 f (x)d π x is a pathwise integral defined as a limit of left Riemann sums computed along the partition:

t 0 g(x)d π x := lim n→∞ πn g(x(t n j )). x(t n j+1 ) -x(t n j ) . (3)
The quantity

[x] π (t) = µ π ([0, t]) = [x] c π (t) + s≤t |∆x(s)| 2
is called the quadratic variation of x along π. This result has many interesting applications and has been extended to less regular functions [START_REF] Bertoin | Temps locaux et intégration stochastique pour les processus de Dirichlet[END_REF][START_REF] Cont | Pathwise integration and change of variable formulas for continuous paths with arbitrary regularity[END_REF][START_REF] Davis | Pathwise stochastic calculus with local times[END_REF][START_REF] Perkowski | Local times for typical price paths and pathwise Tanaka formulas[END_REF] and path-dependent functionals [START_REF] Ananova | Pathwise integration with respect to paths of finite quadratic variation[END_REF][START_REF] Cont | Functional Ito Calculus and functional Kolmogorov equations, Stochastic Integration by Parts and Functional Ito Calculus[END_REF][START_REF] Cont | A functional extension of the Ito formula[END_REF][START_REF] Cont | Change of variable formulas for non-anticipative functionals on path space[END_REF][START_REF] Cont | Pathwise integration and change of variable formulas for continuous paths with arbitrary regularity[END_REF]. With the exception of [START_REF] Cont | Change of variable formulas for non-anticipative functionals on path space[END_REF][START_REF] Vovk | Itô calculus without probability in idealized financial markets[END_REF][START_REF] Rafal | A superhedging approach to stochastic integration[END_REF], these extensions have focused on continuous paths. Föllmer's definition [START_REF] Föllmer | Calcul d'Itô sans probabilités[END_REF] contains the condition (L) on the Lebesgue decomposition of the limit µ π : the atoms of µ should correspond exactly to the jumps of x and their mass should be |∆x(t)| 2 or, equivalently, the discontinuity points of [x] π should coincide with those of x, with ∆[x] π (t) = |∆x(t)| 2 . This condition can not be removed: as shown by Coquet et al. [START_REF] Coquet | Natural decomposition of processes and weak Dirichlet processes[END_REF], there are counterexamples of continuous functions x such that (1) converges to a limit with atoms. Conversely, one can give examples of discontinuous functions for which (1) converges to an atomless measure. If this condition is not satisfied, then the pathwise integral (3) fails to satisfy

∆ t 0 x(s-)dx(s) (t) = x(t-)∆x(t),
at each t where condition (L) is not met. On the other hand, this condition (L) is not easy to check and seems to require a link between the path x and the sequence of partitions π, making it difficult to apply to sample paths of stochastic processes.

In this work, we revisit Föllmer's concept of pathwise quadratic variation along a sequence of partitions and show that it has hitherto unsuspected links with the Skorokhod topology. In particular, we show that in order to obtain a robust notion of pathwise quadratic variation applicable to sample paths of càdlàg processes, one must reformulate the definition of the quadratic variation as a limit, in Skorokhod topology, of discrete approximations defined along the partition. This leads to a simpler definition of pathwise quadratic variation which holds in any dimension and, rather than requiring the Lebesgue decomposition of the pathwise quadratic variation as an extra condition, yields it as a consequence.

Outline We begin by recalling Föllmer's definition of pathwise quadratic variation and variations of it which have been used in the literature. We then introduce a new definition of quadratic variation for real-valued càdlàg functions based on the Skorokhod topology and prove equivalence among the various definitions. Section 3 extends the results to vector-valued functions: we show that, unlike Föllmer's original definition in which the one dimensional case plays a special role, our definition applies regardless of dimension, thus simplifying various statements regarding quadratic variation for vector-valued functions. Finally, in Section 4, we show that our approach leads to simple proofs for various properties of pathwise quadratic variation.

2 Pathwise quadratic variation for cadlag functions

Let π := (π n ) n≥1 be a sequence of partitions π n = (t n 0 , ..., t n kn ) of [0, ∞) into intervals 0 = t n 0 < ... < t n kn < ∞; t n kn ↑ ∞ with vanishing mesh |π n | ↓ 0 on compacts. By convention, max(∅ ∩ π n ) := 0, min(∅ ∩ π n ) := t n kn . Denote D := D([0, ∞),
µ n := πn (x(t n i+1 ) -x(t n i )) 2 δ(t n i ) (4)
converges vaguely to a Radon measure µ on [0, ∞) with [x] π (t) = µ([0, t]), such that [x] c π defined by [x] c π (t) := µ([0, t]) -s≤t (∆x s ) 2 is a continuous increasing function. [x] π (t) = µ([0, t]
) is then called the quadratic variation of x along π, and admits the following Lebesgue decomposition:

[x] π (t) = [x] c π (t) + s≤t |∆x(s)| 2 . ( 5
)
We denote Q π 0 the set of x ∈ D satisfying these properties. At this point let us point out a link between vague and weak convergence of Radon measures on [0, ∞), a link which is well known in the case where the measures are sub-probability measures: Lemma 2.1. Let v n and v be non-negative Radon measures on [0, ∞) and J ⊂ [0, ∞) be the set of atoms of v, the followings are equivalent:

(i) v n → v vaguely on [0, ∞). (ii) v n → v weakly on [0, T ] for every T / ∈ J. Proof. Let f ∈ C K ([0, ∞)) be a compactly supported continuous function. Since J is countable, ∃ T / ∈ J; supp(f ) ⊂ [0, T ]. Now (ii) ⇒ ∞ 0 f dv n = T 0 f dv n -→ T 0 f dv = ∞ 0 f dv ⇒ (i). Suppose (i) holds, let T / ∈ J and f ∈ C ([0, T ], • ∞ ). Since f = (f ) + -(f ) -,
we may take f ≥ 0 and define the following extensions:

f (t) := f (t)1I [0,T ] (t) + f (T ) 1 + T -t 1I (T,T + ] (t) f (t) := f (t)1I [0,T -] (t) + f (T ) T -t 1I (T -,T ] (t), then f , f ∈ C K ([0, ∞)), 0 ≤ f ≤ f 1I [0,T ] ≤ f ≤ f ∞ and we have ∞ 0 f dv n ≤ T 0 f dv n ≤ ∞ 0 f dv n .
Since v n → v vaguely and T / ∈ J, thus, as n → ∞ we obtain Proposition 2.1. If x ∈ Q π 0 , then the pointwise limit s of

0 ≤ lim sup n T 0 f dv n -lim inf n T 0 f dv n ≤ ∞ 0 f -f dv ≤ f ∞ v ((T -, T + ]) -→ 0,
s n (t) := ti∈πn |x(t i+1 ∧ t) -x(t i ∧ t)| 2 (6)
exists, s = [x] π and s admits the Lebesgue decomposition:

s(t) = s c (t) + s≤t (∆x s ) 2 . ( 7
)
Proof. If x ∈ Q π 0 , define q n (t) := πn ti≤t (x(t i+1 ) -x(t i )) 2 ,
the distribution function of µ n in (4). Since µ n → µ vaguely, we have

q n → [x] pointwise at all continuity points of [x] (Lemma 2.1 & [12, X.11]). Let I be the set of continuity points of [x]. Observe (q n ) is monotonic in [0, ∞) and I is dense in [0, ∞), if t / ∈ I, it follows [12, X.8] that [x](t-) ≤ lim inf n q n (t) ≤ lim sup n q n (t) ≤ [x](t+) = [x](t).
Thus, we may take any subsequence (n k ) such that lim k q n k (t) =: q(t). Since x ∈ Q π 0 and the Lebesgue decomposition (5) holds on [x], we have

≥0 ([x](t + ) -q(t)) + ≥0 (q(t) -[x](t -))= [x](t + ) -[x](t -) →0 → |∆x(t)| 2 . ( 8
) If t ± ∈ I, πk := π n k and t (k) j := max{π k ∩ [0, t)}, the second sum in (8) is lim k ti∈π k ; t-<ti≤t (x(t i+1 ) -x(t i )) 2 = lim k ti∈π k ; t-<ti<t (k) j ≥0 (x(t i+1 ) -x(t i ) 2 +(∆x(t)) 2 ≥ (∆x(t)) 2
by the fact that x is càdlàg and that t / ∈ I. We see from (8) that q(t) = [x](t) as → 0. Since the choice of the convergent subsequence is arbitrary, we conclude that q n → [x] pointwise on [0, ∞). Observe that the pointwise limits of (s n ) and (q n ) coincide i.e.

|s n (t) -q n (t)| = (x t (n) i+1 -x(t)) 2 + 2(x t (n) i+1 -x(t))(x(t) -x t (n) i ) (9)
converges to 0 by the right-continuity of x, where t

(n) i := max {π n ∩ [0, t]} and that x ∈ Q π 0 , Prop. 2.1 follows.
Denote Q π 1 the set of x ∈ D such that (s n ) defined in (6) has a pointwise limit s with Lebesgue decomposition given by [START_REF] Cont | Weak approximation of martingale representations[END_REF]. Then Q π 0 ⊂ Q π 1 and we have:

Proposition 2.2. If x ∈ Q π 1 , then the pointwise limit q of q n (t) := πn ti≤t (x(t i+1 ) -x(t i )) 2 (10)
exists. Furthermore q = s and admits the Lebesgue decomposition:

q(t) = q c (t) + s≤t (∆x s ) 2 . ( 11
)
Proof. Since the pointwise limits of (s n ) and (q n ) coincide by [START_REF] Coquet | Natural decomposition of processes and weak Dirichlet processes[END_REF]. Prop. 2.2 now follows from

x ∈ Q π 1 .
Denote Q π 2 the set of x ∈ D such that the quadratic sums (q n ) defined in (10) have a pointwise limit q with Lebesgue decomposition [START_REF] Davis | Pathwise stochastic calculus with local times[END_REF]. Then

Q π 0 ⊂ Q π 1 ⊂ Q π
2 and we have:

Proposition 2.3. If x ∈ Q π 2
, then q n → q in the Skorokhod topology.

Proof. Since x ∈ Q π 2 , we have q n → q pointwise on [0, ∞) and that (q n ), q are elements in D + 0 . By [16, Thm.VI.2.15], it remains to show that s≤t

(∆q n (s)) 2 n→∞ -→ s≤t (∆q(s)) 2 on a dense subset of [0, ∞). Let t > 0, define J := {s ≥ 0|(∆X s ) 2 ≥ 2 }, J n := {t i ∈ π n |∃s ∈ (t i , t i+1 ]; (∆X s ) 2 ≥ 2 } ⊂ π n and observe that s≤t (∆q n (s)) 2 = πn ti≤t (x(t i+1 ) -x(t i )) 4 = J n ti≤t (x(t i+1 ) -x(t i )) 4 + (J n ) c ti≤t (x(t i+1 ) -x(t i )) 4 . ( 12
)
Since x is càdlàg and that |π n | ↓ 0 on compacts, the first sum in ( 12) converges to J s≤t (∆x s ) 4 and the second sum in ( 12)

(J n ) c ti≤t (x(t i+1 ) -x(t i )) 4 ≤ sup (J n ) c ti≤t (x(t i+1 ) -x(t i )) 2 (J n ) c ti≤t (x(t i+1 ) -x(t i )) 2 ≤ q(t)
for sufficiently large n [6, Appendix A.8] hence

lim n s≤t (∆q n (s)) 2 = J s≤t (∆x s ) 4 + ≤ q(t) lim sup n (J n ) c ti≤t (x(t i+1 ) -x(t i )) 4 .
By the Lebesgue decomposition [START_REF] Davis | Pathwise stochastic calculus with local times[END_REF], we observe J s≤t (∆x s ) 4 ≤ q(t) 2 and that

lim n s≤t (∆q n (s)) 2 = s≤t (∆x s ) 4 = s≤t (∆q(s)) 2
as → 0.

Denote Q π the set of càdlàg functions x ∈ D such that the limit q of (q n ) exists in (D, d).

Then Q π 0 ⊂ Q π 1 ⊂ Q π 2 ⊂ Q π and we have: Proposition 2.4. Q π ⊂ Q π 0 and q = [x].
Proof. Let x ∈ Q π and I be the set of continuity points of q. [16, VI.2.1(b.5)] implies that q n → q pointwise on I. Since q n ∈ D + 0 and I is dense on [0, ∞), it follows q ∈ D + 0 . Denote µ to be the Radon measure of q on [0, ∞), observe the set of atoms of µ is J := [0, ∞)\I and that (q n ) are the distribution functions of the discrete measures (µ n ) in (4). Thus, by (Lemma 2.1 & [12, X.11]), we see that µ n -→ µ vaguely on [0, ∞).

If t > 0, put t

(n) i := max{π n ∩ [0, t)}. Since |π n | ↓ 0 on compacts, we have t (n) i < t, t (n) i ↑ t and t (n) i+1 ↓ t. Observe that ∆q n (t) = (x(t i+1 ) -x(t i )) 2 , if t = t i ∈ π n . 0, otherwise. (13)
If ∆q(t) = 0, [16, VI.2.1(b.5)] implies that ∆q n (t

(n) i ) → ∆q(t).
Hence, by the fact that x is càdlàg , (∆x(t)) 2 = lim n ∆q n (t i ) for all n sufficiently large, else we will contradict ∆q(t) > 0. Thus, (∆x(t)) 2 = lim n ∆q n (t

(n) i ) = ∆q(t). If ∆q(t) > 0,
(n) i ) = lim n ∆q n (t n ) = ∆q(t)
and the Lebesgue decomposition (5) holds on q.

By Def.2.1, we have q

= [x] hence Q π ⊂ Q π 0 .
Theorem 2.1. Let

• Q π 0 be the set of x ∈ D satisfying Definition 2.1.

• Q π 1 the set of x ∈ D such that (s n ) defined in (6) has a pointwise limit s with Lebesgue decomposition given by [START_REF] Cont | Weak approximation of martingale representations[END_REF].

• Q π 2 the set of x ∈ D such that the quadratic sums (q n ) defined by (10) have a pointwise limit q with Lebesgue decomposition [START_REF] Davis | Pathwise stochastic calculus with local times[END_REF].

• Q π the set of càdlàg functions x ∈ D such that the limit q of (q n ) exists in (D, d).

Then:

(i) Q π 0 = Q π 1 = Q π 2 = Q π . (ii) If x ∈ Q π 0 , then [x] π = s(x) = q(x) = q(x).
(iii) x has finite quadratic variation along π if and only if

q n (t) := πn ti≤t (x(t i+1 ) -x(t i )) 2
converges in (D, d).

(iv) If (q n ) defined by [START_REF] Coquet | On non-continuous dirichlet processes[END_REF] converges in (D, d), the limit is equal to [x].

Proof. These results are a consequence of Prop. 2.1, 2.2, 2.3 and 2.4.

We see that the two defining properties of [x] in Def. 2.1 are consequences per Thm. 2.1. The following corollary treats the special case of continuous functions.

Corollary 2.1. Let x ∈ Q π , s n defined as in (6), (q n ) defined by [START_REF] Coquet | On non-continuous dirichlet processes[END_REF]. (ii): Let T > 0, • (t) the supremum norm on D([0, T ]) and observe that

i q n → [x] uniformly on compacts in [0, ∞) if and only if x is continuous. ii If q n → [x] uniformly on compacts in [0, ∞), then s n → [x]
s n -[x] (t) ≤ q n -[x] (t) + s n -q n (t).
Since (i) implies x ∈ C, (ii) now follows from uniform continuity of x and (11).

Remark 2.1. The converse of (ii) is not true in general.

Remark 2.2. We note that some references have used the pointwise limit of the sequence

p n (t) := πn ti+1≤t (x(t i+1 ) -x(t i )) 2
together with the Lebesgue decomposition (5), to define [x]. To see why this is not the correct choice, take [START_REF] Vovk | Itô calculus without probability in idealized financial markets[END_REF]Sec. 6] proposes a different notion of pathwise quadratic variation along a sequence of partitions, which is shown to coincide with Föllmer's definition under the additional assumption that the sequence of partitions π is refining and exhausts all discontinuity points of the path [START_REF] Vovk | Itô calculus without probability in idealized financial markets[END_REF]Prop. 6.3 & 6.4].

t 0 / ∈ π, put x(t) := 1I [t0,∞) (t) then obviously [x](t 0 ) = lim s n (t 0 ) = lim q n (t 0 ) = 1 but lim p n (t 0 ) = 0. Remark 2.3. Vovk
This requirement of exhausting all jumps can always be satisfied for a given càdlàg path by adding all discontinuity points to the sequence of partitions. However if one is interested in applying this definition to a process, say a semimartingale, then in general there may exist no sequence of partitions satisfying this condition. And, if this requirement of exhausting all discontinuity points is removed, then Vovk's definition differs from Föllmer's (and therefore, fails to satisfy the Ito formula (2) in general).

By contrast, our definition does not require such a condition and easily carries over to stochastic processes without requiring the use of random partitions (see Theorem 4.1). It is known that x, y ∈ Q π does not imply x + y ∈ Q π [7, 20] so one cannot for instance define a quadratic covariation [x, y] π of two such functions in the obvious way. This prevents a simple componentwise definition of the finite quadratic variation property for vector-valued functions. Therefore, the notion sense of Def. 2.1.

Quadratic variation for multidimensional functions

We now give a criterion for x ∈ D to have finite quadratic variation without any reference to the Lebesgue decomposition (5) on the limit measure µ:

Property 1. x ∈ Q π 0 if and only if (q n ) defined by (10) is a Cauchy sequence in (D, δ).
Proof. This is a consequence of Thm. 2.1 and that (D, δ) is complete.

One of the main advantages of having convergence in the J 1 topology is that it ensures convergence of jumps in a regulated manner. It comes in handy when accessing the limit of q n (t n ) as n → ∞.

Property 2. Let x ∈ Q π 0 , for each t ≥ 0, we define t n := max{t i < t|t i ∈ π n }, then t n -→ t; t n ≤ t n =⇒ q n (t n -)-→[x](t-), t n -→ t; t n < t n =⇒ q n (t n ) -→[x](t-), t n -→ t; t n ≥ t n =⇒ q n (t n ) -→[x](t), t n -→ t; t n > t n =⇒ q n (t n -)-→[x](t).
In particular, the sequence (t n ) is asymptotically unique in the sense that any other sequence (t n ) meeting the above properties coincides with (t n ) for n sufficiently large.

Proof. This is a consequence of Thm. 2.1 and [16, VI.2.1].

Given a càdlàg process X (i.e. a (D, F)-measurable random variable), a natural quantity to consider is P(X ∈ Q π 0 ). This only makes sense however if Q π 0 is F-measurable. This 'natural' property, not easy to show using the original definition (Def. 2.1), becomes simple thanks to Theorem. 2.1:

Property 3 (Measurability of Q π 0 ). Q π 0 is F-measurable. Proof. By Thm. 2.1, Q π 0 = Q π and by definition, Q π is the J 1 convergence set of x -→ πn ti≤• (x(t i+1 ) -x(t i )) 2 , n ≥ 1 on D. Since D is completely metrisable, the claim follows from [12, V.3].
Föllmer introduced in [START_REF] Föllmer | Calcul d'Itô sans probabilités[END_REF][START_REF] Föllmer | Dirichlet processes[END_REF] the class of processes with finite quadratic variation ('processus à variation quadratique'), defined as càdlàg processes such that the sequence S n (t) := πn ti≤t (X(t i+1 ) -X(t i )) 2 converges in probability for every t to an increasing process [X] with Lebesgue decomposition:

[X](t) = [X] c (t) + s≤t ∆X(s) t ∆X(s).

The pathwise Itô formula (2) can be applied to this class of processes, which is strictly larger than the class of semimartingales [START_REF] Coquet | On non-continuous dirichlet processes[END_REF].

Theorem 4.1. Let X be an R d -valued càdlàg process, define a sequence of (D d×d , δ)-valued random variables (q n ) by q n (t) := πn ti≤t (X(t i+1 ) -X(t i ))(X(t i+1 ) -X(t i )) T then the following properties are equivalent:

(i) X is a process with finite quadratic variation.

(ii) (q n ) converges in probability.

(iii) (q n ) is a Cauchy sequence in probability.

In addition, iv If (q n ) converges in probability, the limit is [X].

v The convergence of (q n ) to [X] is UCP if and only if X is a continuous process of quadratic variation [X].

vi (q n ) converges (resp. is a Cauchy sequence) in probability if and only if each component sequence of (q n ) converges (resp. is a Cauchy sequence) in probability.

Proof. We first remark that (D d×d , δ) is a complete separable metric space [START_REF] Jacod | Limit theorems for stochastic processes[END_REF], hence by [13,

  R) the space of cadlag functions and C := C([0, ∞), R) the subspace of real-valued continuous functions. We equip D with a metric d which induces the Skorokhod J 1 topology [16, Ch. VI]. Denote D + 0 ⊂ D to be the subset of non-negative increasing right-continuous functions null at 0. Definition 2.1 (Föllmer1981). x ∈ D has (finite) quadratic variation [x] π along π if the sequence of measures

Denote

  D m := D([0, ∞), R m ) and D m×m := D([0, ∞), R m×m ) to be the Skorokhod spaces[START_REF] Billingsley | Convergence of probability measures[END_REF][START_REF] Skorokhod | Limit theorems for stochastic processes[END_REF][START_REF] Jakubowski | On the Skorokhod topology[END_REF], each of which equipped with a metric d which induces the corresponding Skorokhod J 1 topology [16, Ch. VI]. C m := C([0, ∞), R m ) the subspace of continuous functions in D m . We recall Theorem 2.1 from the one dimensional case n = 1 that (Def. 2.1) and (Thm. 2.1.iii) are equivalent.

  there exists [16, VI.2.1(a)] a sequence t n → t such that ∆q n (t n ) → ∆q(t) > 0. Using the fact that x is càdlàg , t n → t, (13) and [16, VI.2.1(b)], we deduce that (t n ) must coincide with (t

	(n)

  by uniform convergence. Put t n := max{t i < t|t i ∈ π n }, since q n → [x] in the Skorokhod topology, we also have ∆q by[START_REF] Dudley | Real analysis & probability[END_REF] and [16, VI.2.1(b)]. If ∆[x](t) > 0, then [16, VI.2.1(b)] implies t n must coincide with t for all n large enough, but t n < t for all n, hence ∆[x](t) = 0 which implies ∆x(t) = 0 by Prop. 2.4 and (5). Since t is arbitrary, we conclude that x ∈ C.

	uniformly on
	compacts.
	Proof. (i): The if part follows from Prop. 2.4, (5) and [16, VI.1.17(b)]. The only
	if part: Let t ≥ 0, it is well known that
	∆q n (t) → ∆[x](t)

n (t n ) → ∆[x](t)

  Lemma 9.2.4], the Cauchy property is equivalent to convergence in probability. By [13, Thm. 9.2.1], we can pass to subsequences and apply Prop. 3.2, Thm. 3.1 & Cor. 3.1 pathwise to X, the claims follow.

Acknowledgements We thank Rafal Lochowski, Pietro Siorpaes and the referee for useful comments.

Supported by EPSRC Doctoral Training grant 1824430.

of quadratic variation in the multidimensional setting was originally defined in [START_REF] Föllmer | Calcul d'Itô sans probabilités[END_REF] as follows: Definition 3.1 (Föllmer1981). We say that x := (x 1 , . . . , x m ) T ∈ D m has finite quadratic variation along π if all x i , x i + x j (1 ≤ i, j ≤ m) have finite quadratic variation.

The quadratic (co)variation [x i , x j ] is then defined as

which admits the following Lebesgue decomposition:

The function [x] π := ([x i , x j ]) 1≤i,j≤m , which takes values in the cone of symmetricsemidefinite positive matrices, is called the quadratic (co)variation of x.

Note Def. 3.1 requires first introducing the case m = 1. The following definition, by contrast, avoids this and directly defines the concept of multidimensional quadratic variation in any dimension:

We shall now prove the equivalence of these definitions. Define, for u, v, w ∈ D

and q (w) n := q (w,w) n . Note that the Skorokhod topology on (D m , d) is strictly finer than the product topology on (D, d) m [16, VI.1.21] and that (D, d) is not a topological vector space [START_REF] Jacod | Limit theorems for stochastic processes[END_REF]VI.1.22]. The following lemma is essential:

Note that the sequence t n is chosen from the partition points of π n , independently of u, v ∈ D.

Put q := lim n q (u,v) n

. If ∆q(t) = 0, [16, VI.2.1(b.5)] implies that ∆q

Using the fact that u, v are càdlàg , t n → t and ( 16), we deduce that (t n ) must coincide with (t (n) i ) for n sufficiently large, else we will contradict ∆q(t) > 0. Put t n := t

n + q (y) n + 2q Proof. This is a consequence of ( 14), [START_REF] Jakubowski | On the Skorokhod topology[END_REF]

Some applications

We now show that our approach yields simple proofs for some properties of pathwise quadratic variation, which turn out to be useful in the study of pathwise approaches to Ito calculus. Denote D := D([0, ∞), R) and D d×d := D([0, ∞), R d×d ) to be the Skorokhod spaces, each of which equipped with a complete metric δ which induces the corresponding Skorokhod (a.k.a J 1 ) topology. Denote F to be the J 1 Borel sigma algebra of D (a.k.a the canonical sigma algebra generated by coordinates). Recall that Q π 0 is the set of paths with finite quadratic variation along π in the