Alberto Jovane
email: alberto.jovane@inria.fr

Pierre Raimbaud

Katja Zibrek

Claudio Pacchierotti

Marc Christie

Ludovic Hoyet

Anne-Hélène Olivier

Julien Pettré
email: julien.pettre@inria.fr

Warping Character Animations using Visual Motion Features

Keywords: Computers and Graphics, Formatting, Guidelines

Despite the wide availability of large motion databases, and recent advances in exploiting them to create tailored contents, artists still spend tremendous efforts in editing and adapting existing character animations to new contexts. This motion editing problem is a well-addressed research topic. Yet only a few approaches have considered the influence of the camera angle, and the resulting visual features it yields, as a mean to control and warp a character animation. This paper proposes the design of viewpointdependent motion warping units that perform subtle updates on animations through the specification of visual motion features such as visibility, or spatial extent. We express the problem as a specific case of visual servoing, where the warping of a given character motion is regulated by a number of visual features to enforce. Results demonstrate the relevance of the approach for different motion editing tasks and its potential to empower virtual characters with attention-aware communication abilities.

Introduction

Numerous interactive applications, e.g. video games or Virtual Reality immersive simulations, rely on motion capture techniques to animate human characters thanks to the excellent trade-off they provide between computational budget and animation quality [START_REF] Bodenheimer | The process of motion capture: Dealing with the data[END_REF]. To extend the usability of such motion captured animations beyond what was initially recorded, a long-standing objective in the field of computer animation is to design efficient techniques to edit motion clips, originally taking inspiration from signal processing editing approaches [START_REF] Bruderlin | Motion Signal Processing[END_REF]. Motion editing techniques act on various features of animation trajectories, e.g., blending similar motions to generate new ones [START_REF] Rose | Verbs and adverbs: Multidimensional motion interpolation[END_REF], concatenating motion clips [START_REF] Ashraf | Dynamic Time Warp Based Framespace Interpolation for Motion Editing[END_REF] or accounting for interactions between characters during concatenation [START_REF] Hubert | Interaction patches for multi-character animation[END_REF] . Motion style transfer has also been addressed [START_REF] Brand | Style Machines[END_REF], as well as adjusting kinematic constraints from geometric relationships in changing environments [START_REF] Arikan | Synthesizing constrained motions from examples[END_REF][START_REF] Andrew W Feng | An examplebased motion synthesis technique for locomotion and object manipulation[END_REF].

This topic is a long-standing one -we voluntarily picked some of the oldest and seminal papers -and a very large body of literature can be found on each of the various facets of motion editing [START_REF] Guo | Adaptive motion synthesis for virtual characters: a survey[END_REF][START_REF] Van Welbergen | Real Time Animation of Virtual Humans: A Trade-off Between Naturalness and Control[END_REF]. It is however noticeable that existing techniques rarely consider that human movement is often performed in relation to an observer. This is particularly true when motion is considered as a nonverbal communication channel [START_REF] Robert | Non-verbal communication[END_REF]. Indeed, in such communication tasks, humans control their movements by fundamentally taking into account how it can be visually perceived by an observer. For instance, while waving at someone -a typical voluntary nonverbal communication gesture to attract attention -one makes sure his/her hand is visible to this person, e.g. adjusting body orientation, waving amplitude and speed to make the motion salient enough, as well as moving his/her face and eyes to enable gaze contact and ensure that attention is successfully attracted. This example clearly highlights the links between the kinematics of a motion and the visual features perceived by an observer. This notion of visual motion features are defined in relation with an observer's point of view and Field of Vision (FoV). It covers features such as visibility and centrality of limbs, coverage of limbs and of their motion, generated optical flow, motion amplitude in the view plane, etc.

As visual features of motions can be of prime importance, the objective of this paper is to devise a motion editing technique that enables controlling visual motion features of a given character relatively to an observer. In other words, given a character motion m, the corresponding set of visual features s as perceived by an observer, and the desired visual motion feature values s * , we search for the warping operation w to synthesise the warped motion m w with the desired s * values. We view the relevance of our visual feature approach when animating characters in social interactions, or in immersive Virtual Reality applications where characters are meant to adjust themselves to the users' state in the virtual scene.

Motions can be synthesised from the analysis of visual properties. We see examples in animation, e.g. where images of the environment are exploited to generate character's movements [START_REF] Cao | Long-term human motion prediction with scene context[END_REF], as well as in robotics, e.g. where camera feedback can guide the kinematics's properties of the motion of robotics's limbs. Indeed, our approach is inspired by robotics visual servoing [START_REF] Franc | Visual servo control. I. Basic approaches[END_REF], a technique to control a robot's motion based on visual sensor feedback, e.g. a camera. In analogy with visual servoing, we want to control a character motion with respect to an observer position. Following this analogy, our motion warping goal is formulated as a visual task, i.e. to reach s * . Unlike previous approaches where motion editing goals generally rely on the kinematic properties of the motion, we directly set the visual properties we want for the edited motion. However, a major difference with visual servoing solutions is that we want to perform motion warping, not full motion control.

Our main contribution is a new visually driven motion editing framework that enables to manipulate the visual features of an existing motion through an observer's viewpoint. We believe that such visual editing techniques will help enhanced awareness and reactivity in interactive virtual agents, thus we present a user study in Virtual Reality where we demonstrate the advantages of our approach compared to a standard technique. Additionally, we will present a set of use cases to illustrate how our technique can adapt existing content (motion capture) to new contexts, and may empower virtual agents with the ability to capture attention.

State of the art

In the large body of literature about motion capture editing, our work falls in this category of solutions that can synthesise, control or adjust data-driven character animations on the fly. Different editing aspects were explored in the field such as concatenating new motion sequences from differently ordered ones, e.g. motion graphs, motion fields or motion matching techniques [START_REF] Arikan | Interactive motion generation from examples[END_REF][START_REF] Lee | Interactive Control of Avatars Animated with Human Motion Data[END_REF][START_REF] Arikan | Synthesizing constrained motions from examples[END_REF][START_REF] Kovar | Motion graphs[END_REF][START_REF] Lee | Motion Fields for Interactive Character Locomotion[END_REF][START_REF] Holden | Learned motion matching[END_REF], interpolating example motions of the same kind to control some kinematic features [START_REF] Rose | Verbs and adverbs: Multidimensional motion interpolation[END_REF][START_REF] Kovar | Flexible Automatic Motion Blending with Registration Curves[END_REF][START_REF] Park | On-line motion blending for real-time locomotion generation[END_REF], performing temporal adjustments of motion (e.g. time-warping to align motions before transiting or blending) [START_REF] Ashraf | Dynamic Time Warp Based Framespace Interpolation for Motion Editing[END_REF][START_REF] Glardon | PCA-based walking engine using motion capture data[END_REF][START_REF] Adistambha | Motion classification using Dynamic Time Warping[END_REF][START_REF] Zhou | Generalized time warping for multi-modal alignment of human motion[END_REF][START_REF] Switonski | Dynamic time warping in classification and selection of motion capture data[END_REF], retargeting motions to new characters of various morphology [START_REF] Gleicher | Retargetting Motion to New Characters[END_REF][START_REF] Choi | Online motion retargetting[END_REF][START_REF] Villegas | Neural Kinematic Networks for Unsupervised Motion Retargetting[END_REF], satisfying kinematic constraints when played in new environments (e.g. contact displacement) [START_REF] Witkin | Motion Warping[END_REF][START_REF] Lee | Interactive Control of Avatars Animated with Human Motion Data[END_REF][START_REF] Edmond | Spatial Relationship Preserving Character Motion Adaptation[END_REF][START_REF] Ali Al-Asqhar | Relationship Descriptors for Interactive Motion Adaptation[END_REF], performing stylistic variations [START_REF] Brand | Style Machines[END_REF], maintaining physical interactions between characters while editing [START_REF] Hubert | Interaction patches for multi-character animation[END_REF][START_REF] Kim | Synchronized multi-character motion editing[END_REF][START_REF] Men | GAN-based reactive motion synthesis with class-aware discriminators for human-human interaction[END_REF].

Even though all these solutions would play on character motion properties and on their visual properties, we are not aware of existing work trying to directly provide control over visual motion features as we aim to do. The closest contributions mainly propose through-the-lens techniques, with which trajectories in the image space, such as hand-drawn sketches [START_REF] Guay | Space-time sketching of character animation[END_REF] or spatial-temporal constraints [START_REF] Ciccone | Tangentspace optimization for interactive animation control[END_REF], enable the control of kinematic chains. A similar sketch-based approach was proposed to warp existing animations using motion trails [START_REF] Choi | SketchiMo: sketch-based motion editing for articulated characters[END_REF]. Solutions rely on constrained optimisation, minimizing a projected difference with the input sketches, while respecting the original animation and disambiguating depth. The perceptual effect of edited animations was also explored to assess the loss in animation quality [START_REF] Arikan | Compression of Motion Capture Databases[END_REF]. Some works in virtual cinematography [START_REF] Jiang | Camera keyframing with style and control[END_REF] study the complementary problem of optimizing the point of vision of a virtual scene to respect visual constraints over the motion of the virtual actors.

Establishing relations between motion variables and motion visual features can however be an interesting solution in the frame of nonverbal communication situations where vision is a major perception channel. Nonverbal communication refers to all modes of communication that do not use speech. This includes facial expression, para-linguistic, proxemics and touch, as well as body gestures and postures [START_REF] Harrigan | New handbook of methods in nonverbal behavior research[END_REF]. During social interactions, a continuous exchange of such signals is possible, mainly because people are able to see each other [START_REF] Cañigueral | The role of eye gaze during natural social interactions in typical and autistic people[END_REF]. This means that the interpretation of social signals involving a mutual interaction between the observer and the other(s) person(s) should be expressed in the reference frame of the observer, namely his/her FoV. This visual perception for social interaction then enables to infer on a person's intention [START_REF] Blakemore | From the perception of action to the understanding of intention[END_REF][START_REF] Knoblich | Evolving intentions for social interaction: from entrainment to joint action[END_REF], personality [START_REF] Neff | Evaluating the Effect of Gesture and Language on Personality Perception in Conversational Agents[END_REF], as well as emotions [START_REF] De Gelder | The perception of emotion in body expressions[END_REF][START_REF] Roether | Critical features for the perception of emotion from gait[END_REF].

In this context, authors have proposed several approaches to provide animated characters with personality or emotion, such as Durupinar et al. . [START_REF] Durupinar | Perform: Perceptual approach for adding ocean personality to human motion using laban movement analysis[END_REF] who used Laban Motion Analysis, or Randhavane et al. [START_REF] Randhavane | EVA: Generating Emotional Behavior of Virtual Agents Using Expressive Features of Gait and Gaze[END_REF] who played on the kinematic parameters of gait and gaze. Elaborate solutions for style transfer were also proposed [START_REF] Holden | Fast neural style transfer for motion data[END_REF], where style can encompass aspects of personality or emotional state. But, it is important to note that these methods are not designed to express these features in relation to the FoV of the observer. A visual-optimization approach is however presented by Huang and Kallmann [START_REF] Huang | Planning motions and placements for virtual demonstrators[END_REF], where character motion, posture and location are synthesised in relation to an observer. Despite presenting a comprehensive framework and sharing similar motivation than our approach, their work is however limited to a set of predefined actions and only considers visibility constraints for character placement. We believe that our approach extends this character-observer relation to general motions and directly links posture adaptation with vision-based features.

To conclude, to our knowledge, there is no previous attempt to link low-level motion control variables to perceived visual features with the intent to control them, which is the objective set for our work.

Method

In this Section, we describe the mathematical formulation of our approach. Section 3.1 introduces the main concepts, Section 3.2 presents the different visual motion features we consider when warping the animations, Section 3.3 details the design of our warping operators, and Section 3.4 explains the regulation of the visual motion features using our warping operators.

Overview

The aim of our approach is to provide designers or interactive applications with a warping controller that adjusts a character animation using view-dependent visual motion features. The input of our system is (i) an animated character, (ii) a set of designer-specified visual motion features to fulfill (e.g. visual coverage, vertical extension), and (iii) a camera angle or trajectory that views the character in its environment. Our system adjusts the character animation on a per-frame basis by satisfying user-defined visual motion features in an inverse design approach, from features to parameters.

Technically, we propose to express the problem as a specific instance of the eye-to-hand visual servoing principle [START_REF] Dombre | Robot manipulators: modeling, performance analysis and control[END_REF] in which a camera, fixed or animated in the world, observes the motion of one or multiple kinematic chains. Rather than driving the velocity of kinematic chains from on-screen velocities -as implemented in traditional visual servoing tasks [START_REF] Espiau | A new approach to visual servoing in robotics[END_REF] or with through-the-lens control [START_REF] Gleicher | Through-the-Lens Camera Control[END_REF], our objective is to develop a control law that updates velocities in the kinematic chain by regulating the difference between globally measured visual motion features and expected ones in the 2D camera space. This requires the design of (i) estimators that are able to measure the values of the expected visual motion features from the given camera, (ii) warping units that alter the parameters of the kinematic chains, and (iii) a control loop that exploits the difference between estimated and expected features to drive the warping operators. Our overall approach is described in Figure 1.

Estimators of visual motion features

The perceptual mechanisms by which humans look at, read, and understand images are well studied nowadays. Typical features such as chrominance, contrast and motion in the spectator field of view are well-known bottom-up key factors that influence audience attention. Attention is also driven by a number of top-down factors such as object semantics (faces draw strong attention), cultural background, and tasks to perform [START_REF] Kimura | Computational models of human visual attention and their implementations: A survey[END_REF].

With this in mind, we propose estimators to measure visual motion features, that are computational characteristics designed to measure how well the motion of a character is perceived from an observer's viewpoint. Moreover, these features represent here a proxy for visual attention. As such, they provide a view-dependent metric influenced by the character motion, the lighting in the scene and by potential occluders, and therefore provide a mean to control the amount of perceived motion in screen space.

For a given time frame t, we express the visual motion features as a time dependent vector s t :

s t = [s 1 (t), ..., s V (t)] T = s(m(t), q(t), Ω(t)) (1)
where s i (t) is a function of a) the character pose specified by m(t), b) the observer pose specified as a camera pose q(t), and c) the state of the environment Ω(t) that accounts for the rest of the scene, notably any lights and geometries that may affect the visibility of the virtual agent from the observer viewpoint.

The function s(m(t), q(t), Ω(t)) performs the scene and character rendering from the given camera, and computes the visual motion features.

Visual motion features:. In this work we consider the following features:

• apparent static coverage measures how much of a character's projected image is perceived in a frame (accounting for visibility and lighting);

• apparent static extension measures the horizontal occupancy (resp. vertical occupancy) as a ratio between the left-most and right-most pixels (resp. bottom-most and top-most) of the character on the screen width (resp. height), also accounting for visibility and lighting.

• apparent motion coverage measures how much of a character's motion from one frame to another is perceived in the image;

Likewise, designers are free to define and use additional visual motion features for their own application, at the condition that these can be represented as scalar values.

In practice, visual motion features are computed through hardware rendering and straightforward image analysis. At each time step, a frame is rendered from the observer's viewpoint and only the perceived pixels of the virtual agent are kept. A pixel is considered as perceived if it is not occluded, or if its luminance is under a given threshold (e.g. in a shaded or dark area). Visual motion features are estimated through pixel operations such as counting or comparing coordinates and distances in the image space.

Semantic layers on body representations. Sub-meshes of virtual characters are tagged so as to identify specific parts (face, arms, chest, legs, inside of hands). This enables us to arbitrarily activate or deactivate body parts according to the performed motion, e.g. to focus and render only the waving hand in a waving character case. In addition, rigid objects can be attached to the skeletal joints and visual motion features can be computed on them (e.g. an additional piece of clothing or holding a sheet of paper).

Fig. 1. Overview of our approach. From an input sequence of a character animation, we first estimate different visual motion features on the current pose, considering the environment, the observer's viewpoint, and a visual target (blue). Then, multiple plausible motion modifications are computed manipulating warping units (yellow), and the ones that minimise the visual error between the current state and the target are applied to the output motion. This process is repeated for the whole motion over a control loop (red).

Motion warping units

We first rely on a classical skeletal structure with joints using a tree of kinematic chains. An animation of the skeletal structure is defined as a set of keyframes along with an interpolation technique. We can thus define a function m(t) that computes the current pose of the character at time t, expressed as a vector of the degrees of freedoms of the characters joints denoted θ t :

θ t = [θ 0 , .., θ K ] T = m(t) (2)
where K represents the number of degrees of freedom of the skeletal structure. Rather than regulating visual features by controlling simultaneously the whole vector θ of joints of a character (which may create unexpected or unrealistic changes in the body poses), we propose to define specific groups of joints in the skeletal representation and define these as motion warping units. These groups are defined to provide a localised control on a character (e.g. only the spine, only the arms, only the head plus shoulders), which is a classical approach when designers need to locally warp motions without impacting the whole body (see Table 1). Furthermore, we design our motion warping units in a parameterised way that maintains the coupling between parameters of the kinematic chain by using a linear combination given a warping factor. This enables small warpings on the animations without loosing the nature of the motion.

Our motion warping unit is therefore defined as a function ω k (ω, t) that, given a scalar displacement value ω, computes a sparse pose offset vector at time t affecting only the joint angles in the warping unit. For a given warping unit, the derived pose offset vector is added to the current pose m(t) to generate the warped animation. The index k represents the k-th warping unit, and ω k (ω, t) yields a vector of size K with a zero-value for the degrees of freedom that are not offsetted. Our motion warping unit defines a pose offset vector in which each offset is a weighted linear combination of ω:

∆θ = ω k (ω, t) = [w k 0 ω, .., w k K ω] T (3)
The scalar value w k i is a weighting constant specific to a given warping unit k and degree of freedom i of the kinematic chain and can be viewed as a stiffness coefficient, traditionally used when manipulating inverse kinematic chains. The linear combination with ω ensures a coupling in the offset computation of animation parameters. The corresponding value ∆θ i represents the i-th kinematic angle offset computed by the warping operator.

Each motion warping unit therefore computes a pose offset vector ∆θ i , and all offset vectors are aggregated on the current character pose to create the warped motion (see Equation 4). The magnitude and direction of the computed offset vectors are driven by a vector [ω 1 , .., ω k ] T of warping unit parameters, the value of which is computed by the visual servoing task (see Section 3.4). The overall warped parameters m w (t) of the animation at each time t are given by:

m w (t) = m(t) + W k=0 ω k (ω, t) (4)

Driving warping units through visual motion features

Our objective is to compute the optimal set of warping unit parameters ω at each time t of the animation from a given set of desired visual motion features s t , through a control loop. We first express the relation s t = f t (ω) where f t computes the estimation of visual motion features. As a direct relationship between s t and ω exists, our goal is to solve this equation to obtain ω from s t . Due to the strong non-linearity of the relation between visual motion features and kinematic parameters, a classical approach is to study the problem in the velocity space.

Given this set of visual motion features s t that depend both on a camera viewpoint at time t and a set of warping unit parameters ω, the differential ṡt expresses how the variations in the visual features are related to the camera and the character animations.

Introduction on visual servoing. As defined, this problem is a specific case of a eye-to-hand visual servoing problem where a specified velocity in the image space of a fixed camera looking at a kinematic chain is used to drive its degrees of freedom [START_REF] Espiau | A new approach to visual servoing in robotics[END_REF]. This visual servoing relation is generally defined as:

ṡt = L s V n J n (θ) θ + δs δt
where J n (θ) is the Jacobian of the kinematic chain, V n the kinematic tensor transformation from the camera to the character, L s the interaction matrix, and δs δt describes the variations of s caused by a movement of the camera (n represents the number of degrees of freedom of the kinematic chain). This relation defines the correlation between the variation in visual motion features and the variations in degrees of freedom of the kinematic chain. We rely on this formulation to express our problem in terms of the warping unit parameters ω. ṡt = J s ω + δs δt Jacobian computation using finite differences. Each element of the Jacobian matrix J s encodes a partial derivative of visual motion features values (s) over each warping unit feature (ω):

J s = δs q δω k q,k , q ∈ [0..V], k ∈ [0..W] ( 5
)
where V is the total number of visual motion features and W is the total number of warping units. A forward evaluation enables us to compute a variation in visual motion features ∆s from a variation in degrees of freedom ∆θ (for small enough variations):

∆s = J s ∆θ (6)
To solve the problem, we therefore reverse Equation 6 by approximating J -1 using a damped least square method, and we obtain the variation of degree of freedom, that we expressed as variation on ω (linear combination of θ, see Section 3.3).

∆ω = J -1 ∆s (7)
The input vector ∆s is classically computed as the difference between the expected features and the measured features e = s *s t and capped with a maximum threshold:

∆s = e i f ∥e∥ ≤ DS max DS max e ∥e∥ otherwise (8)
In practice the computation of the Jacobian J s at any time t requires to evaluate each visual motion feature for each 2 * W variation of warping unit parameters. This is performed using finite differences, where variations in the agent motion need to be rendered to assess visual features.

We finally build the Jacobian matrix J s as:

J s =         s ω + k q -s ω - k q 2ω k         q,k,t (9)
We then estimate the inverted Jacobian, and use it to extract the warping direction ∆ω (see Equation 7). An additional clamping is applied to the obtained vector to smooth the final motion modification. Finally, the new warped motion is computed using Equation 4. Implementation details and the setting of the expected feature s * , for different case studies, are detailed in the following section followed by a user evaluation performed using Virtual Reality.

Results

We demonstrate our method over three case studies, with one virtual agent acting in front of one observer. We focus on upperbody nonverbal communication mainly using head, torso, arm and hand body parts, although the proposed method is suitable to control any body limb. Of environment conditions and related visual targets, we explore: i) the influence of changes in the observer's viewpoint, ii) the influence of occlusion or lighting conditions, and iii) the potential exaggeration of extraverted traits of a motion. Our accompanied video provides additional examples. Results and performances are discussed in this section, the method is more generally discussed in Section 6.

Implementation

We implemented our technique using Unreal Engine. The approach runs at interactive frame-rates (>30fps on a computer equipped with an Intel Core i7-9850H CPU @ 2.60GHz, 32GB of RAM, Nvidia Quadro T2000) and can be used in interactive and non-interactive contexts. Estimators (see Section 3.2) were implemented using shaders, while the visual motion feature vector was computed through multiple rendering passes. The warping operators were built above Unreal control rigs (see Section 3.3) which provide a direct access to the degrees of freedom of the skeletal structure of animated characters. Our virtual agent is based on a Meta-humans model. The baseline motions (unwarped) were either recorded using a motion capture system (Xsens suit) or taken from a public database (Mixamo). During the execution, the control loop (see Section 3.4) generates (2 * W) + 1 copies of the virtual agent, at each frame, from the observer point of view: 1 copy is used as a reference (unwarped motion) for visual motion features evaluation and for the target definition, whilst the 2W others are used to compute J s by rendering warped motions, for each of the W warping units, in both warping directions.

To encode the different adapting behaviours, the designer first needs to decide the relevant body parts on which visual motion features are computed, then select the subset of these features to control. For each visual motion feature the designer can specify the magnitude and direction of its change over time, relative to the current value or to an absolute visual target. The magnitude affects mostly the responsiveness, and the direction defines the sign of the adaptation. Both values define the final visual target s * (see Section 3.4). Consequently the designer needs to choose the set of active warping units, i.e. parts of the body which animations will be warped (see Table 1). We will describe and justify these choices in each example. 2), we study the influence of viewpoint changes for a character with and without an object. The desired visual feature consisted in increasing the visual coverage. In the middle (cases 2.1 and 2.2), we study the influence of solid and sparse occluders. Visual coverage was also used as the regulating visual motion feature. On the right (case 3) we explore how extraverted effects can be created by applying upper body warping units, and regulating their apparent vertical and horizontal extension. In the picture, the amplitude of the gesture of the warped character is wider with respect to the original non-warped one, making it feel more extraverted.

Joint Movement

Case studies

Case 1 -viewpoint changes. The purpose of this first study is to experiment the influence of viewpoint change on the animations to edit. Scenario 1.1: In this scenario, the virtual agent aims at catching the attention of the observer while performing a two-hand waving motion. We recorded the baseline motion assuming that the agent was facing the observer. The objective is to modify this motion for a different camera angle, in a way that the agent better captures the observer's attention according to his/her position. We performed the visual evaluation with the face and the waving hands as body parts linked to the visual target. We selected visual coverage as the visual motion feature and specified a value to maximise frontal visual appearance. Also, when positioning the observer at different distances from the camera, we used the control of the apparent horizontal extension to adapt the waving amplitude. The warping units related to the agent's spine, neck and arms were selected since they affect the waving motion of both hands. Scenario 1.2: Here, the virtual agent shows to the observer an object held in its right hand. Again, our baseline animation was recorded with the observer facing the agent. The challenge is to adapt arm and hand motions to the observer's point of view in such a way that the object shown appears in the center of the observer's FoV. Regarding the relevant limbs related to the visual target evaluation, we selected the head and the additional external object (a tablet) held in the hand. Similarly to the previous scenario, we aimed for an increment of the visual coverage of the face and the tablet screen to maximise their visual appearance. The selected warping units were the ones related to the spine, neck and right arm of the agent since the motion was performed with this arm only.

Informal analysis: Scenarios 1.1 and 1.2 are illustrated in Figure 2 left. They were tested with different viewpoint parameters, namely, position, orientation, static/dynamic motions (see also our companion video). Results show that our approach successfully adjusts motions to changing viewpoints. Indeed, the first scenario demonstrates how the waving amplitude adjusts according to the observer's distance. In the second scenario we show how a unitary motion could be rapidly warped, to fit the duration of the action, with the same target of maintaining the visibility of a relevant object shown in the observer's FoV. We also show that our approach allows for the control of multiple limbs by generating subtle variations on a motion without affecting its original purpose. One could argue that similar results, especially Scenario 1.2, could be replicated using an inverse kinematics (IK) approach. This is only partly true, as our approach based on visual motion features also integrates environmental conditions such as lighting or scene layout (e.g. bring an object in front of someone, in light) with the exact same setting. Case 2 -occlusion/visibility. In this case study we aim at exploring through two scenarios the warping of agent limb motions to ensure proper visibility from the observer's viewpoint.

Scenario 2.1: In the first scenario, we recorded a one-hand waving motion, fully visible to a facing observer. The objective is to adapt this motion to improve its visibility by accounting for environment effects -occluding or partially occluding the animation. We considered as relevant limbs the ones that usually help capturing the observer's attention at a distance i.e. the waving hand, the face and the upper torso. For the same reason, the visual coverage was selected as the active visual motion feature. In this case, an increment of this feature for the hand, face and upper torso would improve the perceived visibility of the motion. The selected warping units influenced the spine, the right arm and hand.

Scenario 2.2: Here, our agent tries to hide from the observer; to achieve this, we captured a crouching motion as if someone was hiding behind an object, and used it for the original animation. The objective here was to adapt the motion to make the agent less visible from the observer's viewpoint by hiding behind occluders. A decrement on visual coverage (the selected visual motion feature) was selected as target. Finally, we also tried to simultaneously increment the perceived visual coverage of a specific limb while hiding the rest, e.g. maintaining eyes visible. We selected as warping units those related to the spine and the neck, and as relevant limbs all the upper body ones.

Informal analysis: Scenarios 2.1 and 2.2 are illustrated in Figure 2 center. These scenarios test different visibility parameters: static/dynamic occluders, static/dynamic lighting, different kinds of obstacles (see also our companion video). Results show that our method successfully adapts motions to visibility conditions and targets. Indeed, scenario 2.1 demonstrates how a waving motion initially recorded in a clear view situation facing a frontal observer can be adapted in other visibility conditions, by bending and adjusting the configuration of the arm towards a space where it was more visible. Scenario 2.2 shows how a crouching motion can be adapted to increase its hiding purpose, and we also show that our method enables combining this purpose with additional minor behaviours such as maintaining the top of the head visible. Such results demonstrate the advantage of our visual approach for these kinds of situations, by enhancing and adapting motions in different visibility conditions.

Case 3 -expressivity. Here, our aim is to experiment how visual features could be exploited to control the expressivity of a motion with our approach. For the current example, we aim at influencing extraversion, one of the Big Five traits of personality [START_REF] Lewis R Goldberg | An alternative" description of personality": the big-five factor structure[END_REF]. This trait describes someone who typically captures the attention of an observer, is enthusiastic, energetic and sociable. In this scenario, the virtual agent is performing communication gestures as if it was talking. For the original animation we motion captured an actor conversing with the experimenter in a neutral way. The objective here is to adapt the arm and the hand motions to tune the trait extraversion of the agent, taking into account the observer's viewpoint. We selected warping units related to the head, elbow, shoulders, arms and hands, and the selected limbs were arms and hands. Visual coverage, apparent vertical and horizontal extensions were selected for the visual motion features. The visual target was then to increase/decrease both extensions to make the agent appear more/less extraverted. Indeed, Neff et al. [START_REF] Neff | Evaluating the Effect of Gesture and Language on Personality Perception in Conversational Agents[END_REF] described several modifications to the character's gestures with a perceptual effect of creating an extraverted personality -by increasing spatial scale of the strokes, elbow rotations outwards (arm swivel) and increasing the shoulder raise. On the opposite, by decreasing the same characteristics, we define an introvert personality, additionally, for this case, we have controlled Visual coverage of the head to control the head posture.

Informal analysis: Scenario 3 is illustrated in Figure 2 right (see also our companion video). In this case study, we show that our method can modify gestures in similar ways than previous studies [START_REF] Neff | Evaluating the Effect of Gesture and Language on Personality Perception in Conversational Agents[END_REF], which should also modify the perceived extraversion of the character.

Performances

To objectively evaluate the computational load of our approach, we also present some information about the computational resources required for the application of our method.

The core computation runs on a dedicated thread at a frequency set between 3 Hz and 5 Hz, according to the selected parameters (e.g. warping units and visual features number). This thread is provided with the rendered scene and variations and outputs the warping parameters for the virtual character. At each frame, the separated rendering thread updates the posture of the character following the warping result.

Figure 3 shows the effects of the number of selected warping units (horizontal axis) and apparent visual features (blue and orange curves) on computation time. Values were captured on a machine equipped with Intel Core i7-9850H CPU @ 2.60GHz, 32GB of RAM, Nvidia Quadro T2000. Warping units size has a stronger impact on performance for the computation of variations (order of magnitude 10 2 ms in Figure 3 left), due to the need of generating and evaluating each variation of each warping unit. Nevertheless, the computation time is impacted linearly with the size of warping units. However, this heavy computation also further justifies the definition of warping units as a summary of the structure's degrees of freedoms, instead of computing variations on individual joints separately. On the other hand, the computation of the visual feature's level of the character, and the consequent target's definition (Figure 3 center), depends only on the number of the active apparent visual features. Finally, we show that the time to invert the Jacobian depends linearly on its dimensions V:W -number of visual features per number of warping units -and has minor impact on computation time (order of magnitude 10 -1 ms on Figure 3 right). In the presented implementation, the maximum possible size for the Jacobian matrix is 4:22, but none of the presented case required more then half of this dimensions.

Evaluation

Since our main objective is to design reactive virtual humans who are able to interact and initiate an interaction with users, we designed an experiment in VR, where a virtual agent attempts to catch the user's attention by waving at him/her (similarly to case 1 and 2 from Section 4.2). In such a scenario, we aim at evaluating whether participants are able to detect if a waving motion is directed towards them or towards another agent. We also compare our technique to a standard approach based on orientating the root of the character. We hypothesize that our technique will outperform the simpler approach, showing higher a success rate to detect the target of the waving motion, as well as maintaining or increasing the naturalness of the interaction.

Procedure

Sixteen unpaid participants, volunteered to the experiment (3F, 13M; age: avg=25 ± 3, min=20, max=30). They were all naive to the purpose of the experiment, had normal or correctedto-normal vision, and gave written and informed consent. The study conformed to the declaration of Helsinki, and was approved by the local ethical committee (*LEFT BLIND FOR REVIEW*). Participants were first asked to read and fill up a consent form. They were equipped with a HTC-vive's headset, headphones and two controllers. The experiment was performed in a standing position and participants were able to take a break between each trial if needed. The evaluation takes place in a virtual office, as presented in Figure 4. The participant is spawned in a line of virtual customers, who are waiting to be called by one of the virtual office clerks, using a waving motion of the hand. In total there are three clerks, each one sitting behind a desk. In between the customers and the clerks there are additional agents and static objects that occlude the view of some of the clerks.

The participant stands in one of the three central positions of the customers group, and is exposed to one of the following Fig. 4. The disposition of the agents in the virtual scene: starting from the bottom (green circles) the position of five customers, including the three (filled green circles) where the participant could be spawned. In the middle (highlighted in white) is the main area of visual occlusion, composed of objects and virtual agents. On the top: the three desks, each with a virtual employee (blue squares). Yellow, orange and red lines denote the degree of partial occlusion in the combined view direction between customers and clerks (yellow for low degree of occlusion, orange for medium, and red for high).

interactive situations: one of the employee waves in 3 possible directions: the participant (i), one of the other virtual customers to the left (ii) or to the right (iii) of the participant. The motion of the employee is animated using two techniques: a straightforward orientation of the whole body of the agent towards the target (SF adaptation), and our technique (visualWarping) parameterized with an increment in the apparent visual coverage of the head and the waving hand, using all the warping units affecting its two limbs. Additionally, for both techniques, we implemented the same gaze controller to orient the eyes of the agent towards the selected customer. In total, participants performed 2 training trials and 54 experimental trials presented in a randomized order (2 techniques × 3 directions × 9 repetitions). After each trial, participants were asked to: (i) answer the following question "According to you, to whom was the interaction directed towards?" with the possibility of selecting one of the customer's position, including their own, and (ii) to rate the level of realism of the interaction on a scale from 1 (low) and 9 (high).

At the end of the experiment, we also collected answers using a post-experiment questionnaire composed of questions related to demographics (gender, age and familiarity with VR), the strategy used ("What strategies did you use to identify to whom the interaction in the VR experience was directed to?") as well as self-reported free comments.

Analysis

For each participant and each condition, we computed the answer accuracy (i.e., the ratio of correct answers across repetitions), as well as the level of realism (the average level across repetitions). To evaluate the effect of the technique while considering the effect of the interaction direction of the waving motion, we conducted a two-way repeated measures Aligned Rank Transform (ART) Anova. When relevant, we performed Tukey post hoc pairwise comparisons. Statistics were performed using R and the level of significance was set to 0.05. Results are presented using mean±SD.

Results

Results showed a significant main effect of the technique on accuracy answers (F(1,15)=7.09, p=0.017, η 2 p =0.32), with better accuracy with visualWarping (0.67±0.2) than with SF adaptation technique (0.59±0.25), as illustrated in Figure 5, left. There was also a main effect of interaction direction (F(2,30)=18.92, p<0.001, η 2 p =0.356), where participants were less accurate when the virtual human was interacting to the right of the participants (Figure 5, right). There were no significant interaction effect (F(2,30)=2.19, p=0.12), but, as illustrated in Figure 6, there is a tendency for our method to be more efficient than SF Adaptation in direct and right relative interaction direction. Regarding the perceived level of realism, results showed only a main effect of the interaction direction (F(30,2)=3.97, p=0.03, η 2 p =0.21), with a lower realism when the virtual human is interacting to the right than towards the participant (Cf. Figure 7).

Discussion

Overall, the presented results highlight the impact that vision-based editing techniques might have on human-agent interaction in Virtual Reality. Even though we have not detected a relevant improvement in the perceived realism of the interaction, meaning that participants rated our edited interaction as realistic as the oriented one, we have observed a pertinent increase in the accuracy, thus participants could interpret easily the agent's aim when adapted with our technique. In apparent contrast with the current discussion, 100% of the participants mention eye's behaviour as one of the main and sometime the only (25%) feature used to identify the interaction's target. In fact this is not surprising, and even though eyes play a crucial role in user-agent interaction [START_REF] Steven | Eye contact: A nonverbal determinant of speaker credibility[END_REF][START_REF] Kleinke | Gaze and eye contact: a research review[END_REF], they are not always reliable, especially in situations when the proximity between the potential targets is narrow, or the eyes are not clearly visible (e.g. when the agent is far or not oriented towards the player). Hence, the results suggest that our technique can control body non-verbal communication, mentioned as secondary or primary feature in 75% of the answers, to enhance the interpretation of the waving orientation.

Moreover, it is important to discuss the disproportion we obtained between different relative directions of the target, for both techniques. We believe that this effect is caused by the design of the scene. Indeed, the right-most customers positions present, in general, a higher degree of occlusion relative to the clerks' desks, than the lefts ones (see Figure 4). Under this circumstance, we have reported a negative effect in realism and accuracy, for the relative right targets, which is partially mitigated in the accuracy by our technique.

Discussion and Conclusion

This paper proposes a motion warping technique that enables linking low-level motion variables with FoV-dependent visual motion features. Our results show that our approach is effective and applies to various cases, such as adjusting motions to changes in an observer's position, environment or lighting conditions. We also believe that this technique can be exploited to influence the expressions conveyed by animations (e.g. intro vs. extraversion) thereby helping designers to fine-tune the personalities of their characters or having virtual characters adapt their non-verbal communication towards observers (or avatars). Our proof of concept and the user experimentation in Virtual Reality validate the visual servoing scheme and yield a general and promising solution. While the method presents some analogies with inverse kinematics methods, through-the-lens techniques, or line of action control, it offers higher levels of control than just positions and velocities of joints in the image space.

Limitations and future works. Currently, our method is limited in multiple ways. First, it requires a prior selection of visual motion features to be controlled, warping units to activate, or magnitude and direction of visual features. Methods to select relevant combinations of parameters would improve the practical usability of the approach. Inappropriately chosen parameters can generate unwanted results that are partially filtered out by joint limits and stiffness. Additional kinematic filters of motion editing (e.g. balancing the center of mass, remaining in the human motion manifold, preventing self-collisions) could help reduce the negative effects of parameter's tuning. In addition, we only explored a limited set of spatial visual features while operators could also perform time warping, and measure features over a sliding window rather than on a per-frame basis. Directly integrating computational saliency techniques [START_REF] Neil Db Bruce | On computational modeling of visual saliency: Examining what's right, and what's left[END_REF] closer to visual attention mechanisms could also help to guide the warping of animations. Additional saliency biases could be added to account for top-down attention mechanisms specific to characters (e.g. focus of attention on head, eyes and hand movements) to build an attention-driven approach.

Second, the design of our warping units remains empirical. Existing work to automatically define rigging functions [START_REF] Holden | Learning Inverse Rig Mappings by Nonlinear Regression[END_REF] could improve over our solution. In our case, we could explore means to automatically correlate low-level motion variables with the variations of visual motion features, with the difficulty that these relations depend on the motion performed, the desired goal of editing, and the observer's position.

At this stage, we also left apart the question of setting the appropriate levels of visual motion features editing. We partially address this question for specific use cases, i.e. the catchingattention scenario where the levels are set as if observed from an optimal angle, defined during motion capture. Still it remains challenging in other situations, in particular in the case of controlling motion expressivity. By which level a feature should be adapted to change the expression of motion? Which parameters or kinematics limitations enable reaching the desired editing levels? A data-driven approach would be relevant to address these questions. The difficulty is twofold: one is to gather the required amount of data to capture feature level variations with corresponding semantics, the other one is to deal with human variability in such behaviours.

In this work, we focus on upper-body motion, with the intention of control gestures that are salient for non-verbal interaction. However, we believe that the current paradigm, based on controlling high-level visual features, can be applied to an heterogeneous set of motion synthesis tasks: lower-body motion, i.e. steps generation, or action planner, i.e. which hand to use in the waving motion, etc. Such synthesis tasks would help improve the realism of the interaction.

In our examples, we explore the warp of a single character. Undoubtedly, it is relevant to discuss the case in which multiple characters are warped simultaneously. Considering a group of characters seen from a single point of view, our approach would adapt their configuration to optimize a joint set of ap-parent visual features for that point of view. In terms of performances, this would require the evaluation of variations for each single character, but, as in Section 4.3, the total consumption would depend mostly on the total joint number of selected warping units. Rendering would not require additional computation. The dual case considers multiple point of views for a single character. Even though this approach is feasible, it would require the management of a compromise between possibly contradicting visual targets.

Conclusion.

In conclusion, the technique we propose is a viewpoint dependent motion editing approach that exploits a number of visual features from an external observer's point of view to drive and warp animations. As a result, this can empower creative artists, but also autonomous characters with means to control the information they convey to an observer.

Fig. 2 .

 2 Fig.2. Results from our three use-cases, displayed from the point of vision of the observer. Character parts/objects on which the estimation of visual features is performed are highlighted. On the left (cases 1.1 and 1.2), we study the influence of viewpoint changes for a character with and without an object. The desired visual feature consisted in increasing the visual coverage. In the middle (cases 2.1 and 2.2), we study the influence of solid and sparse occluders. Visual coverage was also used as the regulating visual motion feature. On the right (case 3) we explore how extraverted effects can be created by applying upper body warping units, and regulating their apparent vertical and horizontal extension. In the picture, the amplitude of the gesture of the warped character is wider with respect to the original non-warped one, making it feel more extraverted.

Fig. 3 .

 3 Fig. 3. Average computation time for three main parts of our technique, depending on the selected number of warping units. Left: time to compute the variations of visual feature values. Middle: time to compute the visual target in relation with the current visual feature values. Right: time to invert the Jacobian matrix. Results consider either one (blue) or four (orange) active apparent visual features.

Fig. 5 .

 5 Fig. 5. Main effects of the Technique and Interaction direction on participants' answer accuracy.

Fig. 6 .

 6 Fig. 6. Participants' answers accuracy depending on the technique and interaction direction.

Fig. 7 .

 7 Fig. 7. Main effect of interaction direction on the perceived realism.

Table 1 .

 1 Defined pose warping units for upper-body motions. The last three are independent for each arm.

		Bend forward-backward
		Bend left-right
		Rot. vertical axis
		Bend forward-backward
	Neck-head	Bend left-right
		Rot. vertical axis
		Flexion-extension
	Shoulders	Abduction-adduction
		Medial-lateral rot.
	Elbow	Flexion-extension Rotation on its axis
		Flexion-extension
	Wrist	Ulnar-radial deviation
		Supination-pronation

Acknowledgments

The authors wish to thank the participants of the experiment. This work received funding from the European Union under grant agreement No 856879 (H2020 ICT-25 RIA PRESENT project), as well as the Brittany Region, project SAD-2019-REACTIVE, No 19023.