Renée El

Melhem Bât

Pascal Blaise

M Stéphane

ÉLECTROTECHNIQUE E E A Électronique

M Philippe Delachartre
email: philippe.delachartre@insa-lyon.fr

Bénédicte Lanza

Mme Sandrine
email: sandrine.charles@univ-lyon1.fr

Mme Sylvie

M Hamamache Kheddouci

M Stéphane Benayoun
email: stephane.benayoun@ec-lyon.fr

Stéphanie Cauvin

M Jocelyn Bonjour

M Christian Montes

Animated characters are a crucial component of compelling and lively virtual worlds. Their motion is influenced by complex behaviors, and the production of expressive character animation heavily relies upon animators' skills, training and knowledge. Ever since computers have been used in this creative process, researchers have strived to develop intuitive, interactive and precise tools and interfaces to facilitate animators' work. In the last decade, deep learning methods have raised a lot of interest in the computer animation community, capitalizing on the increasing availability of real-life captured motion data to train generative models able to synthesize new movement. While the studied application allow for the generation of new realistic animation, the process unfortunately leaves little room for animators' control. This thesis therefore proposes to use neural networks to learn the subtle complexities carried by motion data and to use the extracted information in the design of user-centric animation tools which fit in the existing animation production workflow. We propose two approaches allowing a user to edit a character's pose at a given time frame, first by directly manipulating its skeleton, and second through higher-level pose parameters.

Département de la Formation par la Recherche et des Études Doctorales (FEDORA) Bâtiment INSA direction, 1 er étage 37, av. J. Capelle 69621 Villeurbanne Cédex fedora@insa-lyon.fr Référence : TH0955_Léon VICTOR L'INSA Lyon a mis en place une procédure de contrôle systématique via un outil de détection de similitudes (logiciel Compilatio). Après le dépôt du manuscrit de thèse, celui-ci est analysé par l'outil. Pour tout taux de similarité supérieur à 10%, le manuscrit est vérifié par l'équipe de FEDORA. Il s'agit notamment d'exclure les auto-citations, à condition qu'elles soient correctement référencées avec citation expresse dans le manuscrit.

Par ce document, il est attesté que ce manuscrit, dans la forme communiquée par la personne doctorante à l'INSA Lyon, satisfait aux exigences de l'Etablissement concernant le taux maximal de similitude admissible.

Context

Animated virtual characters are present everywhere in video media such as films, animated movies or video games. They can often be found in the spotlight, telling stories or embodying players, but also in the background, populating vivid worlds where they strengthen the spectators' immersion. The way these characters move, act around their world and interact with each other is an important point of focus for their creators, and producing high quality, expressive and diverse animation is a crucial aspect of the production process.

Humans have been interested in depicting moving scenes for as long as they have been drawing. Early animation thus has centuries of history, starting with side-by-side wall paintings, to puppets and shadow play, and up to the photorealistic digital humans of today. The modern understanding of the discipline arose recently, following the democratization of the cinematic industry during the late 19th century. The beginning of the 20th century saw the development of the traditional animation technique in which animation artists draw characters by hand, image per image. This period also marks the first public success of animated production, first through short-format cartoons, quickly followed by feature-length films. Nearing the end of the century, computers made their way in the animation creative process. In their first appearances digital tools were used to reduce the burden of repetitive work and improve production times, but still operated on flat 2D images. The focus however quickly shifted to the 3D world, driven by the fast growth of computing power and progress in computer graphics. Nowadays, the most common approach to animation production is entirely computer-based, and is the result of virtual cameras filming virtual scenes in which 3D models act, even in cases where the final image aims for a 2D look. With the rise of 3D video games, the growing scope of animation feature films, and the looming prospect Chapter 1. Introduction 2 of high-quality immersive experiences in virtual reality, animation is in ever-increasing demand.

The human eye perceives a lot of information through motion: it is a complex behavior through which characters convey intent, emotions and personality. A same gesture can be performed differently depending on the unique identity of the performer, and a character can achieve the same action in a variety of ways. Creating high-quality motion that accurately depicts a character's actions and convey its personality and internal state is a difficult task, and animators undergo long and meticulous training to master their craft. The role of computer animation researchers is then to imagine and develop new tools and interfaces to help artists transform their imagination into tangible art, and bring their characters to life.

Motivation

In this thesis, we are interested in proposing a new set of such tools. We are also interested in the information present in existing motion data, and in ways it can be extracted then exploited in the design of the tools. Our approach thus falls in the "data-oriented" category of animation research, which has been a topic for the community since its very beginning.

During the last decade, neural networks and more specifically deep learning methods have seen an explosion in popularity as the computational power and amount of data they require have become accessible to larger audiences. They have proven to excel at learning to model complex data distributions from database of existing samples, and at approximating complicated functions. Their usage have quickly grown out of their original image classification root: they are now studied in application to all sorts of data types [START_REF] Qin | Remote Sensing Single-Image Resolution Improvement Using A Deep Gradient-Aware Network with Image-Specific Enhancement[END_REF][START_REF] Delconte | Tree Defect Segmentation Using Geometric Features and CNN[END_REF], and towards various goals, up to synthesizing new unseen data from the modelled distribution. The animation field has not been spared, and, starting from 2015, a larger and larger amount of research has been dedicated to applying neural networks to motion data. However, as shown in the following literature review, the vast majority of existing work in this direction has focused on animation synthesis, i.e. the task of generating new motion, occasionally guiding the generation through a high level control signal such as an expected locomotion trajectory or a past context of existing motion. While these methods produce impressive results for on-the-fly synthesis, they are limited when it comes to artistic control. Animators creating new motion need precise control over their tools' output, so that they can author an animation that precisely fit their expectation for it. With existing methods, if the generated motion is not satisfying, the only solution is to retrain the whole models with a different dataset.

This observation motivates the objective of this thesis: exploiting the modelling power of deep learning methods to extract meaningful information from animation data, and using it to power tools which facilitate the artists' job while leaving them in control of the final output. We therefore propose methods that apply the insight learned from data to more fine-grained operations and fit in the existing animation pipeline. To achieve our goal, the proposed tools focus on being unintrusive, controllable, easy to use, and fast enough to allow interactive use.

Thesis contributions & organization

The work presented in this thesis revolves around the idea of leveraging existing data in the design of atomic operations in the animation workflow. We select and focus on the task of skeleton manipulation, during which the user edits a character's pose at a specific time stamp. This choice is made considering that it is an essential step in the process that, in its usual format, heavily relies on the animator's understanding of complex skeleton constraints. We present a new pipeline in which said constraints are learned and handled by a few neural networks, while the users can focus on editing a character's pose through simple and efficient manipulators.

Our contributions constitute the building blocks of this pipeline, which are presented throughout this thesis.

The first chapter gives an overview of the literature in computer animation, presenting the fundamental concepts, the efforts that lead to the current workflows used to produce animation, and the most recent experiments aiming to reduce the burden of animators, including animation data modelling with, among other methods, neural networks. Through this presentation it also illustrates the shortcomings of existing methods that our work attempts to resolve.

The second chapter presents the general idea of this thesis, which is to use neural networks to learn an alternative (latent) representation space for animation frame data, then edit latent pose representation, leaving the handling of complex skeleton constraints to the learned mapping functions. We present the dataset used to train our neural networks and discuss two approaches to building the latent space.

A first application of the space is then presented in the fourth chapter. We focus on the task of pose editing through the manipulation of some of its joints, similarly to a virtual puppet. Two methods are presented and discussed.

Chapter five investigates the animation style metaphor and its applications in the thesis' paradigm. We present a pose editing method in which the pose is modified following some high-level but objective pose parameters, using a learned editing function.

Finally, the fourth chapter concludes this thesis by summarizing our findings, discussing them and suggesting further improvements.

Introduction

Ever since the early days of computers, animators and researchers have looked for how they could be used to improve the animation process, which results in a large body of literature. The goal of this thesis being to explore how neural networks can be used to provide smart animation tools, it is a direct continuation of this past effort. This chapter gives an overview of the previous work our research builds upon.

Nowadays two dominant methods to produce animation coexist. Key-framed animation is the direct successor of traditional pen-and-paper animation: animators manipulate a character, setting its poses at selected time frames. The motion is created by playing these key-frames in sequence, interpolating between them if necessary. Motion Capture (MoCap) techniques instead focus on recording real-life actors moving, then applying the captured motion to virtual characters. The work presented in this thesis makes use of both methods: while we aim to develop tools that fit in the key-frame animation pipeline, we take advantage of the naturalness and relative accessibility of MoCap data to train neural networks. While our work depends on the existence of faithful motion data, the capture process itself is a broad topic, and is kept out of this report for shortness sake1 . Similarly, the animation of character faces, expressions and speech motion are specific and widely studied subjects[SS18; HP22], out of the scope of this review. We instead focus on the different steps used to produce key-frame animation, to explain the context and the limitations we aim to alleviate.

The first section 2.2 gives an overview of what "animation data" entails, and how it is applied to animate character models. Then in 2.3 we discuss the different existing methods used to design an animation through the manipulation of key frames. Section 2.4 we present the efforts towards reusing existing animation data, leading to recent work dedicated to modelling it through deep learning methods. Finally, in section 2.6, we describe the different proposed approaches to edit an animation through the metaphor of its "style".

Chapter 2. Literature review 6

Skeletal animation

In computer graphics, virtual characters are represented by three-dimensional polygon meshes on which texture images and physically-based materials are applied. Their motion is driven by an underlying hierarchical structure of joints (or bones) which collectively form the skeleton (Fig. 2.1). When the skeleton moves, it drives the deformation of the mesh by applying its bones' transformations to the vertices, in a process called skinning [START_REF] Chadwick | Layered Construction for Deformable Animated Characters[END_REF]. The influence of a given bone on a given vertex is specified by a weight factor that artists can tweak to achieve the desired deformation behavior. The modeling of the character and the skinning process are out of the scope of this thesis, as we are only interested in the skeleton and its motion. Animation data consists of a time series of skeleton configurations (key poses) at specific time stamps (key frames). A pose is usually defined by the transformation (translation, orientation, and scale) of each bone in the skeleton. At runtime, the computer interpolates between these poses to display a smooth animation (Fig. 2.2) [START_REF] Burtnyk | Computer-Generated Key-Frame Animation[END_REF]. Within that framework the role of an animator is then to design the key frames to produce the character's motion. This skeleton representation has multiple advantages: It is mainly a memory-efficient representation for animation data, as it stores only per-frame bones transformation and fixed vertices bone weights, instead of per-frame per-vertex transformations for mesh animation. Additionally, its simplicity makes it an effective interactive interface for animators, which can manipulate it much like a puppet to achieve desired poses [START_REF] Burtnyk | Interactive Skeleton Techniques for Enhancing Motion Dynamics in Key Frame Animation[END_REF].

Key-frame editing

The common key-frame animation synthesis pipeline can be split in two phases: posing and interpolation control. During the first, animators manipulate the skeleton to define key poses at specified key frames. In the second, they control the various processes used to fill the gaps between key-frames.

Pose manipulation

Virtual characters' skeletons can be highly complex: they are composed of many joints, each of which can have specific restrictions, such as orientation limitations.

They must also respect many constraints specific to the character they represent.

For instance, the distance between each pair of joint must remain constant; or can conditionally allow for a limited amount of stretch, and the limbs must never overlap when accounting for the morphology of the character. Manually parameterizing each joint of a skeleton to respect these constraints is next to impossible, even more so when multiple poses are aligned and must also respect temporal coherence.

A substantial part of the literature is then dedicated to describing methods which facilitate the manipulation of the skeleton. These tools are in direct contact with animators, so their focus is on ease-of-use as well as real-time interactivity. We review the different approaches to this kind of interfaces, starting with forward and inverse kinematics methods, to sketching interfaces, data-driven methods, and finally interfaces in three dimensions. The most straightforward skeleton manipulation tool is Forward Kinematics (FK): the parameterization of one joint in the chain is the result of combining those of all the joints up the chain. This way the modification of a joint's position or orientation is echoed down the chain to the end effector (see Fig. 2.3). Using this method, an animator can change a joint's orientation to edit the pose, ensuring the constant length of each bone.

Forward and inverse kinematics

The opposite technique, Inverse Kinematics (IK), finds a suitable parameterization of the chain given a target parameterization of the end-effector. This can be used as an effective interface for skeleton manipulation where animators can drag a joint over a desired position or orientation while pinning some others in place (pin-and-drag interface [START_REF] Yamane | Natural Motion Animation through Constraining and Deconstraining at Will[END_REF]). The IK problem is an ill-posed one, as a given target configuration may have one, multiple, or no solution. A recent review of IK techniques in computer graphics [START_REF] Aristidou | Inverse Kinematics Techniques in Computer Graphics: A Survey: Inverse Kinematics Techniques in Computer Graphics[END_REF] proposes to split the existing solutions in four families (Analytic, Numerical, Data-driven2 , and Hybrid) depending on their characteristics, analyzing their strong and weak points. The following section presents a restricted summary of their review focusing on the methods that are best suited for real-time IK applied to character skeletons.

Analytical solutions attempt to find a formulation of the problem which yields an exact solution given a target configuration [START_REF] Raghavan | Inverse Kinematics of the General 6R Manipulator and Related Linkages[END_REF][START_REF] Kallmann | Analytical Inverse Kinematics with Body Posture Control[END_REF]. They have the advantage of being reliable and of having global solutions, but they can be computationally costly. Finding an analytical solution is also be next to impossible for complex, strongly-constrained systems, such as a character's skeleton. They therefore don't see much use in pose edition, but are good candidates for robotics.

Numerical solutions formulate a cost function for the problem and iteratively minimize it. They can be divided in three categories: Netwon, Heuristic, Jacobian. Newton solutions use the quasi-Newton method to approximate the objective function [START_REF] Nocedal | Sequential Quadratic Programming[END_REF], but much like analytic solutions, they are too complex and computationally costly for the use-case at hand. Jacobian methods use the Jacobian of the system (a matrix of the partial derivatives of each joint's configuration with respect to their angles), as a linear approximation to the problem [START_REF] Samuel R Buss | Introduction to Inverse Kinematics with Jacobian Transpose, Pseudoinverse and Damped Least Squares Methods[END_REF]. The use of an approximation allows for lighter computations and faster inference, which made Jacobian solutions popular during the 1980s and 1990s. In the recent years they have however fallen out of favor due to the demanding environment of real-time interactive software. Heuristic methods do not try to generalize the IK problem and instead rely on simple operations that they can repeat iteratively to reach an acceptable solution. This process allows them to be extremely computationally efficient, at the cost of having no consideration for the naturalness of the produced pose. This drawback can be attenuated by extending the methods to respect more constraints, which is fortunately easy to do in most of them. Their speed make them the currently preferred solution in animation software, so we next describe in deeper details two of the most popular heuristic IK algorithms: Cyclic Coordinate Descent (CCD) and Forward and Backward Reaching Inverse Kinematics (FABRIK).

The CCD algorithm [LY84; WC91] operates iteratively to minimize the distance and rotation error between the end effector and the target. At each step, the end-effector is rotated to align with the target. Then, each joint down to the root is rotated to minimize the error. The process is repeated until the end-effector reaches the target. The success of CCD can be attributed to its simplicity: it is easy to implement and has low computational cost. The algorithm has also received multiple improvements, allowing it to handle different kind of constraints [START_REF] Welman | Inverse Kinematics and Geometric Constraints for Articulated Figure Manipulation[END_REF] or multi-chains problems [Shi+01; KM05; KMA05].

FABRIK [START_REF] Aristidou | FABRIK: A Fast, Iterative Solver for the Inverse Kinematics Problem[END_REF] is an iterative solver that gradually modifies the positions of the chain's joints to minimize the distance between the target and the end effector, operating in two stages (forward and backward). In the forward stage, the end-effector is placed on the target. The other joints are updated one by one down to the root, and placed on the line between their current position and their child's new one, respecting the distance between them. In the backward stage, the same process is followed, only the other way around: the root is placed back on its original positions, and the joints up the chain are "pulled back" one by one to it. The two phases are repeated until the end effector is close enough to the target. The advantages of FABRIK are its simplicity and flexibility: the algorithm has been proven to always converge when the target is placed within reach [START_REF] Aristidou | Extending FABRIK with Model Constraints[END_REF], and has been extended to handle many kinds of constraints [MC13; ACL16; TY17].

Hybrid solutions usually divide the IK task in sub-problems, and solve each of them with a specific method. For example, some approaches reduce the search space of a numerical solution by using a preliminary analytical step [START_REF] Lee | A Hierarchical Approach to Interactive Motion Editing for Human-like Figures[END_REF]. Others use data-driven methods for the same purpose: Agrawal et al. [START_REF] Agrawal | Task-Based Locomotion[END_REF] learn a prior which is used to warm-start FABRIK, yielding more natural looking human poses.

Forward and inverse kinematics are at the core of most modern animation software and lead to precise results, but can be tedious to use from an artist's perspective.

Other techniques and interfaces have then been studied to alleviate this problem. A body of work has then taken interest in translating this workflow to computer animation (some of the literature's methods are illustrated in Fig. 2.5). The main difficulty in adapting the sketch abstraction is the reconstruction of a full 3D model from a flat 2D drawing: the additional depth dimension cannot be precisely inferred, which is a source of ambiguity.

Sketching interfaces

Early work along this idea required users to draw stick figures, which are commonly used as rough depictions of characters, and have the advantage of corresponding to the widely used skeleton model. The stick-figures could either be sketched from scratch or over an existing doodle, and either on paper [TBv04; CON05] or directly on the computer [START_REF] Fekete | TicTacToon: A Paperless System for Professional 2D Animation[END_REF]. They are then used to generate a 3D pose. More recently, some investigated a simpler abstraction inspired by the workflow of traditional cartoonists [LB84; Bla48], the Line of Action (LoA). The LoA is a single smooth stroke over the character's body which indicates the general aesthetic shape of the pose, as illustrated by Fig. 2.6. It has been shown to be an intuitive interface to produce expressive poses, manipulating either the skeleton [START_REF] Guay | The Line of Action: An Intuitive Interface for Expressive Character Posing[END_REF] (illustrated in Fig. 2.5 (B)) or the character mesh directly [Özt+13]. The method has been generalized to allow the user to provide his own curves [START_REF] Hahn | Sketch Abstractions for Character Posing[END_REF].

Data-driven Inverse Kinematic

A more recent group of methods leverage existing real-world animation data to generate poses which correspond to some provided user constraints. The general idea is to match the constraints by producing a new feasible parameterization extracted from a database, or to synthesize a new one similar to the database. We present data-driven IK methods here as a majority of them is interested in generating a full character pose based on the target parameterization of a few of the skeleton's joints, which is in turn applied to character posing.

The initial approach from Rose et al. [START_REF] Rose | Artist-Directed Inverse-Kinematics Using Radial Basis Function Interpolation[END_REF] uses radial basis functions to interpolate between multiple poses whose joints most closely match the provided targets. Grochow et al. [START_REF] Grochow | Style-Based Inverse Kinematics[END_REF] propose to model the probability distribution of the database using Gaussian Process Latent Variable Models. The new pose is generated at run time by optimizing a function which represents the likelihood of a given pose to satisfy the constraints. The method was later improved to take into account the physical properties of motion [START_REF] Liu | Learning Physics-Based Motion Style with Nonlinear Inverse Optimization[END_REF].

Another family of methods suggest searching large pose databases for the closest match. The main limitation is the complexity of the search: in a naïve interpretation it grows linearly with the size of the database. To reduce this complexity, the proposed methods suggest reducing the dimensionality of the data through PCA [CH05; RB09], or speeding up the searching with the help of kd-tree clustering [Krü+10; WTR11] or maximum a-priori (MAP) estimations [START_REF] Wei | Intuitive Interactive Human-Character Posing with Millions of Example Poses[END_REF]. Other learning methods have been applied to character posing, including hierarchical clustering [START_REF] Eng | Learnt Inverse Kinematics for Animation Synthesis[END_REF], sparse dictionary learning [START_REF] Ranch | Interactive Character Posing by Sparse Coding[END_REF] and multi-variate Gaussian distribution models [START_REF] Huang | Multi-Variate Gaussian-Based Inverse Kinematics[END_REF].

3D posing interfaces

While all the previously described methods allow easier manipulation of character poses, they collectively face a final limitation: their goal is to handle a 3D object, the skeleton, through a 2D interface, the mouse and keyboard. Such indirect manipulations are notoriously hard to master and require the animators to be trained with a specific software interface. To mitigate this limitation, a group of methods focuses on direct manipulation interfaces. A first group of such work focuses on building tangible pose editing devices, in the form of actuated puppets [Kne+95; Yos+11; Jac+14], allowing animators to pose a physical representation of the character (see Fig. 2.7). Another one is interested in Virtual Reality interfaces [START_REF] Lamberti | Is Immersive Virtual Reality the Ultimate Interface for 3D Animators?[END_REF], which offer opportunities to solve the problem by embodying the user in a full 3D world. Interfaces have been explored where the user manipulates the skeleton in 3D through IK handles [START_REF] Suma | FAAST: The Flexible Action and Articulated Skeleton Toolkit[END_REF] or by tracking its hands and interpreting their motion as strokes similar to the Line of Action [START_REF] Garcia | Spatial Motion Doodles: Sketching Animation in VR Using Hand Gestures and Laban Motion Analysis[END_REF].

In-betweening

Once key poses have been set, the software is tasked to generate intermediate poses (generally called in-betweens) between them to produce motion. To achieve this, the straightforward approach that comes to mind is to linearly interpolate between key-frames. Linear interpolations however suffer from limitations [START_REF] Doris | Interpolating Splines with Local Tension, Continuity, and Bias Control[END_REF], as illustrated in Fig. 2.8:

• linear interpolation of orientation values provoke lengths distortions;

• sudden changes in the direction of motion might produce non-smooth animation;

• irregular time-placement of key-frames can lead to discontinuity in speed; A common solution to the first problem is to represent joint rotations by quaternions, and to interpolate using the spherical linear interpolation function (Slerp) [START_REF] Shoemake | Animating Rotation with Quaternion Curves[END_REF]. The second one was solved by smoothing the edges using cubic interpolation splines (using Catmull-Rom splines [START_REF] Doris | A Computer System for Smooth Keyframe Animation[END_REF] or B-splines [START_REF] Steketee | Parametric Keyframe Interpolation Incorporating Kinetic Adjustment and Phrasing Control[END_REF]), and the third by allowing the user to parameterize them through control points [START_REF] Bartels | Speed Adjustment for Key-Frame Interpolation[END_REF][START_REF] Doris | Interpolating Splines with Local Tension, Continuity, and Bias Control[END_REF]. Taken all together these solutions amount to a set of parametric curves (often called animation curves) that the animator can manipulate to control the interpolation process (Fig. 2.9).

This formulation of the animation curve is currently prevalent in production setups.

The resulting motion's naturalness is however still dependent on the user's skill and carefulness, and so some more work has been dedicated to improving the process. A group of work propose to optimize the animation curves to satisfy all sorts of constraints. They have been applied to ensure that the motion remains physically plausible [SN88; LC95], and particular focus has been put on detecting and avoiding self collisions [Bad+94; Neb99].

More recently, another group focused on refining the curve editing process itself, making use of user-in-the-loop optimizations to gradually modify the curves with regard to the user input [KG18; CÖS19].

Data-oriented solutions have also attempted to improve the process, learning real-world constraints from motion capture data. The general idea is to generate the in-between frames while using this knowledge to ensure that the generated poses are all realistic. The first approaches used statistical modelling combined with optimization [START_REF] Chai | Constraint-Based Motion Optimization Using a Statistical Dynamic Model[END_REF] or Gaussian processes [START_REF] Grochow | Style-Based Inverse Kinematics[END_REF]. Recently, recurrent neural networks (RNN) have been trained to produce in-betweening for a single specific character [START_REF] Zhang | Data-Driven Autocompletion for Keyframe Animation[END_REF][START_REF] Félix | Recurrent Transition Networks for Character Locomotion[END_REF]. Such methods are described further in the deep learning section below.

Reusing motion data

Producing animation through the interpolated key frames workflow as described in the previous section is a complicated process, and creating high quality motion remains a time-consuming task even for experimented artists. MoCap can allow arguably faster production, but the capture is not a light process. Actors, technicians and animators must be coordinated, and most production studios do not have the necessary technology on-site, so careful planning is mandatory. A missing or unfit animation clip arising in the middle of production involves re-shooting it and thus important time costs.

This complexity in the acquisition of motion data has led to an interest in ways to re-use existing animation and a fair amount of research has been devoted to the question.

Motion signal processing

The first group of work that proposed a solution to alter and reuse existing motion took inspiration from existing signal and image processing literature [START_REF] Bruderlin | Motion Signal Processing[END_REF]. In these approach the motion of each animation parameter (such as a joint's rotation) is considered as a sampled signal, which can be filtered and decomposed in multiple resolution scales. The applications shown in the original include tweaking an animation by editing the values of individual bands, changing the timing of a clip (time-warping [START_REF] Witkin | Motion Warping[END_REF]) or using the signals as interpolation targets to blend multiple clips. Following work have shown applications in adaptation to random terrain or morphing the character's skeleton size [LS99; LS01], or altering the style characteristics of the original motion, by applying random noise to different resolution scales [START_REF] Perlin | Real Time Responsive Animation with Personality[END_REF], or by applying a style transformation computed as the difference between a "neutral" motion clip and another "stylized" one [START_REF] Amaya | Emotion from Motion[END_REF].

Motion blending

The goal of motion blending is to interpolate between two (or more) distinct animation clips to generate a seamless transition between them. While the blending operation can be used on its own to create new motion, it can also be used to alleviate the amount of original animation needed in a production. For example, if the short transitions between the longer clips of a character's animation set are automatically generated, animators can focus on refining the details of the most characteristic parts of its movement.

The very first approaches to blending used linear interpolations to satisfy user-provided constraints, such as IK targets, either directly in the animation parameter domain [GRG96; WH97], or in the spectral domain through Fourier decomposition [START_REF] Unuma | Fourier Principles for Emotion-Based Human Figure Animation[END_REF] or the aforementioned multi-resolution technique [START_REF] Bruderlin | Motion Signal Processing[END_REF]. The main restrictions of these early approaches lies in the linear interpolation operator which cannot represent the subtlety of human motion and thus limits blending to similar input clips. A more in-depth analysis of some motion blending techniques, including numerical evaluations and comparisons, can be found in the review by Feng et al. [START_REF] Feng | An Analysis of Motion Blending Techniques[END_REF].

Motion graphs

One of the objectives of data-driven animation synthesis is to organize a database of existing motion clips in a way which allow playing them back to back, on a same character, without breaking the animation. This type of organization is also important in interactive media such as video games: virtual avatars must be able to move and act across the world while responding to user input, which means seamlessly transitioning from the current animation to the one corresponding to the latest player input. The problem grows exponentially with the number of available actions available to the player, as each new task could be combined with the existing ones. To tackle this complexity, a group of work has studied the idea of building animation graphs, in which each edge represents an animation clip, and each node represent a pose, which represents a transition point between two (or more) clips. In this configuration, traversing the graph produces a single smooth animation in which each clip is played with no noticeable transition.

In their first appearances, motion graphs were fixed and structured by hand. used in the same way as the manual ones, i.e. searched and traversed to satisfy user provided constraints.

The searching costs of a motion graph can grow exponentially along with the number of edges. Another group of work thus focuses on improving the process, using specialized path-search algorithms [START_REF] Safonova | Construction and Optimal Search of Interpolated Motion Graphs[END_REF] or pre-computing cost functions [LL04; SMM05].

The "quality" of motion graphs have also been studied, with methods to evaluate the capabilities of a graph in a given scenario [START_REF] Paul | Evaluating Motion Graphs for Character Animation[END_REF] or to enhance their connectivity [START_REF] Zhao | Achieving Good Connectivity in Motion Graphs[END_REF].

Some others propose alternative graph definitions where the graphs are denser and more structured. Shin and Oh [START_REF] Joon | Fat Graphs: Constructing an Interactive Character with Continuous Controls[END_REF] introduce "fat graphs" in which clips between which blending is possible are grouped together. In [START_REF] Heck | Parametric Motion Graphs[END_REF], both node and edges contain multiple animation clips. A walk through these fat graphs can produce even more motion by blending between available animation clip at each step. Having multiple options also allows for finer constraining, at the cost of search efficiency [START_REF] Min | Motion Graphs++: A Compact Generative Model for Semantic Motion Analysis and Synthesis[END_REF].

Motion graphs only allow transitions between clips when reaching nodes, which is limiting in applications where interactivity is paramount. Because they only use pre-existing animation data, coupling them with techniques which perturb the original clips (physics simulation, layered animation...) is difficult, as it would require straying away from pre-determined transitions. To alleviate these limitations, some work has proposed interactive controllers, learning to navigate the graph through reinforcement learning [TLP07; MP07].

Motion graphs are now widespread in the games industry. The graphs are however still hand-built, and used as state-machines to drive animation from character state (Fig. 2.10). Automatic construction approaches have indeed proven difficult to control, which could cause unacceptable latency in the response to player input. The graph construction itself has then become a time-consuming task for animators, which has led the research community to exploring other avenues for animation synthesis.

Motion fields and motion matching

Interactive applications require quick reaction to user input and external perturbations, for which motion graphs come lacking. Another family of approaches has then emerged: instead of modelling the single most suitable motion from any possible character state, as done with graphs, motion fields [START_REF] Lee | Motion Fields for Interactive Character Locomotion[END_REF] suggest modelling a set of suitable motions. The field itself is composed of vectors containing the extracted instantaneous pose and velocities from each motion clip. At each frame, the most suitable motion clip according to the user input is selected interactively and blended into. The control of the flow is based either on Markov Decision Processes (MDP) or reinforcement learning. As a result the current state is always a blend of multiple animations, staying in the general vicinity of them while immediately reacting to user input.

More recently, another method called Motion Matching has emerged from the video game industry. First presented by a team from Ubisoft [BC15; Cla16; Zad16] as a greedy simplification of motion fields, the method keeps the same main idea: at every frame, search for the best suited transition over an entire motion capture database. Motion Matching drops the complex task of constructing the field, as well as the control methods. Instead, the whole database is kept in memory, and the best matching clip at each time step is selected through nearest-neighbor search. The simplicity and controllability of motion matching has quickly drawn attention from many game studios [Har18; Zin19; Büt19] which extended the method to suit their need.

The main limitation of Motion Matching is the memory footprint of the animation database, which scales linearly with the amount of data. The academic field has started to take interest in the method and provided attempts to handle this problem. For instance Holden et al. [START_REF] Holden | Learned Motion Matching[END_REF] replace different parts of the process with neural networks, removing the need to store animation data in-memory.

Statistical modeling

One last family of approaches proposes to construct statistical models of animation data. A proper model can be helpful in analyzing and describing the subtle complexity of motion data, but most importantly in our case, it can be used to generate new motion with respect to its underlying distribution. Borrowing various modeling techniques from other fields and leveraging the increasing amount of available material, the models presented by researchers have grown more and more precise and powerful.

Principal Components Analysis (PCA)

The first statistical models of motion extend existing mathematical models of 3D objects deformations to incorporate motion. They rely on Principal Components Analysis (PCA) to transform animation data in uncorrelated variables and select the most representative ones, resulting in a more compact and general representation. For example, Alexa et al. [START_REF] Alexa | Representing Animations by Principal Components[END_REF] apply PCA on single animation clips directly, in order to learn a new representation decoupled from the geometry. Such representations of motion data have also been used to speed up the search process in a motion database [START_REF] Forbes | An Efficient Search Algorithm for Motion Data Using Weighted PCA[END_REF]. A popular use of PCA has been to use it to reduce key frame data dimensionality, then to model the probability of frame-to-frame transitions (temporal dynamics) with variations of Markov Models [Bow00; BH00; MH00; GJH01].

The reduced subspace representation of motion data obtained through PCA can also be useful in order to reduce the size of the search space of optimization methods. This property has been leveraged to optimize motion to respect various constraints such as physical rules [START_REF] Safonova | Synthesizing Physically Realistic Human Motion in Low-Dimensional, Behavior-Specific Spaces[END_REF], key frames and trajectories [START_REF] Chai | Constraint-Based Motion Optimization Using a Statistical Dynamic Model[END_REF], or sketching interfaces [START_REF] Min | Interactive Generation of Human Animation with Deformable Motion Models[END_REF].

Gaussian Processes

Gaussian processes and their derivatives have also been used to model the distribution of motion data. Thanks to their probabilistic nature they can be used to generate any data points instead of being strictly limited to inter or extrapolations. The seminal work of Grochow et al. [START_REF] Grochow | Style-Based Inverse Kinematics[END_REF] used Gaussian Process Latent Variable Models (GPLVM) to model the pose space and demonstrated their purpose with an interactive pose editing interface. Further work employed recurrent variants of Gaussian Processes to predict future frames [START_REF] Wang | Gaussian Process Dynamical Models for Human Motion[END_REF], automatically respect physical constraints [YL10; WMC11] or build character controllers [START_REF] Levine | Continuous Character Control with Low-dimensional Embeddings[END_REF]. Gaussian methods have produced impressive results, but have not seen much adoption outside the academic field. This can be attributed to their inherently high memory cost, since they need access to the full motion database in memory in order to be used.

Deep Learning

From 2015 and on, encouraged by their success in related fields, deep learning methods appeared in the animation community. They proved to be effective at learning complicated patterns from raw, real-life data, with the additional advantage of not requiring the training data to still be available after training. On top of that, their inference process usually consists in a set of straight-forward deterministic operations that can be executed fast enough for real-time applications. Thanks to their general popularity across research fields and these interesting properties, deep learning-based methods quickly grew to represent a large portion of new work in computer animation research (see Fig. Deep learning methods have been applied to a broad array of animation tasks, and many network architectures have been studied; the combinations and intertwining of both making classifying existing work difficult. In this thesis we choose to separate both part and give an overview of the applications in a first section, only surveying the architectures in a second one.

Applications of neural networks in animation

Motion synthesis The first applications of deep learning in animation followed the general idea of previous work with statistical models: model the distribution of motion data, and generate new motion sequences as varied as possible, whilst ensuring the naturalness of produced motion and in some case respecting some user-provided constraints. In the second half of the 2000s, Taylor et al. [THR06; TH09] pioneered the use of neural networks with this task, studying variations of Restricted Boltzmann Machines (RBMs) to predict the next pose in a motion clip. Over the years, the task remained one of the most studied, but has been derived in a series of flavors.

On one end of the spectrum, deterministic generation applications' goal is to extrapolate an existing motion to closely predict its near future [START_REF] Fragkiadaki | Recurrent Network Models for Human Dynamics[END_REF]. The difficulty then lies in the accurate reconstruction of a truncated motion sequence.

On the other, generative synthesis refers to the task of producing realistic motion sequences from a random seed [START_REF] Holden | Learning Motion Manifolds with Convolutional Autoencoders[END_REF]. Here, the objective is not reconstruction but rather the diversity and realism of the produced animation.

Between the two exist a myriad of similar tasks, whose objective vary between reconstruction accuracy and variety. But they most notably differ in how the synthesis of new animation is constrained. The most common one is to enforce the match with a past context to synthesize various follow-ups to an existing clip [START_REF] Fragkiadaki | Recurrent Network Models for Human Dynamics[END_REF]. The context can however also be a future one, or past and future context can be enforced at the same time [Ber+15; Zv18; Har+20] (in which cases the task becomes similar to motion in-betweening, discussed earlier). Constraints are also used to provide some control to a user, or to restrict the possibility space of the generation, in which cases the synthesis in constrained by a user-provided trajectory signal [START_REF] Habibie | A Recurrent Variational Autoencoder for Human Motion Synthesis[END_REF], action types [Gho+17; Gho+20], or the character's environment [START_REF] Cao | Long-Term Human Motion Prediction with Scene Context[END_REF].

Motion control

The next task attempts to use deep learning to create character controllers which react interactively to user input, as well as the character's virtual environment. Two categories of solutions to this problem respectively train jointactuated, physically simulated agents [Pen+17; Pen+18] and musculoskeletal models of the human body [Gvv13; Nak+18], to act in a physically simulated world. As these techniques are not directly related to the topic of this thesis, we focus on the last category, which aims to generate the animation data used to drive a skeleton.

This family of approaches has been pioneered by Holden et al. [START_REF] Holden | Phase-Functioned Neural Networks for Character Control[END_REF]. In their work the user is able to provide a phase function describing the alternating footsteps of the character, and at each frame the network can provide the next based on the current pose, the current phase, and the desired direction vector, providing an impressive character locomotion controller (see Fig. 2.13). This approach was later improved,

Motion editing

The final part of the deep learning literature for character animation focus on editing, rather than generating, motion. The first of such applications were a byproduct of techniques developed for other tasks. Many early neural methods leveraged learned "motion manifolds" to which real animation data is mapped to and from. Because the decoding operation is trained to produce realistic motion, projecting corrupted or noisy data to the manifold and back to motion space can reproduce and clean up the missing information [HSK16; Büt+17; Wan+21]. Some work has also been dedicated to refining this property, by modifying the training loss to penalize bone-length error [START_REF] Li | Bidirectional Recurrent Autoencoder for 3D Skeleton Motion Data Refinement[END_REF] or by optimizing the latent space [START_REF] Lohit | Recovering Trajectories of Unmarked Joints in 3D Human Actions Using Latent Space Optimization[END_REF].

Some work has also investigated how deep neural networks could help in retargeting animation, so that a motion sequence created for a character could be adapted to another one with a different morphology (see Fig. A last part of the literature is interested in using deep neural networks to extract and transfer the style of a given animation to another. Style being of the focuses of this thesis, these methods are described along with their non-neural equivalents in a later section (2.6.3).

Network architectures

With their immense popularity, deep learning methods have been extensively studied during the past decade, inevitably spawning a broad array of models and architectures.

The animation community has not been spared, and many of those have been applied to motion data, one of its underlying difficulty being its spatio-temporal aspect: the temporal coherency of a sequence of frame is important, but so is the relations between each joints' parameters and the kinematic structure of the skeleton.

Recurrent Neural Networks

One of the first architectures applied to animation data is RNNs, in which the encoding of a pose is fed back to the network when encoding the subsequent ones, to model the temporal dynamics of motion sequences. The seminal work from Fragkiadaki et al. [START_REF] Fragkiadaki | Recurrent Network Models for Human Dynamics[END_REF] uses an encoder-decoder to construct a latent representation of poses, then a RNN to model the dynamics between them. This general architecture was later extended to include structural information on the skeleton [Jai+16; AKH19; GC19], and improved by changing the pose representation from joint angles in Euler representation to joint velocities [START_REF] Martinez | On Human Motion Prediction Using Recurrent Neural Networks[END_REF] or quaternions representation [START_REF] Pavllo | Modeling Human Motion with Quaternion-Based Neural Networks[END_REF].

The main limit of RNNs appear when attempting to generate longer motion sequences is pose collapse, which translates in the network progressively generating the same pose over and over. To reduce the problem, following approaches suggest feeding the training network with either noisy poses [Fra+15; XLM19; KGB19] its own predictions [START_REF] Li | Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis[END_REF][START_REF] Gopalakrishnan | A Neural Temporal Model for Human Motion Prediction[END_REF]. Graph Convolutional Networks Later, some others leveraged the hierarchical structure of the skeleton through (GCN). In these approaches the skeleton is represented as a graph rather than a kinematic tree, and the weights of its adjacency matrix are learned by the network [Büt+17; Mao+19; CSY20; MLS20; Li+20].

Convolutional Neural Networks

Variational Autoencoders Generative models focus on modelling the underlying distribution of a dataset, in order to be able to draw new samples from it and generate unseen data. Among this family, VAEs were the first to be applied to animation data [Hab+17; Yan+18; Du+19; Ali+20]. Much like autoencoders, VAEs learn to nonlinearly map data samples to and from a latent representation. They are in addition trained to constrain the latent samples to be part of a pre-defined distribution, which provides them their generative power: drawing samples from the known distribution only yields valid motion, similar to but not exactly part of the distribution of the training data.

Adversarial approaches Next, a series of work took interest generating diverse motion using another family of generative models: GANs [BKL18; LA18; Wan+20]. In this case, a generator model is trained to produce a new motion sample from a random input. In parallel, a discriminator network is trained, whose task is to distinguish real motion sequences and generated ones. During training, the generator's goal is to fool the discriminator into classifying its generated sequences as real ones, and the quality and variety of the generation improves.

The general idea of using a discriminator to increase the variety of the generated samples has been applied to all kind of other architectures, often to alleviate the lack of encoding capacity of vanilla GANs [KGB19; WCX21; HGM19]. Ensuring that neither the generator nor the discriminator outperform the other during training is however a delicate task, which prevented more general adoption of the idea.

Attention and Transformers

Another class of approaches make use of attention mechanisms in their networks' architecture to allow them to learn to focus on smaller but more important patterns in the skeleton data [START_REF] Mao | History Repeats Itself: Human Motion Prediction via Motion Attention[END_REF][START_REF] Zang | Few-Shot Human Motion Prediction via Learning Novel Motion Dynamics[END_REF]. This method has also recently been generalized for sequences using the popular Transformer architecture

Motion style

Humans are very sensitive to the way each other act, and we are able to perceive a lot of information on the internal state of an individual simply from observing the way he or she moves [BS07]. Conversely, we are also able to tell right away if a virtual character's motion is not "right", although explaining why it is not so might not be that easy [START_REF] Mori | The Uncanny Valley [From the Field[END_REF].

The methods described in the previous sections facilitate and speed up the creation of character animation. However, producing motion that can convey to spectators the same kind of information about the character's personality, intents or emotional state as a real person's movement remains a daunting task. Crafting such animation require precise knowledge in physiology and anatomy on the animator's side, so tools to help them in the process have been studied. In this section, we are interested on methods that rely on the metaphor of motion style as a creative interface.

Nuances in the ways characters (and especially humans) move, and their relations with how we perceive them has been studied extensively, but no consensus was found on a way to categorize them. Thus, we first summarize the different definitions of "style" used in the computer animation community at large. We then expand on the various systems that have been proposed to analyze and quantify motion and its style, and finally review the actual style-based motion synthesis and editing found in the literature.

Definition(s)

The notion of motion style, while commonly used throughout the literature, does not always have the same definition [START_REF] Badler | Digital Representations of Human Movement[END_REF][START_REF] Gallaher | Individual Differences in Nonverbal Behavior: Dimensions of Style[END_REF]. A recent review by Ribet et al. [START_REF] Ribet | Survey on Style in 3D Human Body Motion: Taxonomy, Data, Recognition and Its Applications[END_REF] acknowledges this observation and goes over the literature, proposing a taxonomy of the definitions they find. They separate them into three larger categories: style as a component of motion, style as a variation of motion, and style as individual-related features.

In the first definition, style is considered as a secondary motion which can be applied on top of a primary one [MC90; EA16]. Among this group, some also consider the existence of a "neutral animation" over which any style could be layered [ABC96; Cre+20].

In the second, style is considered as a variation around a general motion theme, although the concept of variation itself is also vaguely defined. Some consider related actions to be different styles in the same theme (i.e. walking, running, strutting are all style variations of the same theme, which could be "locomotion") [BH00; CL06]. Some generalize the variations to differences in speed or frequency [START_REF] Ma | Modeling Style and Variation in Human Motion[END_REF]. Finally, some others see variations in styles as interpersonal differences, encoding an individual's specific way of moving [Urt+04; WFH07; TMD12].

The last definition, which is perhaps the most common one, is to define styles as discrete semantic features pertaining to an individual's characteristics [Bar+13; Hol+17; AYB17; Cre+17; KL19]. The review further separates the considered features in several categories:

• Emotions ("happy", "sad", "angry", etc.)3 ;

• Biological features (age, gender);

• Physical states ("weak", "strong", "injured");

• Personality features ("cool", "confident", "stressed")

• Behaviors ("catwalk", "childlike", "zombie")

Analysis and quantification

Creating systems to accurately describe, document and quantify human motion has long been a topic of interest, with early work appearing in the fields of anthropology [START_REF] Ray | Background to Kinesics[END_REF], psychology [START_REF] Ekman | The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding[END_REF][START_REF] Mcneill | Hand and Mind: What Gestures Reveal about Thought[END_REF] and linguistics [START_REF] Kendon | Gesticulation and Speech: Two Aspects of the Process of Utterance[END_REF]. Motion notation systems are also interesting to the computer animation community as they can provide meaningful abstractions for the notion of motion style. In the following section, we review the most commonly used ones in the literature.

Laban Movement Analysis (LMA)

The LMA system, developed by dance theorist R. Laban [START_REF] Von | The Mastery of Movement[END_REF], is widely used by dancers, choreographs and actors, but also computer scientists and roboticists. It corresponds to a framework in which motion is described in four separate categories, each category depending on the others. A complete description of LMA is the subject of complete books, in this thesis we merely present a short overview [START_REF] Groff | Laban Movement Analysis: Charting the Ineffable Domain of Human Movement[END_REF]. The categories are as follows:

Body. Relates to structural and physical characteristics of the body. It describes the body parts that are moving, which are in contact, or which are influenced by others. It also describes if and when the body parts are moving simultaneously or sequentially.

Effort. Pertains to the dynamic changes in the expression of movement. It is further separated in four qualitative elements: space (direct or indirect), weight (light or strong), time (quick or sustained), and flow (free or bound). The combination of these elements creates "states" which can be used to describe the instant of change, the rhythmic variations of motion.

Shape. Represents the overall shape by taken the body (such as curvatures and symmetries) and the changes of this shape over time Space. Describes spatial patterns and trajectories, as well as how the body occupies its surrounding space

To better describe the analyzed motion, a notation system, dubbed Labanotation, has been developed along LMA [START_REF] Hutchinson | Labanotation: The System of Analyzing and Recording Movement[END_REF].

The Arousal-Valence scale

The Arousal-valence scale (sometimes Russell's Circumplex Model (RCM) [START_REF] Aristidou | Emotion Control of Unstructured Dance Movements[END_REF]) is another model used to represent emotions, attributed to A. Russel [START_REF] Russell | A Circumplex Model of Affect[END_REF]. In this model emotions are placed on a 2-dimensional plane where the axis represent arousal and valence. Low arousal is associated with boredom, high arousal with excitement, while valence goes from sadness to happiness. Emotions can then be placed in a fuzzy manner on the plane as shown in Fig. 2.15.

Motion descriptors

A segment of the research in computer vision and computer animation has also been dedicated to quantitatively measuring motion, in order to analyze, recognize or compare motion. Early work in this direction operate on video captures and take inspiration from LMA to design quantitative features related to the shape and space qualities [START_REF] Camurri | Analysis of Expressive Gesture: The EyesWeb Expressive Gesture Processing Library[END_REF]. Other approaches rely on emotion models from the psychological field [Pel09; Glo+11], or define descriptors specific to dance motion [START_REF] Fdili | Movement Qualities as Interaction Modality[END_REF].

Some authors then leveraged the growing availability of motion capture data by designing geometric features describing the relationship between skeleton joints over time [START_REF] Müller | Efficient Content-Based Retrieval of Motion Capture Data[END_REF], or LMA-related features [HTY05; Kap+13], although no consensus has been found.

In a recent work, Larboulette and Gibet propose to solve this issue by reviewing the available computable descriptors and aggregating them, providing a useful toolbox of formalized formulas [START_REF] Larboulette | A Review of Computable Expressive Descriptors of Human Motion[END_REF].

Generating stylized motion

The many factors that make an animation belong or not belong to a style are difficult to explain, and even more difficult to reproduce during the creation process. Being able to easily generate stylized animation, and to modify the style of an existing one, would then be an interesting tool to provide to animators. It would allow users to easily experiment with different styles or to generate a wide array of variations from an existing sequence.

In this section we give an overview of the different tools following this idea proposed by the research community. The methods are separated in three categories: style transfer, style-based synthesis and style-based editing.

Style transfer

The first applications of the notion of "style" in computer animation was style transfer, i.e. the task of modifying an existing motion clip to match the style of another one, without modifying its content. This is similar to motion blending (see 2.4.2), although an added difficulty comes from the new explicit separation of aesthetic characteristics and functional role of a specific motion, which is inspired from contemporary work in related fields [TF96; TF00].

Amaya et al. [START_REF] Amaya | Emotion from Motion[END_REF] present the first approach to the task by computing a style translation as the difference between a neutral and a stylized animation. Extending this idea, some others learned to transfer the style using statistical models such as Linear Time Invariant models [START_REF] Hsu | Style Translation for Human Motion[END_REF] or Independent Component Analysis [START_REF] Shapiro | Style Components[END_REF].

In a similar approach some others operate in the spectral domain to extract dynamic characteristic of stylized motion [UAT95; YM16].

Neural networks have also been used to learn to transfer style. Early solutions train specialized networks per style, using RBF neural networks [START_REF] Seyed | Classification and Translation of Style and Affect in Human Motion Using RBF Neural Networks[END_REF] or mixtures of autoregressive models [START_REF] Xia | Realtime Style Transfer for Unlabeled Heterogeneous Human Motion[END_REF] to describe the difference between an input and an output style, that can later be used for style transfer. Inspired by successful style transfer methods for images [START_REF] Gatys | A Neural Algorithm of Artistic Style[END_REF], Holden et al. transfer the style from one motion to another by optimizing the latent representation of the output motion clip to match the Gram matrix of the input one's [START_REF] Holden | A Deep Learning Framework for Character Motion Synthesis and Editing[END_REF]. They later achieve the same result faster by replacing the optimization process by another neural network [START_REF] Holden | Fast Neural Style Transfer for Motion Data[END_REF]. A final group uses supervised setups to learn style characteristics for a semantic label, which can then be used to apply the related style to another motion sequence [Wan+18; Smi+19].

Synthesis

"Stylistic synthesis", in the context of computer animation, refer to the generation of new motion based on a style information, which could be given through examples, or higher level label values.

A first group of work extend existing interpolation approaches to match a user-provided style constraint. In their seminal work, Rose et al. [START_REF] Rose | Verbs and Adverbs: Multidimensional Motion Interpolation[END_REF] use such a method by parameterizing the interpolated motion in verb (content) and adverb (style) classes. Grochow et al. [START_REF] Grochow | Style-Based Inverse Kinematics[END_REF] train per-style GPLVMs to solve the IK problem while generating style-constrained poses. By interpolating the probability distributions of the models they are also able to mix-and-match styles to produce new ones.

Another group employ statistical modeling to produce new motion, but replace or augment the generation prior with a style variable. In this setting, both supervised and unsupervised approach have been employed. Unsupervised approaches take an example stylized motion and extrapolate the available data to produce new sequences in the same style. To achieve this, Brand et al. [START_REF] Brand | Style Machines[END_REF] first describe a learned state machine using Hidden Markov Models, in which motion sequences are organized by shared style characteristics. During synthesis, a style variable vector input can be used to vary the style of the generated motion. This approach was then declined a number of variations, using GPLVMs [START_REF] Wang | Multifactor Gaussian Process Models for Style-Content Separation[END_REF], PCA [START_REF] Urtasun | Style-Based Motion Synthesis[END_REF], or CRBMs [START_REF] Taylor | Factored Conditional Restricted Boltzmann Machines for Modeling Motion Style[END_REF]. More recently, VAEs have used for their ability to model the probabilistic nature of human motion and to generate large variation of motion in the same style [START_REF] Du | Stylistic Locomotion Modeling and Synthesis Using Variational Generative Models[END_REF] The supervised approach instead generate motion from a higher-level style label which is learned from previously labeled dataset. Torresani et al. [START_REF] Torresani | Learning Motion Style Synthesis from Perceptual Observations[END_REF] learn a multidimensional style space using hand-labeled Laban notations, then a mapping from this space to animation parameters. In the same spirit Min et al. [START_REF] Min | Synthesis and Editing of Personalized Stylistic Human Motion[END_REF] use multilinear analysis to model in two dimensions, "identity" and "style" after PCA reduction, allowing them to synthesize new parameterized motion. Generative probabilistic neural networks such as invertible flows have also been used [START_REF] Wen | Autoregressive Stylized Motion Synthesis With Generative Flow[END_REF] and demonstrated their ability to model the style of a given input and generate variations on its theme.

Finally, taking the style metaphor from a biomechanical simulation point of view, Liu et al. [START_REF] Liu | Learning Physics-Based Motion Style with Nonlinear Inverse Optimization[END_REF] have tackled problem by defining style in terms of muscle preferences and musculoskeletal parameters, which are constrained during the simulation to generate stylized motion.

Editing

The last group of work gathers methods that provide animators with tools to manipulate motion through style parameters. The idea, first formulated by Bishko in 1992 [START_REF] Bishko | Relationships between Laban Movement Analysis and Computer Animation[END_REF], revolves around the observation that the previously described notation systems (Laban's in this context) are parametric, and so are computer animations, and so a mapping between them could be found. As she puts it: "It is possible to create animation software based on these parameters. This would create a metaphor of the dancer's movement processes, allowing animators a closer relationship to the physical feeling of movement while maintaining the kinematic process that is the animator's craft".

The initial implementation of this idea by Chi et al. [START_REF] Chi | The EMOTE Model for Effort and Shape[END_REF] thus hand-craft mappings from Laban Effort and Shape parameters to animation parameters, and show that users can manipulate them to edit animation. This approach has then been extended, first with a natural language interface [ZCB00], then by replacing the arbitrary mappings with learned ones [START_REF] Seyed | Modeling and Transformation of 3D Human Motion[END_REF]. Durupinar et al. [START_REF] Durupinar | PERFORM: Perceptual Approach for Adding OCEAN Personality to Human Motion Using Laban Movement Analysis[END_REF] for example label a motion dataset with Laban notations as well as OCEAN personality traits through a user study, then learn the mappings from OCEAN to Laban, then to Laban to motion parameters. This way, users can edit animation through empirical "personality" knobs4 . In the same spirit, Aristidou et al. [ACC15; Ari+17] learn a bidirectional mapping from Laban features to the Arousal-Valence scale, then describe an editing framework in which a user can modify a motion clip by manipulating the emotions on the scale.

Conclusion

Creating high-quality, appealing motion is a complex task. Animating a character according to one's vision, while depicting the desired action and conveying the right emotion requires an advanced knowledge of anatomy and physics rules. The literature depicts a wide variety of approaches to create and edit motion data, ranging from precise, user-controlled skeleton manipulation, to automatic generation by extrapolating a database.

Learning-based approaches have received their fair share of interest, and deep learning methods in particular have shown impressive potential at modelling animation data. However, as shown by this review of the literature, they have until now been applied to high-level tasks, and their output is difficult to control from a user point of view. This is the limitation that this thesis focuses upon: our goal is to leverage the powerful modelling capacities of neural network in the context of expressive and controllable animation tools.

Introduction

As highlighted in our review of the literature, neural networks have recently proven capable of modeling the complexity of motion data, and of synthesizing convincing new animation by generalizing from a database of examples. They have been successfully used to predict missing frames in motion clips, to generate locomotion animation from a high level control input, or to retarget an animation to a new skeleton. These methods produce impressive results, however they share the common drawback of being difficult to control. Animation is a precise craft, and animators expect to be able to tweak the output of their tools to fit their artistic intent. And in these existing applications, said output is pretty much out of their hands: if the generated motion does not meet their expectations, the only solution is to record some more motion capture data, add it to the dataset, re-train the models and cross fingers in hope that the desired behavior will appear.

Leveraging the subtle information learned by the networks to provide animators with smarter tools while leaving them in control therefore appears like an interesting venue for research. This observation motivates the work presented in this thesis, which proposes to explore the application space for such neural networks-powered animation editing tools. Among the different building blocks of the key-frame animation pipeline, we focus on pose editing.

A large part of the difficulty in posing a character comes from the required understanding of its motion range, possibilities and limitations, which are difficult to grasp and replicate. This information is however inherently contained in motion capture data. We therefore propose a group of methods for data-driven pose editing with the goal of shifting more of the task of assessing realism from the artist to the computer, and to provide easier access to non-experts.

The general idea of this thesis is to rely on neural networks to automatically learn the constraints from MoCap data. In this chapter, we first give a general overview of the system (3.2), which creates a latent space on which poses can be mapped for edition, while the handling of subtle constraints is handled by the mapping functions. In section 3.3 we present the pose dataset used to train our neural networks, and discuss our choices in data representation. Finally, section 3.4 describe two different approaches in the construction of the latent pose editing space, and the network architectures used in both cases. A character pose in animation data represents the parameterization of the character's underlying skeleton at a given frame. Formally, the set of possible poses X for a given skeleton is a subset of the space of skeleton parameterizations S = R J×P , with J the Chapter 3. A latent space for pose editing 29 number of joints making up the skeleton and P the number of values parameterizing a single joint (i.e. position, rotation, scale, etc. The choice of P is discussed in the next section). Any skeleton configuration in S does not correspond to a visually correct or even plausible pose. Those are only produced by parameterizations that respect a set of constraints. Some of said constraints are straightforward, such as coherent bone lengths and limits on joints orientations, but some others are not, for example the sense of equilibrium expressed by a pose, the intent and emotions it gives off. These constraints are what makes pose editing difficult: in order to manipulate a pose while remaining in the realm of possible poses, one must specify and respect them.

System overview

Much like some other approaches from the data-driven family, we attempt to circumvent the problem by learning the implicit constraints directly from ground-truth data. A high-level illustration of our system can be seen in Fig. 3.1. We construct an alternative pose editing space, Z = R d (d is a hyperparameter designing the dimension of a pose's latent representation). Two transformation functions are also defined to map samples from one space to the other: the Encoder E : X → Z and Decoder D : X → Z. We also introduce M t : Z → Z, a manipulator function which modify a pose to satisfy a given user-provided target t, and operates on latent samples. For the remaining of this thesis the three functions are modeled as neural networks whose parameters will be trained to fit our need.

Within this system, the implicit skeleton constraints may be enforced in one of two ways. In the first, the latent space is shaped in a specific way, so that E maps any latent sample to a possible pose. In the second, its shape is not enforced at all. Instead, M learns to navigate Z to only generate samples that will map to possible poses.

This chapter discusses the matter of designing E and D, and thus constructing the latent space, to accommodate both those ways. The specific design of M is explored in further chapters.

The pose dataset

In order to train the different networks, we build a large dataset of poses, obtained by aggregating multiple existing motion capture databases. This section describes these sources, the pre-processing step used to uniformize all their animation clips, then discuss the format in which the poses are fed to the networks.

Pose representation for neural networks

The choice of data representation plays an important role in the success of neural networks methods, and different representations can hide or reveal important factors of variation in the data [START_REF] Bengio | Representation Learning: A Review and New Perspectives[END_REF]. Animation data can be represented in many ways and the question of which is best suited to train neural network is a standing issue for the research community. One part of the differences lies within the pose representation (parameterized by J and P in our notation), which we aim to model, and so for which we need to make a choice. We first give an overview of the options used in the literature, discussing the strong and weak points of each. We then discuss the reasons behind our pick and describe the input format used in the work that follows.

Literature

Pose data is most commonly represented as a hierarchy of joints angles, where each joint is parameterized by its orientation relative to its parent. Being the representation of choice in the traditional animation pipeline, it might seem a natural pick for neural networks tools, as it would make their integration in existing workflows straightforward. It is however not the case, mostly due to individual rotations raising issues when it comes to modeling, and multiple representations of rotations themselves have been studied.

The first problem is a well-known one: all rotation representations parameterized by 3 components (such as Euler or axis-angle) are subject to a singularity called the gimbal lock, "loosing" a degree of freedom if two of their rotation planes align. To avoid it, early work used exponential maps representations where the singularities only happen with larger angles [START_REF] Taylor | Modeling Human Motion Using Binary Latent Variables[END_REF]. This approach was fit for the modelling of smaller angles, and was used to produce motion by generating pose offsets, but it is no longer a viable strategy for longer-horizon modelling. In order to avoid the problem, unit quaternions have been used in place [START_REF] Pavllo | QuaterNet: A Quaternionbased Recurrent Model for Human Motion[END_REF].

It has however been pointed out that quaternions, and all the previously used rotations representations, are subject to a second problem: they are non-continuous, in the sense that two parameterizations can amount to the same angle (i.e. 0 is equivalent to 2π) [START_REF] Grassia | Practical Parameterization of Rotations Using the Exponential Map[END_REF][START_REF] Saxena | Learning 3-D Object Orientation from Images[END_REF]. This is a specifically an issue when it comes to neural network modelling, as the discontinuities generate erroneous comparisons between close values. A recent study has highlighted this shortcoming, and proposed to avoid it by introducing new 5 and 6 dimensions representations [START_REF] Zhou | On the Continuity of Rotation Representations in Neural Networks[END_REF], but its usage is not yet standard in the field.

The hierarchical aspect of angular representations is also limiting, as multiple angular errors in the kinematic chain accumulate up to large positional errors down the chains (Fig. 3.2). This problem has been tackled by computing the positional errors after finding the joints position through an expensive (in terms of computation during training) forward kinematics pass, or by weighting the loss terms of the joints to give a greater penalty to errors close to the root of the chain [START_REF] Ghorbani | Probabilistic Character Motion Synthesis Using a Hierarchical Deep Latent Variable Model[END_REF]. The alternative option is to represent the pose using each joint's position, usually relative to the skeleton's root bone. Compared to angular representations, positions do not suffer from error accumulation, and the Cartesian coordinate system is continuous and free from singularities. However, positional representations do not ensure constant bone lengths the way angular ones do.

Since it is possible to convert from one parameterization to the other, and since both offer different flaws, none is strictly superior to the other. Rotations are closer to the format used in actual animation systems, while positions have historically been chosen for modelling, as the bone length difference can be alleviated at a low cost through post-processing. Some work has also been dedicated to hybrid or redundant representations [Abe+20; LLL18; Hol+20] and tend to indicate that these would be beneficial, but no definitive formulation has been agreed upon.

Input format for our neural networks

Considering these differences and limitations, and the fact that this thesis' goal is to explore new use cases for neural networks in animation rather than to improve their modeling power, we stick to positional representations, which allows us to build upon previous modelling work more easily. To accommodate for inevitable bone length errors, we suggest a simple work-around to the problem, see 3.3.3. However and fortunately, since our approach considers neural networks as generic tools for animation, it can easily be adapted to accommodate newer architectures and representations once they have stabilized and reached their full potential.

Our input skeleton, visible in Fig. 3.3, is composed of 21 joints. The pose tensor x representing its parameterization which is fed to the networks is obtained by concatenating the position of each joint in pose space, i.e. related to the pelvis' one.

In other words, our choice of pose representations uses J = 21, P = 3, and X = R 63 .

Source animation datasets

The success of data-oriented animation techniques is heavily reliant on the availability of large amounts of high-quality motion data. Fortunately, with the improvements brought to motion capture technology making it more and more accessible, such data is no longer as scarce as it used to be, and animation researchers can count on a large choice of motion datasets freely available online [Uni03; Ion+14; FP14; Vol+14; Mah+19; Har+20].

We build the pose dataset by processing a subset of these. While a larger dataset would undoubtably increase the performance of our models [START_REF] Sun | Revisiting Unreasonable Effectiveness of Data in Deep Learning Era[END_REF], the goal of this thesis is not to pursue a state-of-the-art value, but rather to explore different Chapter 3. A latent space for pose editing 32 application spaces for deep learning models. With this in mind, we restrain ourselves to a large-enough dataset of 1,5 million poses which we find sufficient to support our arguments, with the added benefits of lowering the training times of our models. Our source databases are: the CMU MoCap Database [Uni03], Emilya [START_REF] Fourati | Emilya: Emotional Body Expression in Daily Actions Database[END_REF], and the clips from the Edinburgh University [START_REF] Holden | A Deep Learning Framework for Character Motion Synthesis and Editing[END_REF].

The Carnegie Mellon University Graphics Lab Motion Capture Database (CMU) [START_REF]CMU Graphics Lab Motion Capture Database[END_REF] is perhaps the most commonly used motion dataset available. It contains 2605 motion recordings, acted by 144 subjects, over a large variety of actions, from simple locomotion samples to sports and multi-subjects interactions, which amount to roughly 10 hours of footage.

The Emilya database [START_REF] Fourati | Emilya: Emotional Body Expression in Daily Actions Database[END_REF] is interested in how emotions are expressed in daily actions. It contains recorded motion clips during which 11 actors and actresses were asked to perform 7 "every day" actions while acting 8 emotions, for a total of about 25 hours of motion capture data. The affective states where induced by reading different scenarios to the performers.

The Edinburgh Locomotion dataset [START_REF] Holden | A Deep Learning Framework for Character Motion Synthesis and Editing[END_REF] is focused on longer, uninterrupted locomotion clips, and was designed specifically for deep learning applications. Actors were asked to move around randomly on the capture set at different speed in order to record more variations of natural motion. It contains less data than the other two (about 2 hours worth of motion capture), but contains more random and continuous motion.

Preprocessing

The different source database present differences in the data they contain, and in order to create the dataset which we can then use to train neural networks, we pre-process them to bring them to a common standard. This procedure is inspired from the format conversion presented by Holden et al. [START_REF] Holden | A Deep Learning Framework for Character Motion Synthesis and Editing[END_REF], although we adapt some of its step to fit our needs.

Each animation clip is retargeted to a standard target skeleton following the scheme proposed by Holden et al. [START_REF] Holden | Learning Motion Manifolds with Convolutional Autoencoders[END_REF]. First, the joint orientations of the source skeleton are copied to the target one for each matching joints. Then, the source skeleton is scaled to match the target's bone lengths. Finally, the joints of the target skeleton are moved to match the new position of the source's one with a full-body IK pass [START_REF] Yamane | Natural Motion Animation through Constraining and Deconstraining at Will[END_REF].

We then extract singular poses from the clips. For each frame, the global translation is removed, and each joint's position is calculated relative to the pelvis (root) joint. We find the forward direction of each pose by computing the cross product of the vertical axis with the average of vectors formed by the left-right shoulders and hips axis, then rotate them so that they all face the same direction. We also sub-sample each clip to 30 frames per second, to avoid filling the dataset with highly similar poses and speed up training times. The parameterizations of each joint are then normalized by subtracting the mean and dividing by the standard deviation of their value over the dataset, as is commonly done to prepare data before feeding it to neural networks, then concatenated to form the final input tensors.

It is also worth noting that although the dataset is composed of individual poses, it is not yet randomized, and neighboring poses from the same original clip are kept close together. This feature is leveraged later in this thesis when we require similar poses in terms of feasible motion from one to the other. For the same reason, very short clips (below one seconds) are excluded, as are noisy ones and those with little motion. After this cleaning step, the final pose dataset is composed of about 1,5 million poses.

Post-processing

It is a common observation with neural networks working with joints position that the generated positions can be jittery, and the resulting poses can suffer from slight variations in bone lengths. Our models are no exception, and while the variation is not visually detectable most of the time, they can accumulate to be quite noticeable when a pose generated by the networks is fed back to them. We propose an optional post-processing step which can be applied to generated poses to ensure constant bone lengths. We use the backward step from the FABRIK IK solver [START_REF] Aristidou | FABRIK: A Fast, Iterative Solver for the Inverse Kinematics Problem[END_REF], which iteratively pulls back each joint of the skeleton's kinematic chains with a very light computation cost. In our implementation using the kinematic chains illustrated in Fig. 3.3, this post-processing step takes only 1.5 milliseconds.

The algorithm for one of the skeleton's kinematic chain is outlined in 1. The joints are denoted p n , with p 0 the root and p n the end-effector. d i represents the original length of the bone between p i and p i+1 , initialized once with the reference skeleton.

Algorithm 1 The FABRIK backward pass [AL11]

for i = 1, . . . , n -1 do r i ← |p i+1 -p i | {Find the new bone length between p i and p i+1 } λ i ← d i /r i p i+1 ← (1 -λ i )p i + λ i p i+1 {"
Pull" p i+1 back to honor bone length} end for

Architectures

We study two approaches for the design of the latent pose space. In the first the latent space is constructed without constraints by a baseline autoencoder, which serves the role of the E and D functions. In the second, we instead force certain properties onto the latent space, by using a scaffolding of autoencoder and GANs. For both cases, we describe the neural networks architectures and the training parameters used in further experiments.

Baseline autoencoder

The idea of building a latent space for complex data is not new, and so for a baseline experiment, upon which we can base further improvements later on, we turn to a staple in the literature: autoencoders. Autoencoders are made up of two neural networks tasked to learn an efficient encoding of some complex data. As illustrated in Fig. 3.4, the encoder E learns to map real data points to a learned, usually more compact, latent space; and the decoder D learns to map them back to the original data space. As this architecture closely matches our previously described system, autoencoders are a natural choice. In their original flavor, they also have the advantage of being conceptually simple and easy to train, which are interesting properties with regard to our goal of designing accessible tools. On another hand however, the latent space they produce is unknown and might not be suited for some applications. For example, it might not be convex and therefore interpolations within it would not be decoded in plausible poses. For the same reason, they lack the generative power some other architectures have.

In our implementation, illustrated in Fig. 3.5 the encoder network is composed of two fully connected layers with 200 neurons and Rectified Linear Unit (ReLU) [NH10] activations, followed by an output layer with no activation. The output layer's size is based on the number of dimensions d in which the latent representations are encoded.

We empirically find that d = 64 yields a good balance of representation accuracy and inference speed. The decoder is the exact reversed replica and uses the same set of weights. The encoder and decoder's parameters, respectively θ and ϕ, are found by minimizing the squared error between poses sampled from the dataset x and their reconstructed equivalent, as shown in Eq. 3.1. Note that with our networks having a low number of units, we found no benefits to adding a sparcity penalty term to limit the number of neurons active at the same time.

min

θ,ϕ L ae (θ, ϕ) = E x∼p data x -D E(x) 2 2
(3.1)

The autoencoder is trained for 20 epochs with batches of 256 poses, using the Adam optimizer [START_REF] Diederik | Adam: A Method for Stochastic Optimization[END_REF] with a learning rate of 0.0001.

Autoencoding Generative Adversarial Network (AEGAN)

Overview

In our second experiment, we turn to more complex architectures in order to get rid of some limitations our baseline latent space presented. More specifically, we seek to ensure that the built latent space is convex, so that any modification made to a latent pose maps back to another plausible one. Such a property would come with additional advantages in an editing setup, such as coherent interpolation and being able to generate plausible poses by drawing random samples from the known latent distribution.

Unfortunately, none of the major approaches to generative modelling using neural networks fit our specifications: autoencoders' latent spaces are unconstrained and GANs [START_REF] Goodfellow | Generative Adversarial Networks[END_REF],only learn to generate new data similar to the training one, lacking the possibility to encode existing samples, which is required in an editing setup. VAEs [START_REF] Diederik | Auto-Encoding Variational Bayes[END_REF] would be good candidates, however, they are known to struggle to generate highly precise samples [START_REF] Zhao | Towards Deeper Understanding of Variational Autoencoding Models[END_REF].

We thus turn to a hybrid approach, similar to the one proposed by Lazarou [START_REF] Lazarou | Autoencoding Generative Adversarial Networks[END_REF] to create a latent space of images in which they illustrate good interpolation properties. Their architecture, a scaffolding of autoencoders and GANs, is dubbed Autoencoding Generative Adversarial Network (AEGAN). We adapt their work to operate on pose data rather than images. The method uses four separate networks, illustrated in Fig. 3.6: an encoder E, a generator G, a pose discriminator DP and a latent vector discriminator DL. The encoder and decoder are trained to map from pose to latent space and back, and use feedback from the discriminators to shape the desired latent space.

Networks architectures

The skeleton is a hierarchical structure made of several kinematic chains, and each joint's parameterization has an impact on other joints down the chain. In this more complex approach, we also attempt to explicitly model this spacial structure by incorporating the Structured Prediction Layers (SPL) proposed by Aksan et al. [START_REF] Aksan | Structured Prediction Helps 3D Human Motion Modelling[END_REF] in place of fully-connected layers. SPL splits dense connections in multiple smaller layers, connected themselves following the kinematic chains of the skeleton, as illustrated in Fig. 3.7. The encoder network is composed of a single SPL layer of 252 neurons (12 neuron per joint embedding) and ReLU activations, followed by a fully connected layer with 32 neurons and a hyperbolic tangent (tanh) function. Instead of its usual role of inducing nonlinearity in the network, the output tanh function restricts the latent vectors' values to [-1, 1], so that the bounds within which a random one can be drawn and decoded in a generative fashion is known. The decoder is the opposite with a fully-connected layer with 252 neurons, ReLU activations, and an SPL layer with 63 outputs units.

The pose discriminator is the same as the encoder except for the last activation which is a Sigmoid function. The latent discriminator is a multilayer perceptron with 4 fully connected layers of 256 neurons with ReLU activations. The last activation is once again a Sigmoid function, which is used to clamp the discriminators' outputs to [0, 1] so that it can be interpreted as a "real or generated" label.

Loss functions

The pose discriminator DP outputs a single scalar representing the probability for a given pose to be "real" over being generated. During training, its weights ψ are optimized to correctly classify raw samples from the pose dataset as well as ones generated from randomly drawn latent vectors. This is achieved by maximizing the loss function shown in Eq. 3.2.

max ψ L DP (ψ) = E x∼p data log DP (x) + E z∼pz log 1 -DP G(z) (3.2)
In the same manner, DL discriminates between random latent samples and encoded poses. Its parameters γ are found by maximizing Eq. 3.3.

max γ L DL (γ) = E x∼p data log 1 -DL E(x) + E z∼pz log DL(z) (3.3)
The convexity of the latent space is dictated by the interaction between the discriminators and the autoencoder. As G tries to fool DP , it is encouraged to turn any point from the latent space in a valid pose. At the same time the latent discriminator encourages the encoder to encode poses into latent vectors indistinguishable from random ones, and so to use the latent space to its full extent. We therefore build a convex latent space in which the real poses are evenly spread. E and G are thus trained conjointly to achieved two objectives: to reconstruct a pose with fidelity and to fool the discriminators. These objectives are translated in a three-terms loss function: Eq. 3.4.

min θ,ϕ L ae (θ, ϕ) = L recon (θ, ϕ) + L adv (θ, ϕ) (3.4)
The first term is the reconstruction loss: Eq. 3.5.

min θ,ϕ L recon (θ, ϕ) = E x∼p data x -G E(x) 2 2 + E z∼pz z -E G(z) 2 2
(3.5)

It is built similarly to the simple autoencoder's one, which minimizes the reconstruction error of an encoded-then-decoded pose. However, the network must also ensure that a randomly drawn latent vector can be decoded then re-encoded to an identical one which is enforced by its second term. Note that in this form, the loss is similar to the cycle consistency loss used in CycleGAN architectures [START_REF] Zhu | Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks[END_REF], with the difference that the consistency is enforced between the known pose domain and a latent one, rather than two known image domains.

The second term is the adversarial loss (Eq. 3.6), whose terms represent the objectives with regard to the discriminators: (1) and ( 2

L adv (θ, ϕ) = E z∼pz log DP G(z) (1) + E x∼p data log DP G E(x) (3) + E x∼p data log DL E(x) (2) + E z∼pz log DL E G(z) (4) (3.6)

Training parameters

The networks are trained for 200 epochs with a batch size of 256 poses. We use the Adam optimizer [START_REF] Diederik | Adam: A Method for Stochastic Optimization[END_REF] with a learning rate of 0.001 for the pose discriminator and 0.0002 for the encoder, decoder and latent discriminator. In order to regularize the performance of each networks during the training process, the encoder-decoder and latent discriminator are trained with respectively two and three times as many samples as the pose discriminator.

Comparisons

The presented architectures share the common goal of constructing a latent pose space but differ in their approach to the problem. This section discusses the stronger and weaker aspect of both.

The first difference lies in the desired shape and properties of the latent space. To observe this difference in structure, we conduct an experiment in which we encode 5000 real poses from the dataset and visualize them on a 2D plane using t-Distributed Stochastic Neighbor Embedding (t-SNE [HR02; MH08]). To evaluate the convexity of the spaces, the same amount of random latent points are drawn from a uniform distribution with the same bounds as each space. In order to give a comparison point, the real pose samples are also visualized, along with random equivalents. The results are shown in Fig. 3.8. The experiment in the pose space illustrates the difference between the skeleton parameterization space S and its constrained subspace of believable poses X . Random points are drawn from S and therefore do not necessarily satisfy the constraints that make them par of X . As a result, in the visualization, they are clearly separated from encoded poses, which are part of X .

In the autoencoder's space, the same separation is visible, although less clear. This is expected, since the model does not enforce any structure on its latent space. We can however observe that sequences of poses, forming animation clips, are still clustered together.

Finally, in AEGAN's space, the separation is blurred. While some existing poses are still encoded on the edges of the space, random and encoded poses are for the most part undifferentiated.

While considering the structural properties of the latent space only, AEGAN has a clear advantage, and it could seem like a straightforward choice for any application. However, this structures comes at a cost: the presence of multiple discriminators make training AEGAN significantly more complicated than training a baseline autoencoder.

It is necessary to balance the performance of the generator as well as those of the discriminators, so that none of them become too efficient at "beating" the other.

Regularizing the training is a known difficulty of adversarial-based approaches, and one that has prevented their general adoption. For the same reason, the training times for AEGAN that for the autoencoder (About 4 hours for AEGAN against 15 minutes for the AE, during our experiment on the same hardware).

Conclusion

This chapter gives an overview of the general method explored in this thesis. Neural networks are used to train a latent pose space, which can then be used for pose editing. By learning functions that map existing pose samples to and from the space, the networks learn the subtleties of the constraints that a skeleton configuration must respect in order to be able to be considered an actual character pose. Two architectures are described, with different expectations regarding the latent space properties: a simple autoencoder, AE and a more complex one, AEGAN, which incorporate two discriminators to enforce the constructed latent space's structure.

Both approaches are capable of reaching their goal. They however display different advantages and drawbacks: AEGAN's latent space is smooth and convex, which means that it can be used to interpolate between samples without the risk of considering an improper pose. In contrast, AE's latent space does not have such safeguards, and exploring it requires caution. Obtaining these interesting properties come at the cost of a more complex and lengthy training phase. As a result, selecting one or the other in further applications mostly depends on the benefits of a more structured latent space in the considered use-case. In cases where such benefits are not obvious, then the simplicity of the AE might be a better solution.

Introduction

Once the latent space has been constructed in one of the two ways depicted in the previous chapter, we turn to exploring its capabilities. The first considered application is full-body Inverse Kinematics solvers.

Full-body IK is an extension of the original IK problem in which a character's skeleton in its entirety is considered, instead of a single kinematic chain. The goal is thus to find a parameterization of the skeleton where one or more joints are closest to Chapter 4. Full-body Inverse Kinematics in the latent space 41 user-provided target positions or orientations, while respecting the pose constraints. Generic IK solvers focus on enforcing the bone lengths and (in some cases) joint orientation limits. Pose constraints are more subtle in nature, and enforcing them is the most difficult part of the task: in a real-world scenario, the constraints are hand-crafted by the animators for each new character.

This is where the latent framework proves useful: since we made sure that the decoding function produces poses similar to the training dataset, solving the problem in the latent space ensures that the skeleton constraints are respected, without having to explicitly enforce them.

This chapter discusses two methods leveraging the latent space for full-body Inverse Kinematics. The first makes uses of the constrained, convex latent space to solve user-provided constraints through optimization. The second trains a neural network to navigate the unconstrained latent space for the same purpose.

Optimization in the latent space

Method

In our first approach, we propose an optimization-based solution in a known latent space to solve the IK problem. The general idea, illustrated in Fig. 4.1, is to gradually modify the latent vector representing the original pose to satisfy the user-provided target. Since the latent representation lies on a convex latent space (built with the AEGAN architecture), any and all solutions considered during the optimization process maps back to a valid pose.

L ik (x, t) = n i=0 (x i -j i ) 2 (4.1) Algorithm 2 Latent space optimization z ← E(x) % Encode i ← 0 while λ ≥ ϵ and i ≤ max_iter do i ← i + 1 z ← O(z) % Optimize x ′ ← D(z) % Decode λ ← L ik (x ′ , t) end while return x ′
To ensure constant maximum time, the process can be stopped when a maximum number of iteration (max_iter) is reached. In this setting, our contribution is more largely the construction of a smooth latent space, in which many existing optimization methods can be used. In the later experiments O uses stochastic gradient descent (SGD), but could easily be swapped for another. In order to evaluate the results of our method we integrate our solver in an example posing software and compare its outcome with a comparable, non-neural method: FABRIK [START_REF] Aristidou | FABRIK: A Fast, Iterative Solver for the Inverse Kinematics Problem[END_REF]. We pick FABRIK for the traits that make it a popular IK solver: its simplicity and fast convergence times. We implement a full-body human skeleton solver as described in [ACL16] but stay as close as possible to our method setup process by not manually implementing any joint orient constraints. FABRIK working on kinematics chains with no prior on the human skeleton, it may end up with unrealistic poses, whereas our optimization process exploring the latent space results in poses satisfying the constraints without breaking the implicit skeleton rules. Just like with the other IK methods, the distance between limbs is constant but additionally self-occlusion is avoided, and the poses appear natural. In (1) and (2) the skeleton leans on one side in order to reach a target above its shoulder, giving way to its arm. The legs also move slightly to appear in balance. In (3), our method makes the skeleton twists it upper body to face the two targets on its side. (4) illustrates the limits of FABRIK without specifying many constraints: when trying to reach targets on opposite sides, the shoulders move forward unnaturally. With our methods, the torso structure is implicitly learned, and the algorithm finds a more suitable solution.

Results

While the final output of the latent optimization-based IK solver are good, it does not completely fill our expectations. The main limiting factor is the execution time: even when operating on a smooth space, the optimization is an iterative process which is not guaranteed to find a satisfying solution within a bound time. In practice and using our baseline implementations, solving a single pose takes up to half a second, which is not enough to allow a user to interactively manipulate the skeleton. In our second approach, we propose to train another neural network to solve the IK problem in the latent space, replacing the optimization process. Since neural networks are capable of efficiently modeling complex data distributions, enforcing the smoothness of the latent space is not as beneficial as with the optimization-based approach. This second method therefore uses the latent space built by a simple autoencoder. This choice is additionally motivated by our goal for usability, as the training of an autoencoder is way simpler than that of AEGAN.

Learning full-body inverse kinematics solvers 4.3.1 Method

The high-level idea of the method is illustrated in Fig. 4.3: the autoencoder builds a latent pose space, and the solver model operates on this space to solve the full-body IK problem. In this section we first present the architecture and training of the solver network, then describe a methodology using multiple instances of the solver model at once to work with a varying amount of targets.

Pose solver network

An instance of the solver model S T (along with its parameters set β T ) is specialized to solve the IK problem for a set of specific targets T = {t 0 , . . . , t n }. It is trained to generate a new pose from an input pose and the desired targets locations. As it operates on the latent space built by the autoencoder, it more precisely accepts and outputs a latent pose vector, i.e. with x ′ T the concatenated target positions, ẑ = S T (z ⊕ x ′ T ). While most solvers will only be concerned with a single joint, some situations might benefit from having multiple targets linked together (for example, if attempting to move both hands in contact of an object).

The network is composed of three fully connected layers with 126 neurons and ReLU activations, and an output layer whose dimensions match the latent space's.

During training, we randomly sample an input pose x from the dataset and feed it to the network. We also sample a second pose x ′ to use as a target. Since the network will be used in an interactive fashion by the user, with at least one operation per frame, it is expected that the differences between the original positions of the effectors and their target to be relatively small. To reflect this, the target x ′ is sampled from the same source clip as x, among one of its 20 frames of the input. This association thus serves two purposes: first, ensuring that the target pose is reachable from the starting one, and second, restricting the difference between any two poses seen during training from being too wide. The network's weights β T are optimized to minimize the loss function in Eq.4.2 designed to represent its high level objective: reaching the targets with the associated joints while retaining a realistic pose. We guide the network toward this objective by using a slightly modified mean squared error function, separating each pose x in two sets of joints: x T the joints associated to the targets, and x r the others. We introduce a constant k to give more relative importance to the target term of the function, so that the non-targets joints of x ′ are only used to nudge the final result toward a plausible pose. In our experiments k is set to 0.01. The idea behind this trick is that the target pose sampled from the dataset is not an absolute truth to be reached at all cost, but should rather be considered as a guide, the main goal being reaching the targets.

x = D S E(x) ⊕ x ′ T min β T L s T (β T ) = E (x;x ′ )∼p data ∥x T -x ′ T ∥ 2 2 + k • ∥x r -x ′ r ∥ 2 2 (4.2)
In our experiments, an instance of the solver model is trained for 5 epochs with the Adam optimizer [KB15] using a learning rate of 0.0001 and a batch size of 256.

Solving other targets configurations

The solvers as presented here are designed to edit a pose while respecting only a few targets at once (one to two in our examples). In a real-world scenario however, a user might want to manipulate a character through different effectors. In cases where the effectors are used one after the other, it is possible to keep the weights set of different solvers at hand, and to switch to the corresponding one when required. Scenarios in which multiple effector targets change position during the same frame are rare in an interactive format, but can be envisaged in cases where the targets are moved in one step, and resolved in another. Multiple solvers might also be required to "pin down" a joint while moving another. In these cases, another strategy is necessary.

We propose to handle these cases by running multiple solver instances in sequence, each one using the output of its predecessor as input, i.e. for three different solvers S T 1 , S T 2 , S T 3 , each specialized for a target group T i , the resulting modified latent pose is given by ẑ = (S T 3 • S T 2 • S T 1 )(z). Since the embeddings functions are separate from the solvers, this also represents an efficiency gain, as the encoding and decoding only needs to be run once per frame. showcases an example of how our method can be used to edit a pose by moving the targets around. In this case, a single solver with the targets set to both skeleton's hands is used. We once again compare against FABRIK in the same setup as the previous section.

Results

Our solver yields poses satisfying the constraints without breaking the implicit skeleton rules: the distance between limbs is constant, self-occlusion is avoided, and the poses appear natural. The side-by-side comparisons with FABRIK's results highlight the limits of working on kinematics chains with no prior on the human skeleton.

Example (1) illustrates how the targets are used as guides rather than fixed, unbreakable rules. While FABRIK extends the full body, our solver generates a new pose where the torso is slightly twisted towards the right-hand target while the legs are spread to mimic maintaining balance. Even though our method is aimed toward beginner animators, experienced ones could also find it useful. It could for example be used as a fast prototyping tool to flesh out the pose, while switching to more accurate and manipulation-heavy tools to focus on the details later on.

Some additional results of the neural IK solver, used in an interactive context, can be seen in the accompanying video of one of our papers: https://www.youtube.com/ watch?v=h8nzECEtw3c.

Combined solvers

Figure 4.5 demonstrates an example with the multi-solver setup described in 4.3.3. In this example three solvers are used at once: one for the two hands, one for the two ankles and one for the head. Compared to the FABRIK result, our method yields a plausible pose: the skeleton is bent down to meet the head target, but the general orientation of the pose is kept intact. The limbs also retain some sort of curvature rather than fully extending unnaturally. Here again some targets are not strictly reached, as the pose generated by earlier solvers in the chain are modified by the others further down, but the guidance provided by the targets is respected. This setup also incurs slightly slower runtimes (see Table 4.1) but is still faster than FABRIK.

Run times

At run-time, the complexity of the solver is fixed and regardless of the targets' positions, a single pass through the networks, which can be seen as just a few matrix multiplications, is enough to produce a result pose. This property coupled with the relatively small size of the networks allow for a fast solving process, as highlighted in modest size of the networks: around an hour for the auto-encoder and 15 minutes for the solvers, on a single GPU.

Memory footprint

An advantage of neural networks is the low memory footprint they hold. While other data-driven pose design methods require a reference to the pose database (or a compressed version of it) to be kept in memory, neural networks only require their trained weights. These can be quite heavy as well in the case of large models, but as ours are quite small, so are their weights. As a comparison point, the Gaussian Processes-based method NAT-IK [WTR11] discloses a 30 MB memory footprint while our full-body solver only takes up 826 kB (including the weights for the autoencoder and each solver).

Comparison with other pose edition approaches

Huang et al. [START_REF] Huang | Multi-Variate Gaussian-Based Inverse Kinematics[END_REF] proposed a general comparison chart for full-body IK methods, ranking common approaches by speed and subjective quality. Adding our solution to the chart (Fig. 4.6) highlights the useful spot it fills by striking a good balance between speed and accuracy. To the best of our knowledge, this work presents the first method leveraging neural networks for pose edition. It stands apart from previous learning-based approaches as the first one to combine real-time edition speed with fully learned skeleton constraints. In comparison, NAT-IK [START_REF] Huang | Multi-Variate Gaussian-Based Inverse Kinematics[END_REF] uses soft learned constraints but still requires explicit, manual ones to be set. Style IK [START_REF] Wu | Natural Character Posing from a Large Motion Database[END_REF] does not, but the poses are not generated in real time.

Conclusion

This chapter presented two different approaches sharing a common goal: to leverage the latent pose space in the context of a full-body IK solver. In the first, latent poses representation are optimized, taking advantage of the enforced properties of AEGAN's latent space to restrict the search space. In the second, another neural networks learns to navigate the unrestricted latent space built by an autoencoder.

Both methods show similarly satisfactory results, and enable the user to easily manipulate a character skeleton. Learning from a large dataset of ground truth poses allows us to avoid manually specifying the complex constraints of the human skeleton, and to only generate plausible poses.

The methods however differ in efficiency. The optimization-based solution suffers from unbound solve times, and as a result is not suited for interactive editing. The other runs in real time thanks to the efficient inference of neural networks, at the cost of an additional training step.

The work presented in this paper was published in two conference communications:

Introduction

Approaching animation creation under the style metaphor is an interesting idea, with the potential to facilitate the production or modification of animations through the use of higher-level parameters than raw skeleton parameterization. However and as our review of the literature highlights, most of the previous efforts dedicated to this kind of interface focus on applying the style of an animation to another. In cases where style parameters are exposed to the user, the characteristics linked to the available style categories are extracted from hand-labeled data, which takes control away from the user who might have a different expectation for a given style.

Chapter 5. Learned pose manipulators for animation style editing 50 This chapter presents a method which allow users to edit a character pose using "style" constraints, by leveraging the latent pose space presented in an earlier chapter. We first discuss the limits of the commonly used approaches to motion style editing, then describe the objective pose parameters we use in place of the usual high-level style labels, and a method to learn to enforce them. Finally, we propose a method to propagate style changes made to a pose to neighboring frames to facilitate the editing of whole animation clips.

Limits of existing style editing techniques

If the definition of style as a separation from an animation's content is commonly accepted in the computer animation literature, the limits of what the term itself entails are not clearly defined themselves. Some associate an animation's style to emotions experienced by the character, such as "joy", "anger" or "sadness" [START_REF] Fourati | Emilya: Emotional Body Expression in Daily Actions Database[END_REF]. Some link it to existing motion analysis systems, such as Laban's [LU71; Dur+16], or to psychological concepts such as the arousal/valence scale [EF67; Ran+19]. Some others use arbitrary categories, such as "gorilla", "childish", "zombie", or "neutral" [START_REF] Xia | Realtime Style Transfer for Unlabeled Heterogeneous Human Motion[END_REF].

(

(3) This categorization might however be too subjective to be considered as an expressive parameter to provide to users. Indeed, various factors can introduce bias in style categories, leading editing tools based on them to fail to meet the user's expectation.

A first source of bias lies in how the data used to calibrate the style categories is recorded. It is usually not captured in the wild, but rather played by actors [START_REF] Xia | Realtime Style Transfer for Unlabeled Heterogeneous Human Motion[END_REF].

The sequences are thus often overplayed, exaggerated representations of the actor and director's visions of the categories (see Fig. 5.1). In the most grounded acquisition procedures, emotions states are elicited in the subjects by exposing them to predetermined scenario, designed to provoke a specific reaction [START_REF] Fourati | Emilya: Emotional Body Expression in Daily Actions Database[END_REF]. Eliciting emotions is however not a fail-proof process: A subject's personality and individual experience might influence their reaction when being exposed to a prepared material. Even in these settings where precautions are taken, the bias is not totally avoided [START_REF] Mccambridge | Systematic Review of the Hawthorne Effect: New Concepts Are Needed to Study Research Participation Effects[END_REF][START_REF] Henrich | The Weirdest People in the World?[END_REF].

A second source of bias appears when considering differences of expectation regarding a style's characteristics between a future user and the authors of style categories. It is most evident for "fantasy" styles representing a shared cultural reference. For example, the "zombie" style from Xia et al.'s database (Fig. 5.1) depicts a slow, disjointed walk. It is evidently a common depiction of the zombie figure in popular culture, however, it is not the only one: users editing animation using this style category might be surprised if they expected a fast and sleek zombie. The problem also appears when considering categories representing a social trait or behavior ("proud", "strutting", "childish"), built from stereotypical representations which are not universally shared. Notably, the universality of emotions is still actively discussed in other fields, with evidence pointing to the fact that the way we perceive and express emotions is influenced by our cultural specificities [EA02; Bar+19]. In the computer animation field, early research on motion style acknowledged this limitation [START_REF] Amaya | Emotion from Motion[END_REF], but following work lost sight of the nuance and often consider categories pertaining to emotional status as universal. We should remain vigilant when designing tools based on these potentially biased assessments.

One other drawback of existing style editing approaches is that it is not possible to create a new style from the ground up, as the style must be identified and present in an existing clip. In the best ones, the style can be extracted from a single existing stylized clip and applied to others, but it nevertheless needs to exist in the first place.

In a real-world scenario this might not always possible, as animators might want to design a new style on the fly, and to choose themselves the aspects of the motion that define its characteristics.

Pose style parameters

Definition

Even with the limits of style-based approaches in mind, being able to edit an animation sequence by manipulating higher-level parameters is a desirable goal. We thus propose an alternative solution which focuses on leaving the task of defining the characteristics of a style to the user. The general idea is to introduce higher-level pose parameters that can be directly computed from the pose data. Since these are objective, we do not make assumptions on the expectations of users with regard to a style. Instead, they are able to combine multiple of these parameters to fit their own view of a style.

For the sake of an example, imagine an artist editing an animation sequence of a character walking, expressing "cold rage". In his mind, the character should be taking short, fast steps, his arms tense, stretched toward the floor. With previous style edition methods, they might be able to hit an "angry" button, or maybe use a slider to determine how angry the result should be. Unfortunately, it might be that the system was calibrated with another definition of angry in mind: the character now walks faster, the arms bent and swinging, with the fists close to the torso. Using the pose parameters would allow the user to circumvent this problem. By combining a few ones such as the angle formed by the elbows, the distance between the feet, or between the hands and the floor, and editing the animation through them, we can avoid relying on assumed style characteristics. In order to demonstrate the application of our system, the following experiments use three sample metrics applicable to a human skeleton: spine_flexion, shoulders_openness and legs_spread. All three are illustrated in Fig. 5.2. In a real-world scenario, users could introduce other parameters as they see fit, as long as they can provide a function to calculate them from pose data.

Experimental parameters

All three represent the angle formed at the intersection of two vectors ⃗ u and ⃗ v obtained from the pose. Further equations use the θ(⃗ u, ⃗ v) function in Eq. 5.1 to compute the angle at their intersection.

θ(⃗ u, ⃗ v) = arccos ⃗ u • ⃗ v ∥⃗ u∥ • ∥⃗ v∥ (5.1)
The spine_flexion (SF ) parameter represents the angle between the world "up" vector ⃗ y and the skeleton's spine axis. It is computed as shown in Eq. 5.2. It could be used to bend the character forward when in a hurry, or backward when relaxed.

SF (x) = θ(x neck -x pelvis , ⃗ y) (5.2)
Next the metric shoulders_openness (SO, see Eq. 5.3) describes how open the character's torso/shoulders area is. Example usages might be to raise the character shoulders to denote anxiety or shyness, and to open up to denote a more relaxed or proud behavior.

SO(x) = θ(x spine1 -x rshoulder , x lshoulder -x spine1 ) (5.3)
Finally, legs_spread (LS, see Eq. 5.4) denotes how close the character's knees are to each other. Possible usages include restricting walking strides, or tweaking whether the character appears in balance.

LS(x)

= θ(x pelvis -x rknee , x lknee -x pelvis ) (5.4)

Editing a pose through style parameters

With pose style parameters defined and some example ones laid out, we can turn to actually using them to edit a pose. The idea is similar to the Inverse Kinematics solver discussed in the previous chapter: train a neural network to generate a new latent pose from a starting one z and a target value p for P a specific pose parameter : ẑ = M P (z ⊕ p). An instance of the network is specialized for a single pose parameter, and it operates in the latent pose space to implicitly handle skeletal constraints.

To reflect this objective during training, we sample two poses from the same original animation sequence with an offset of n frames. Sampling from a neighboring frame prevents the target pose from being too different from the starting one. This ensures that the network learns to edit a pose rather than to "dream up" an unrelated one matching the target pose parameter. In our experiments, n is sampled randomly for each iteration in n ∈ [-10, 10]. As we target an interactive edition context, aiming for smaller changes in metric is reasonable. Gathered together, the parameter network's weight set β P are found by minimizing Eq. 5.5. min

β P L M P (β P ) = E (x;x ′ )∼x data x ′ -D M E(x) ⊕ P (x ′ ) 2 2
(5.5)

Keeping with the idea of smaller networks we opt for a two-layers perceptron with 126 hidden units per layer and ReLU activation. The input layer's number of units is equal to the size of the auto-encoder's latent dimension plus one (to account for the parameter value). The output is the same size as the latent dimension. We empirically find that this architecture suits our needs, larger/deeper networks yielding no perceptible value.

Training parameters. Each pose parameter network is trained using the Adam optimizer [START_REF] Diederik | Adam: A Method for Stochastic Optimization[END_REF] with a learning rate of 0.0001, and batches of 1024 pose pairs until convergence, which takes around 15 minutes on an NVIDIA Quadro T1000.

Extending to edit animation clips

The editing pipeline presented in this thesis up until this point only operates on a single pose at once. In a real-world scenario however animators typically edit a specific key frame, but expect their changes to be gradually applied to the neighboring ones.

To meet these expectations we propose a method to edit an entire animation sequence through a single set of targets, composed of the user's choice of pose parameters and/or IK targets.

The proposed approach (Fig. 5.3) resembles the traditional animation curve paradigm, using a user-provided weight curve W to control how much each key pose will be impacted by a modification.

To edit an animation A, the user selects a key frame A n and provides their desired target value for each parameter: P 0..i as well as a weight curve W . For each frame A t in the sequence, a latent representation z t is encoded. Each of them is then processed by the pose metric module, resulting in an intermediary latent pose z t . The final modified latent pose ẑt is obtained by blending z t and z t using the frame's weight factor W (t) (Eq. 5.6).

z t = E(A t ) z t = M P (z t ) ẑt = 1 -W (t) • z t + W (t) • z t Ât = D(ẑ t ) (5.6)
In our experiments W is a hat function starting at 0 at the clip's extremities and linearly increasing upon reaching 1 at the selected frame. This weight function is however an external parameter: experiments with a sinusoidal yield comparable results, and the curve could be edited by animators to fit their expectations, much like existing animation curves. We also note that each one of the clip's poses is processed independently of the others. The edition can thus happen in parallel, maintaining real time interactivity.

Results

Pose edition

In Fig. 5.4, a single pose is modified by a user by providing gradually increasing target spine_flexion parameter values. The results respect the desired changes as shown by the measured values below each pose. The resulting poses also illustrate the interesting properties of the latent pose space. At every iteration, the skeleton's arms are slightly raised, and its knees bent, to compensate the loss of balance induced by the flexion of the spine.

Using multiple modules

sf so 2.6

2.8 2.9 2.7 2.8 2.9 1.9

1.6 1.5 (1) (2)
Figure 5.5: Sample results of the pipeline using multiple modules. In (1), shoulders_openness (so) and spine_flexion (sf) are used simultaneously. In (2) the IK solver from the previous chapter is used along the spine_flexion parameter manipulator.

Fig. 5.5 presents some results obtained by running multiple modules in sequence. The first example shows the result of modifying the spine_flexion and shoulders_openness metrics at the same time. Both modifications can be seen on the skeleton: its spine is straightened, and its shoulders lowered. The rest of the pose is also slightly edited to accommodate for those change: the hands are lowered, and knees bent a little further.

The second one shows the result of using the spine_flexion parameter module along with the IK solver for the two hands presented in the previous chapter. The method allows us to find a compromise between each provided constraint. Here, the skeleton's back is straightened, but the hands are lowered to make up for the difference, still reaching for the targets. The output is produced by the pipeline given a selected frame A n , a target value for the legs_spread metric, and a weight curve. At the selected frame, the character's legs are closer to each other. The rest of the clip is less impacted, and the overall walking motion is preserved.

A n A n+1 A n-1 A n-2 A n-3 A n-4 A n-5 A n+2 A n+3 A n+4 A n+5 0

Animation edition

Fig. 5.6 showcases the result of an animation being edited using the animation pipeline.

The user selects a frame A n , provides a target leg_spread parameter value and an interpolation factor curve, a hat function peaking at one at A n and reaching down to 0 on both sides at A n-3 and A n+3 . The remaining frames use a factor of zero.

Each frame pose is processed separately by the network, and the result is given by interpolating the latent representations of the input and output poses, using the frame-specific factor.

As a result, the output animation  is modified, with the impact being most important on the selected frame and gradually lowering on each side. At frame n the output pose is modified the most, and the reduced legs angle is very visible. At frames n -2 and n + 2 this impact is barely visible, while further frames are not modified at all.

Interpolating between original and modified latent poses allow the whole sequence to keep its temporal coherence even though the pipeline processes one frame at a time, with no knowledge of the past and future frame context. We also take advantage of the properties of latent spaces: any latent interpolation result is mapped back to the real pose space, thus avoiding artifacts such as skeleton interpenetration.

Conclusion

The different approaches to animation style editing in the literature often rely on style categories, which suffer from under-considered biases and as such, might be too subjective to be used as-is in editing tools. Expert animators often want to tweak animation clips in a fine and precise manner to fit their vision of a character's motion.

We believe that editing tools should seek to support them with controllable tools. In this chapter an alternative approach was described, making use of objective pose parameters computed on a single pose. This approach leaves the control to users, who can define parameters set which fit their own perception of styles. The proposed Chapter 5. Learned pose manipulators for animation style editing 57 method once again leverages the latent pose space used throughout the thesis to automatically enforce implicit and subtle constraints. We also describe a pipeline in which multiple neural networks act as real-time pose manipulators.

The main shortcoming of this approach is that it is limited to a single standardized humanoid skeleton, with which the networks are trained. This is a common problem with data-driven methods: in order to adapt to a new skeleton, the network would need to be retrained. This could be difficult if access to existing animation data for the new skeleton is limited, but recent progress in animation retargeting [START_REF] Aberman | Skeleton-Aware Networks for Deep Motion Retargeting[END_REF] could help by allowing more accessible data augmentation. The modularity of the parameters helps here as well, as different skeleton will require a new set of parameters.

Contributions summary

Virtual characters play important roles in most computer-generated media, and synthesizing then editing motion data to animate them is a crucial task in production processes. The work presented in this thesis follows a wide array of research efforts which aim to make computer animation always more detailed and expressive, all the while managing the complexity of the creative pipelines.

In recent years, deep learning methods have started making waves in the field, leveraging the impressive data modelling capabilities of neural networks and the growing availability of Motion Capture data to produce new, unseen motion. Applications considered by previous work have however remained quite high-level, generating full motion clips in one go and leaving little room for artistic control. This thesis therefore proposes to explore this blind spot, by applying neural networks to a more restricted task, pose editing, and embedding the resulting models in interactive editing tools that leave control to the artists.

Our method relies on the construction of a latent pose space, on which existing poses can be projected then edited. A large part of the problems encountered while editing a character pose can be attributed to the subtle constraints a character's skeleton's parameterization must respect in order to be considered as a plausible pose. With this in mind, the space is structured to ensure any of its sample represent a valid pose. This condition is handled by neural networks trained to map existing poses to and from the latent space.

In chapter 3, we describe the construction of this space, by detailing two different approaches. The first relies on a simple autoencoder, and results in a space of unspecified shape. To the contrary, in the second, an adversarial method is employed to shape the resulting space and enforce its convexity, providing it with smooth interpolation properties. Both spaces form the base upon which the pose editing methods described later build.

Chapter 4 is dedicated to our first pose editing problem: full-body Inverse Kinematics.

In a first section we propose an optimization-based solution, taking advantage of the smooth nature of the latent space to speed up the process by only considering valid latent poses. This approach yields appealing results, however, its convergence time is not bound, and as such it is not ideally suited for interactive applications. In a second section we develop another solution to the same problem, this time training another neural network to explore an unconstrained latent space. Once trained, the method runs in real-time and yields high-quality results, allowing it to be used as an interactive tool to manipulate a character's skeleton as a puppet.

Finally, in chapter 5, we study the existing methods that propose to edit character poses by manipulating their style, and argue that the concept, as currently applied in computer animation, might build upon biased point of views. We argue for an alternative approach which allow users to compose their own definition of a style by combining objective pose parameters. We propose a system in which the pose parameters are edited in the latent space.

Overall, the work presented in this thesis makes a case for user-centered applications of neural networks in animation. Deep learning approach have already proven to capable of modelling large amounts of data, and generating impressive and complete character motion. However, we believe that their learning power and fast inference speeds make them good candidates to be used in real-time interaction with a user. We hope this document can inspire future work in this direction, and that future animators can rely on data-powered tools to put their idea in motion.

The work presented in this thesis has also been published as part of several papers:

Limitations & perspectives

This thesis' contributions are a first step towards user-controllable, data-driven animation editing tools, focusing on pose editing. The application is however still new and multiple problems still stand out.

Mainly, in this thesis, the pose editing task is considered as a stand-alone process. However, in real scenarios, single poses are always part of a sequence, and form an animation clip. A lot of information, related to velocity, pace, or motion amplitude for example, is lost when separating pose data from its context. We propose an approach to account for the dynamic nature of motion in 5, but the training is still done on pose data alone. Providing neural network models with more of this context might be beneficial, even for tasks related to pose edition. This is especially the case for our style parameters based method, as many of the existing motion descriptors from the literature are not applicable to single poses. An interesting perspective will then be to adapt the neural networks models so that they can extract information from motion sequences, and apply them in contexts similar to ours.

In the same vein, the networks used in this thesis are deliberately kept simple. As shown in the literature review, model architectures specifically dedicated to animation data are currently a heavily studied topic, so applying these modern approaches to our tasks could significantly improve the quality of the results.

Next, our approach is limited to a standardized skeleton. In actual production, each character often has its own, and existing data might not even be available. Future work should focus on alleviating this limit by reusing trained networks on different skeleton topologies. Improvement in motion retargeting might also help to adapt existing motion data to a new skeleton.

This thesis' contributions are still in an early shape, and have not yet been confronted to actual animators. An interesting perspective is thus to lead user-studies to gather feedback on the usability and efficiency of the methods. End-users consideration would help identify more limitations as well as more venue for improvements.

Data-oriented animation is already widely used in the industry, thanks to the increased availability of MoCap technologies. Deep learning methods however, while receiving a lot of attention from the research community, have yet to break in production setups.

Leveraging their power in the context of user-centric tools, in which the software facilitates the animators work while leaving them in control, might be the key to a widespread usage. We hope our work inspires more research towards similar goals, empowering animators in their creative process and helping to populate ever more extraordinary virtual worlds.

Acronyms

List of Figures

2.1 A textured 3D mesh with its underlying animation skeleton (in pink).

vi 6 . 1

 61 Contributions summary . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Limitations & perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Thesis contributions & organization . . . . . . . . . . . . . . . . . . . 3

Figure 2 .

 2 Figure 2.1: A textured 3D mesh with its underlying animation skeleton (in pink).

Figure 2 . 2 :

 22 Figure 2.2: Interpolating between key frames creates a new pose for each frame, creating the illusion of a smooth animation.

Figure 2 . 3 :

 23 Figure 2.3: Forward Kinematics: A modification to the orientation of the joint P 0 at the starting position (1) is propagated down the chain, modifying the positions of both P 1 and P 2 in the resulting position (2).

Figure 2 . 4 :

 24 Figure 2.4: Inverse Kinematics: Given a target position for the end effector P 2 the IK method must find a proper parameterization of both P 0 and P 1 . Multiple solutions exist.

( a )

 a Figure 2.5: Sketch-based posing interfaces In traditional pen-and-paper animation, it is common for animators to start posing by sketching a simplified version of the character to quickly get an idea of what their mental picture of the pose will look like. The draft is then progressively refined

Figure 2 . 6 :

 26 Figure 2.6: The concept of Line of Action ©Animation, P. Blair, 1948 [Bla48].

Figure 2 .

 2 Figure 2.7: A physical pose editing interface through an actuated puppet. From Yoshizaki et al. [Yos+11].

( a )

 a Discontinuity in direction (b) Discontinuity in speed (c) Distortions in length when rotating

Figure 2 . 8 :

 28 Figure 2.8: Linear interpolations limitations for animation, Kochanek 1984 [KB84]

Figure 2 . 9 :

 29 Figure 2.9: Animation curve in Blender. The curve represents the value over time for a single component, parameterized by 3 keyframes (black dots) and the interpolation splines at each of them. The control point of a spline is shown in orange.

Figure 2 .

 2 Figure 2.10: A (simple) animation graph in Godot Engine. Each node represents an animation clip, each edge a possible transition.

  2.11). A succession of reviews have been published [WCW14; AP19;Mou+22] in attempts to keep up with this fast growth.

Figure 2 . 11 :

 211 Figure 2.11: Histogram of the volume of peer-reviewed publications in human skeletal animation using Deep Learning (DL) or Deep Reinforcement Learning (DRL) over the past decade. From Mourot et al. [Mou+22]

Figure 2 .

 2 Figure 2.12: Motion synthesis constrained on a user-provided trajectory signal, from Holden et al. [Hol+15].

  notably by using multiple expert networks focusing on different phases, orchestrated by a gating network [Zha+18; Sta+19; Sta+20]. While most of the early approaches to deep-learning-based motion control focused entirely on locomotion, more recent work have started taking interest in other tasks such as dribbling with a basket ball [LLL18; Sta+21].

Figure 2 .

 2 Figure 2.13: Character control using neural networks, from Holden et al. [HKS17]. At each frame the network generate the next one based on the user's input on a gamepad controller.

Figure 2 .

 2 Figure 2.14: Neural networks have been used to retarget existing animation to a different skeleton. From Aberman et al. [Abe+20].

  Around the same time, the network used proposed by Holden et al.[START_REF] Holden | Learning Motion Manifolds with Convolutional Autoencoders[END_REF][START_REF] Holden | A Deep Learning Framework for Character Motion Synthesis and Editing[END_REF] proposed a derivative of CNNs on fixed-width sequences of motion, using 1-dimensional convolutions on individual pose features to model their temporal dynamics.

  [START_REF] Cai | Learning Progressive Joint Propagation for Human Motion Prediction[END_REF][START_REF] Aksan | A Spatio-temporal Transformer for 3D Human Motion Prediction[END_REF]. Miscellaneous Nowadays, new architecture keep emerging in other fields, yielding ever more impressive results. They inevitably also raise interest in the animation community, and very recent work is interested in modelling motion data using newer architectures such as normalizing flows[START_REF] Henter | MoGlow: Probabilistic and Controllable Motion Synthesis Using Normalising Flows[END_REF], Neural Radiance Fields (NeRF) [He+22], or Diffusion Models[START_REF] Zhang | MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model[END_REF].

Figure 2 .

 2 Figure 2.15: Graphical representation of the 2D arousal-valence scale.Positions of affective terms are placed indicatively.

Figure 2 .

 2 Figure 2.16: Illustration of the style transfer task. A random motion clip in the "neutral" style (top) is transformed to the "proud" style (bottom) [Xia+15].

c h a p t e r A

 r latent space for pose editing 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 System overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 The pose dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Pose representation for neural networks . . . . . . . . . . . . . Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Input format for our neural networks . . . . . . . . . . . . . . . 3.3.2 Source animation datasets . . . . . . . . . . . . . . . . . . . . . 3.3.3 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.4 Post-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Baseline autoencoder . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Autoencoding Generative Adversarial Network (AEGAN) . . . Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Networks architectures . . . . . . . . . . . . . . . . . . . . . . . Loss functions . . . . . . . . . . . . . . . . . . . . . . . . . . . Training parameters . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 3 . 1 :

 31 Figure 3.1: High-level overview of the latent space editing system. The functions E and D map real poses sampled from X to the latent space Z. The function M t operates on latent sample to edit poses with regard to a user-provided target value T .

Figure 3 . 2 :

 32 Figure 3.2: Illustration of the accumulation of errors with hierarchical angular representations: small angle errors in the kinematic chain can quickly accumulate to larger positional errors, especially noticeable in the end effectors (left). Positional representations (left) are much less sensitive to the problem. Figure from Mourot et al. [Mou+22].

Figure 3 . 3 :

 33 Figure 3.3: Illustration of the skeleton used as input. The kinematic chains are shown in different colors. The pose tensor p is obtained by concatenating the positions of each joint.

Figure 3 . 4 :

 34 Figure 3.4: Overview of the autoencoder architecture. The encoder learns to map real samples to the latent space, and the decoder to map latent ones to the pose space. Both networks are trained to minimize the reconstruction error.

Figure 3 . 5 :

 35 Figure 3.5: Architecture details of the encoder and decoder models.

Figure 3 . 6 :

 36 Figure 3.6: High level overview of the AEGAN architecture. E and G function as a traditional autoencoder, but take an additional feedback from both discriminators to enforce latent space convexity.

Figure 3 . 7 :

 37 Figure 3.7: Overview of the Structured Prediction Layer from Aksan et al. [AKH19]. The embeddings of parent joints are added to the input of the embedding layer of a joint.

  ) fool the pose discriminator into classifying generated and encoded-decoded poses as real ones; and (3) and (4) fool the latent discriminator into classifying encoded poses and generated-encoded poses as random latent sample. min θ,ϕ

Figure 3 . 8 :

 38 Figure 3.8: Visual comparison of pose spaces (original pose space, baseline autoencoder's latent space, and AEGAN's latent space). Real poses (darker) and randomly drawn data points (lighter) are embedded in 2D using t-SNE [HR02; MH08].

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.2 Optimization in the latent space . . . . . . . . . . . . . . . .

Figure 4 . 1 :

 41 Figure 4.1: Optimizing in the convex latent space guaranties thatany pose considered during the process will be coherent.

Figure 4 . 2 :

 42 Figure 4.2: Examples of pose edition using latent optimization and FABRIK [AL11]. Targets are shown in red.

  Figure 4.2 shows a comparison of the poses obtained by latent optimization and FABRIK with the same targets set.

Figure 4 . 3 :

 43 Figure 4.3: High level overview of the generation setup. The target joint's positions (red) are matched as closely as possible, while the other joints (green) should be as close as possible to the starting pose. The loss is computed as the difference between the generated pose and the target pose.

Figure 4 . 4 :

 44 Figure 4.4: Starting from a pose and targets for two joints, an IK solver like FABRIK (middle) generates less realistic poses than our neural solver (bottom).

Fig. 4. 4

 4 Fig. 4.4 showcases an example of how our method can be used to edit a pose by moving the targets around. In this case, a single solver with the targets set to both skeleton's hands is used. We once again compare against FABRIK in the same setup as the previous section.

Figure 4 . 5 :

 45 Figure 4.5: Sample results solving multiple targets with a sequence of neural solvers (S hands , S f eet , and S head ). Targets are shown in red.

Figure 4 . 6 :

 46 Figure 4.6: General comparison of various full-body IK methods in terms of speed and quality. Style IK [WTR11], NAT-IK [Hua+17], JDLS [BK05], CCD [WC91] FABRIK [AL11]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.2 Limits of existing style editing techniques . . . . . . . . . . . . . . . . 50 5.3 Pose style parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.3.2 Experimental parameters . . . . . . . . . . . . . . . . . . . . . 52 5.4 Editing a pose through style parameters . . . . . . . . . . . . . . . . . 53 5.5 Extending to edit animation clips . . . . . . . . . . . . . . . . . . . . . 53 5.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.6.1 Pose edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.6.2 Using multiple modules . . . . . . . . . . . . . . . . . . . . . . 55 5.6.3 Animation edition . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Figure 5 . 1 :

 51 Figure 5.1: Examples of walk animations in exaggerated styles from existing motion databases [Hol+15; Xia+15]. (1) "zombie"; (2) "old man"; (3) "childish". Distinguishing each style with no context is not straight-forward.

Figure 5 . 2 :

 52 Figure 5.2: Visual depiction of the three pose parameters used in this paper's examples.

Figure 5 . 3 :

 53 Figure 5.3: Visual overview of the proposed method to echo modifications made to a key pose A n . The pose is encoded then sequentially modified by the pose parameter networks M P0..i to accommodate the parameter targets p 0..i provided by the user. The resulting latent tensor is then blended with the original one by a per-frame factor W (n), then finally decoded. By repeating this operation for each neighboring pose according to the weight curve, the full clip is modified with regard to the provided targets.

Figure 5 . 4 :

 54 Figure 5.4: Incremental modification of a single pose by providing a target spine_f lexion input. On each step, the user's input is a 0.1 increase over the computed parameter value.

Figure 5 . 6 :

 56 Figure 5.6: Editing an animation clip according to the user input.The output is produced by the pipeline given a selected frame A n , a target value for the legs_spread metric, and a weight curve. At the selected frame, the character's legs are closer to each other. The rest of the clip is less impacted, and the overall walking motion is preserved.

6 c h a p t e r Conclusion 6 . 1

 61 Contributions summary . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.2 Limitations & perspectives . . . . . . . . . . . . . . . . . . . . . . . . 59

2 . 2

 22 Interpolating between key frames creates a new pose for each frame, creating the illusion of a smooth animation. . . . . . . . . . . . . . . . 2.3 Forward Kinematics: A modification to the orientation of the joint P 0 at the starting position (1) is propagated down the chain, modifying the positions of both P 1 and P 2 in the resulting position(2). . . . . . 2.4 Inverse Kinematics: Given a target position for the end effector P 2 the IK method must find a proper parameterization of both P 0 and P 1 . Multiple solutions exist. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Sketch-based posing interfaces . . . . . . . . . . . . . . . . . . . . . . . 2.6 The concept of Line of Action ©Animation, P. Blair, 1948 [Bla48]. . . 2.7 A physical pose editing interface through an actuated puppet. From Yoshizaki et al. [Yos+11]. . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Linear interpolations limitations for animation, Kochanek 1984 [KB84] 2.9 Animation curve in Blender. The curve represents the value over time for a single component, parameterized by 3 keyframes (black dots) and the interpolation splines at each of them. The control point of a spline is shown in orange. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 A (simple) animation graph in Godot Engine. Each node represents an animation clip, each edge a possible transition. . . . . . . . . . . . . . 2.11 Histogram of the volume of peer-reviewed publications in human skeletal animation using Deep Learning (DL) or Deep Reinforcement Learning (DRL) over the past decade. From Mourot et al. [Mou+22] . . . . . . 2.12 Motion synthesis constrained on a user-provided trajectory signal, from Holden et al. [Hol+15]. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.13 Character control using neural networks, from Holden et al. [HKS17]. At each frame the network generate the next one based on the user's input on a gamepad controller. . . . . . . . . . . . . . . . . . . . . . . 2.14 Neural networks have been used to retarget existing animation to a different skeleton. From Aberman et al. [Abe+20]. . . . . . . . . . . . 2.15 Graphical representation of the 2D arousal-valence scale. Positions of affective terms are placed indicatively. . . . . . . . . . . . . . . . . . . 2.16 Illustration of the style transfer task. A random motion clip in the "neutral" style (top) is transformed to the "proud" style (bottom) [Xia+15]. 3.1 High-level overview of the latent space editing system. The functions E and D map real poses sampled from X to the latent space Z. The function M t operates on latent sample to edit poses with regard to a user-provided target value T . . . . . . . . . . . . . . . . . . . . . . . . List of Figures 3.2 Illustration of the accumulation of errors with hierarchical angular representations: small angle errors in the kinematic chain can quickly accumulate to larger positional errors, especially noticeable in the end effectors (left). Positional representations (left) are much less sensitive to the problem. Figure from Mourot et al. [Mou+22]. . . . . . . . . . 3.3 Illustration of the skeleton used as input. The kinematic chains are shown in different colors. The pose tensor p is obtained by concatenating the positions of each joint. . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Overview of the autoencoder architecture. The encoder learns to map real samples to the latent space, and the decoder to map latent ones to the pose space. Both networks are trained to minimize the reconstruction error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Architecture details of the encoder and decoder models. . . . . . . . . 3.6 High level overview of the AEGAN architecture. E and G function as a traditional autoencoder, but take an additional feedback from both discriminators to enforce latent space convexity. . . . . . . . . . . . . 3.7 Overview of the Structured Prediction Layer from Aksan et al. [AKH19]. The embeddings of parent joints are added to the input of the embedding layer of a joint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Visual comparison of pose spaces (original pose space, baseline autoencoder's latent space, and AEGAN's latent space). Real poses (darker) and randomly drawn data points (lighter) are embedded in 2D using t-SNE [HR02; MH08]. . . . . . . . . . . . . . . . . . . . . . . . . . . .4.1 Optimizing in the convex latent space guaranties that any pose considered during the process will be coherent. . . . . . . . . . . . . . . . . . 4.2 Examples of pose edition using latent optimization and FABRIK [AL11]. Targets are shown in red. . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 High level overview of the generation setup. The target joint's positions (red) are matched as closely as possible, while the other joints (green) should be as close as possible to the starting pose. The loss is computed as the difference between the generated pose and the target pose. . . . 4.4 Starting from a pose and targets for two joints, an IK solver like FABRIK (middle) generates less realistic poses than our neural solver (bottom). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Sample results solving multiple targets with a sequence of neural solvers (S hands , S f eet , and S head ). Targets are shown in red. . . . . . . . . . . 4.6 General comparison of various full-body IK methods in terms of speed and quality. Style IK [WTR11], NAT-IK [Hua+17], JDLS [BK05], CCD [WC91] FABRIK [AL11] . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Examples of walk animations in exaggerated styles from existing motion databases [Hol+15; Xia+15]. (1) "zombie"; (2) "old man"; (3) "childish". Distinguishing each style with no context is not straight-forward. . . . 5.2 Visual depiction of the three pose parameters used in this paper's examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .List of Figures5.3Visual overview of the proposed method to echo modifications made to a key pose A n . The pose is encoded then sequentially modified by the pose parameter networks M P 0..i to accommodate the parameter targets p 0..i provided by the user. The resulting latent tensor is then blended with the original one by a per-frame factor W (n), then finally decoded. By repeating this operation for each neighboring pose according to the weight curve, the full clip is modified with regard to the provided targets. 5.4 Incremental modification of a single pose by providing a target spine_f lexion input. On each step, the user's input is a 0.1 increase over the computed parameter value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Sample results of the pipeline using multiple modules. In (1), shoul-ders_openness (so) and spine_flexion (sf) are used simultaneously. In (2) the IK solver from the previous chapter is used along the spine_flexion parameter manipulator. . . . . . . . . . . . . . . . . . . . . . . 5.6 Editing an animation clip according to the user input. The output is produced by the pipeline given a selected frame A n , a target value for the legs_spread metric, and a weight curve. At the selected frame, the character's legs are closer to each other. The rest of the clip is less impacted, and the overall walking motion is preserved. . . . . . . . . . FOLIO ADMINISTRATIF THÈSE DE L'UNIVERSITÉ DE LYON OPÉRÉE AU SEIN DE L'INSA LYON NOM : Victor DATE DE SOUTENANCE : 07/04/2023 PRÉNOM : Léon TITRE : Learning-based Interactive Character Animation Edition NATURE : Doctorat NUMERO D'ORDRE : 2023ISAL0028 ÉCOLE DOCTORALE : Mathématiques et Informatique de Lyon (EDA 512) La principale méthode utilisée pour animer un personnage virtuel consiste à éditer le mouvement d'un squelette, sur lequel un modèle sous forme de maillage sera appliqué et déformé au besoin. Pour convaincre le spectateur, le mouvement d'un personnage doit respecter de nombreuses règles implicites, comme le modèle physique qui régit son monde ou les limites de sa morphologie. Ce faisant, il doit aussi rendre explicite l'action réalisée, et laisser transparaître l'état émotionnel du personnage. La subtilité de ces contraintes rend la création d'animation difficile et la production d'une animation réaliste et/ou plaisante repose fortement sur les connaissances, l'expérience et minutie de l'animateur. La démocratisation des techniques de capture de mouvement (Motion Capture, ou MoCap) permet de produire de plus en plus de données de mouvements tirés du monde réel, qui portent intrinsèquement des informations sur ces contraintes. Les travaux présentés dans cette thèse proposent d'exploiter ces données à l'aide de méthodes récentes d'apprentissage automatique, utilisant les réseaux de neurones pour fournir de nouveaux outils aux artistes animateurs. MOTS-CLEFS : Computer Animation, Character Animation, Animation Editing, Neural Networks LABORATOIRE DE RECHERCHE : Laboratoire d'Informatique en Image et Systèmes d'Information(LIRIS) DIRECTRICE DE THÈSE : Véronique Eglin PRÉSIDENT.E DE JURY : XXX COMPOSITION DU JURY : M. Jean-Philippe Vandeborre, M. Renaud Séguier, MMe Sylvie Gibet, M. Sébastien Mavromatis, M. Bertrand Kerautret, MMe Véronique Eglin, MMe Saïda Bouakaz, M. Alexandre Meyer,

table 4

 4

	.1.
	Method	Memory footprint (kB)	Runtime (ms)**
	FABRIK (2)	-	6.56
	Ours (2)	442	1.47 (3.03*)
	FABRIK (5)	-	6.74
	Ours (5)	826	3.36 (4.58*)
		*With post-processing
		**Average over 1000 random iterations

Table 4 .

 4 1: Comparative numeric results of the neural and FABRIK solvers with two and five end-effectors (using the combined solver method). All experiments are run on a single CPU thread.

  CMUThe Carnegie Mellon University Graphics Lab Motion Capture Database.

	Acronyms	62
	RNN Recurrent Neural Network.
	SPL Structured Prediction Layer.
	t-SNE t-Distributed Stochastic Neighbor Embedding.
	VAE Variational Autoencoder network.
	AE Autoencoder network.
	AEGAN Autoencoding Generative Adversarial Network.
	CCD Cyclic Coordinate Descent.
	CNN Convolutional Neural Networks.
	LoA Line of Action.
	LTI Linear Time Invariant.
	MoCap Motion Capture.
	MSE Mean Squared Error.
	NeRF Neural Radiance Field.
	NN Neural Network.
	PCA Principal Components Analysis.

CRBM Conditioned Restricted Boltzmann Machine. DoF Degree of Freedom. ERD Encoder-Recurrent-Decoder. FABRIK Forward and Backward Reaching Inverse Kinematics. FK Forward Kinematics. GAN Generative Adversarial Network. GCN Graph Convolutional Network. GPLVM Gaussian Process Latent Variable Model. ICA Independent Component Analysis. IK Inverse Kinematics. LMA Laban Movement Analysis (LMA). RBF Radial Basis Function. RBM Restricted Boltzmann Machine. ReLU Rectified Linear Unit.

Cette thèse est accessible à l'adresse : https://theses.insa-lyon.fr/publication/2023ISAL0028/these.pdf © [L. Victor], [2023], INSA Lyon, tous droits réservés

For overviews of the state of the art in MoCap technologies one can refer to Zhu et al. [ZL16] or Desmarais et al. [Des+21] on real-time-capture and markerless approaches, respectively. Cette thèse est accessible à l'adresse : https://theses.insa-lyon.fr/publication/2023ISAL0028/these.pdf © [L. Victor], [2023], INSA Lyon, tous droits réservés

Data-driven IK solutions are closely related to full-body posing techniques, so both have been grouped and are discussed in their own section (2.3.1) below. Cette thèse est accessible à l'adresse : https://theses.insa-lyon.fr/publication/2023ISAL0028/these.pdf © [L. Victor], [2023], INSA Lyon, tous droits réservés

Defining and classifying emotions is also a complex topic[START_REF] Kleinginna | A Categorized List of Motivation Definitions, with a Suggestion for a Consensual Definition[END_REF].Cette thèse est accessible à l'adresse : https://theses.insa-lyon.fr/publication/2023ISAL0028/these.pdf © [L. Victor], [2023], INSA Lyon, tous droits réservés

OCEAN is a taxonomy that has been studied in psychology to define the personality of individuals through five semantic categories[START_REF] Lewis | An Alternative "Description of Personality": The Big-Five Factor Structure[END_REF]. OCEAN scores can be derived from the words used to describe a person. The model's relevance and methodological bases have however been criticized[START_REF] Block | A Contrarian View of the Five-Factor Approach to Personality Description[END_REF].Cette thèse est accessible à l'adresse : https://theses.insa-lyon.fr/publication/2023ISAL0028/these.pdf © [L. Victor], [2023], INSA Lyon, tous droits réservés