Cormac Flynn
email: cormac.flynn@wintec.com

Mohammad Ali Nazari

Pascal Perrier

Sidney Fels

Poul M F Nielsen

Yohan Payan

Computational modeling of the passive and active components of the face

Keywords: Face, finite element models, constitutive relationships, skin, muscles, expressions, lips

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The development of computational face models is motivated by their application in many disciplines, including animation, biomedical engineering, and forensics [START_REF] Duan | Craniofacial reconstruction based on multi-linear subspace analysis[END_REF][START_REF] Wu | Generating facial expressions using an anatomically accurate biomechanical model[END_REF][START_REF] Zhang | An eFace-Template Method for Efficiently Generating Patient-Specific Anatomically-Detailed Facial Soft Tissue FE Models for Craniomaxillofacial Surgery Simulation[END_REF]. Clinical uses of face models could also improve patient outcome.

While face animation has been used for realistic talking heads for computational social agents [START_REF] Sagar | A neurobehavioural framework for autonomous animation of virtual human faces[END_REF], the technique still does not cross the uncanny valley. This phenomenon proposes that as artificial faces become more realistic they become more eerie and repulsive to a perceiver [START_REF] Mori | The uncanny valley [from the field[END_REF].

Facial animations fail to cross the uncanny valley partly due to the unrealistic motion of the facial tissue. However, biomechanically driven models, such as finite element models, of the face provide the opportunity to incorporate the subtleties of how facial skin and the underlying layers move physically. Furthermore, when combined with biomechancially based models, such as the upper airway (chapter 10), and tongue (chapter 25), it is possible to have a complete functional model of a talking, breathing, and eating head that captures the underlying physics of motion during these complex movements leading to a more realistic correlation between form and function needed to cross the uncanny valley. These biomechanical face models can also be used for maxillofacial surgical simulations [START_REF] Hannam | Current computational modelling trends in craniomandibular biomechanics and their clinical implications[END_REF] to facilitate planning and training of complex operations to improve speech and mastication outcomes [START_REF] Virgin | Functional Outcomes of Fibula and Osteocutaneous Forearm Free Flap Reconstruction for Segmental Mandibular Defects[END_REF]. Additionally, realistic face prosthetics can be designed to move and deform realistically using these face models [START_REF] Liu | Biomechanical characterization of a low density silicone elastomer filled with hollow microspheres for maxillofacial prostheses[END_REF].

We can use simulations to investigate speech mechanics. Jaw, lip, and tongue coordination for speech is challenging to analyse experimentally. Physically accurate face models can compliment experiments and help determine clinical solutions to patients' speech impediments [START_REF] Stavness | A biomechanical modeling study of the effects of the orbicularis oris muscle and jaw posture on lip shape[END_REF].

Background

The human face comprises many anatomical components that contribute to appearance and function. Several accounts describe facial anatomy [START_REF] Larrabee | Surgical anatomy of the face[END_REF][START_REF] Prendergast | Cosmetic Surgery[END_REF]. Briefly, we summarise the structures in layers from superficial to deep.

Skin

Skin forms the outer face layer and contains the outer epidermis and the underlying dermis.

The epidermis contains several cell layers. The dermis comprises collagen and elastin fibres embedded in a ground matrix. Facial skin thickness and composition varies according to position and person. Its thickness ranges from 0.5 mm on eyelids to 1.5 mm on cheeks [START_REF] Lee | Skin thickness of Korean adults[END_REF][START_REF] Odland | Structure of the skin[END_REF][START_REF] Wilkes | The biomechanical properties of skin[END_REF].

Subcutaneous fat

Under the skin, the face contains several superficial fat compartments. They are the nasolabial fat compartment; temporal-cheek fat pads, which cover the cervical, cheek, and forehead regions; orbital fat pads; and the jowl fat compartment in the lower third of the face. Each compartment is separated by fascial tissue and septae [START_REF] Prendergast | Cosmetic Surgery[END_REF]. With aging, volume loss occurs at different rates in the fat compartments, which leads to less smooth contours on facial surfaces.

Superficial musculoaponeurotic system

The superficial musculoaponeurotic system (SMAS) is a tissue sheet of collagen and elastin fibres, and fat cells extending from the neck up to the forehead [START_REF] Mitz | The superficial musculo-aponeurotic system (SMAS) in the parotid and cheek area[END_REF]. It plays a key role in facial expression by connecting mimetic muscles to the dermis.

Two SMAS types exist. Type I SMAS occurs in the forehead, parotid, zygomatic, and infraorbital areas and comprises fibrous septae. Type II SMAS is a dense mesh of collagen, elastin, and muscle fibres and is found in the lip area [START_REF] Ghassemi | Anatomy of the SMAS revisited[END_REF]. SMAS thicknesses vary throughout the face from 2 mm to 3 mm [START_REF] Har-Shai | Mechanical properties and microstructure of the superficial musculoaponeurotic system[END_REF].

Retaining ligaments

Retaining ligaments are fibrous connective tissue condensations that either anchor skin to bone (also known as true retaining ligaments) or soft tissue layers to each other (also known as false retaining ligaments). The zygomatic ligament anchors the dermis near the inferior border of the zygomatic arch. The mandibular retaining ligament connects mandibular bone to the dermis. The orbicularis retaining ligament anchors the dermis in the anterior orbital rim region [START_REF] Mendelson | Surgical Anatomy of the Midcheek: Facial Layers, Spaces, and the Midcheek Segments[END_REF]. False retaining ligaments include the masseteric and platysma-auricular ligaments. The masseteric ligament is positioned along the anterior masseter muscle border and connects the SMAS and dermis. With aging, the retaining ligaments become weaker, which results in tissue sagging [START_REF] Mendelson | Surgical Anatomy of the Midcheek: Facial Layers, Spaces, and the Midcheek Segments[END_REF].

Mimetic muscles

Beneath the layer of retaining ligaments are numerous thin flat muscles known as mimetic muscles (figure 24.1). In the eye region are five muscle sets: the frontalis is the main elevator of brows; corrugator supercilli also depress brows and cause them to move medially; depressor supercilli have similar functions; obicularis oculi cause eyes to squeeze shut; and procerus pull lower forehead skin down to cause a horizontal crease on the nose [START_REF] Prendergast | Cosmetic Surgery[END_REF].

Around the mouth are levator muscles, zygomaticus major and minor, risorius, buccinator, orbicularis oris, depressor anguli oris, depressor labii, and mentalis. Zygomaticus major and minor originate in the zygomatic bone and insert in the mouth corner to lift it. The levator muscles (labii, labii superioris alaeque nasi, and anguli oris) elevate the upper lip and mouth corner. The risorius extends from the lateral cheek and inserts into the mouth corner, pulling it laterally. It is absent from some people [START_REF] Barbarino | Development and Validation of a Three-Dimensional Finite Element Model of the Face[END_REF]. The buccinator pulls back on the mouth and flattens the cheek. The orbicularis oris is a sphincter around the mouth responsible for lip action.

Depressor anguli oris originates in the mandible and merges with orbiculoris oris depressing the mouth corners upon contraction. Depressor labii inferioris arises from the mandible and extends upwards into the lower lip. Medial to this muscle is the mentalis, which elevates and protrudes the lower lip.

In the nose region, the compressor naris compresses the nasal passage; the dilator naris dilates the nostrils; and the depressor septi moves the nose tip downwards.

Twenty mimetic muscles have to be modelled to represent the whole face [START_REF] Wu | Generating facial expressions using an anatomically accurate biomechanical model[END_REF]. The models presented in later sections model a subset of these muscles to simulate specific facial expressions.

Deep-plane fat

In the deep plane beneath the mimetic muscles lie the suborbicularis oculi, bucal, and galeal fat pads. As well as providing volume and shape to the face, they act as glide planes for free movement of facial muscles. In particular, the suborbicularis oculi fat pad lies underneath the orbicularis region. The bucal fat pad allows movement of the mastication muscles. The corrugator and procerus glide upon the galeal fat pad. Deep plane fat pad volume loss occurs with ageing resulting in visible folds and grooves.

Facial skeleton

The skeleton provides anchoring points for muscles and structural support for facial soft tissues. The main facial bones are the frontal, zygomatic, maxilla, nasal, and mandible bones.

Previous computational models of the face

Facial tissue constitutive relationships

Constitutive facial soft tissue model development has been ongoing for several decades [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF][START_REF] Gerard | Non-linear elastic properties of the lingual and facial tissues assessed by indentation technique -Application to the biomechanics of speech production[END_REF][START_REF] Rubin | A three-dimensional nonlinear model for dissipative response of soft tissue[END_REF][START_REF] Rubin | An elastic-viscoplastic model for excised facial tissues[END_REF]. Many models were developed to simulate soft tissue from other anatomical locations.

Validating constitutive models requires experimental data. There is a relative dearth of such data to determine material parameters for facial soft tissues. Many studies use the Cutometer (Courage and Khazaka Electronic GmbH, Köln, Germany), which applies suction to skin [START_REF] Couturaud | Skin biomechanical properties: in vivo evaluation of influence of age and body site by a non-invasive method[END_REF][START_REF] Ohshima | Relevance of the directionality of skin elasticity to aging and sagging of the face[END_REF][START_REF] Tsukahara | A study of diurnal variation in wrinkles on the human face[END_REF], enabling parameters characterising skin distension and retraction to be identified. These studies demonstrate how age influences facial skin mechanical response [START_REF] Cua | Elastic Properties of Human Skin -Relation to Age, Sex, and Anatomical Region[END_REF] and property variations from person to person [START_REF] Luebberding | Mechanical properties of human skin in vivo: a comparative evaluation in 300 men and women[END_REF]. The format of reported results are not ideal for constitutive model development with certain skin deformation measurements only detailed at a couple of specific times. Other studies using similar suction protocols include Barbarino et al. [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF] and Luboz et al. [START_REF] Luboz | Linear Elastic Properties of the Facial Soft Tissues Using an Aspiration Device: Towards Patient Specific Characterization[END_REF]. A variation on this approach is to blow air normal to facial skin [START_REF] Fujimura | Development of a novel method to measure the elastic properties of skin including subcutaneous tissue: New age-related parameters and scope of application[END_REF]. These approaches cannot characterise anisotropic properties of facial tissues due to axi-symmetrical loading.

Ohshima et al. [START_REF] Ohshima | Relevance of the directionality of skin elasticity to aging and sagging of the face[END_REF] used a Reviscometer (Courage and Khazaka Electronic GmbH, Köln, Germany) to characterise anisotropic properties of three areas on 91 female faces. The approach assumes a linear stress-strain response by relating the wave speed and Young's modulus of skin.

A constitutive model developed to characterise facial tissue anisotropic dissipative response is detailed in work by Rubin and Bodner [START_REF] Rubin | A three-dimensional nonlinear model for dissipative response of soft tissue[END_REF]. This model further develops the approached published in Rubin et al. [START_REF] Rubin | An elastic-viscoplastic model for excised facial tissues[END_REF] and represents tissue as a composite of an elastic fibre component and a dissipative component. The strain energy equation is

W = µ 0 2q (e qg -1), g = g 1 (J) + g 2 ( B) + g 3 (λ 1 ) + g 4 (α 1 ), (1)
where µ 0 is a material constant having the units of stress and q is a dimensionless constant that controls the nonlinearity of the moduli. J is the volume ratio, and B is the deviatoric left Cauchy tensor. λ 1 represents the fibre stretch, and α 1 = Bde • I, where Bde is a pure measure of elastic distortional deformation associated with the dissipative component. g 1 characterises volume dilation; g 2 characterises total distortion; g 3 accounts for fibre stretch; and g 4 represents distortional dissipative deformation.

g 1 (J) = 2m 1 [(J -1) -ln(J)] g 2 ( B) = m 2 ( B : I -3) g 3 (λ 1 ) = m 3 m 4 λ 1 -1 2m4 g 4 (α 1 ) = α 1 -3, (2)
where m 1-3 are dimensionless material parameters related to the response of dilation, distortion, and fibre stretching. Rubin and Bodner [START_REF] Rubin | A three-dimensional nonlinear model for dissipative response of soft tissue[END_REF] evaluated the ability of this strain energy equation to simulate uniaxial tension of in vitro facial skin and SMAS [START_REF] Har-Shai | Mechanical properties and microstructure of the superficial musculoaponeurotic system[END_REF]. While the experimental data was insufficient for simulating tissue anisotropy, the model did simulate the cyclic loading-unloading and stress relaxation response of the samples.

A simplified Rubin-Bodner model has been used in studies on facial tissue mechanics. Barbarino et al. [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF] ignored the fibres and dissipation contributions when characterising five facial points (g 3 = g 4 = 0 in equation ( 2)), reducing the parameter number from fourteen to three independent parameters with physical meaning. µ 0 m 1 represents the initial bulk modulus, µ 0 m 2 represents the initial shear modulus, and qm 2 represents the nonlinearity of the moduli.

Barbarino et al. [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF] also fit a second-order reduced polynomial equation to the average response from the five facial points. nent of the Rubin-Bodner model (g 4 = 0 in Equation ( 2)). Using a similar protocol to Barbarino et al. [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF] and an axi-symmetric finite element (FE) model, they determined in vivo material parameters that simulated the averaged response of three facial points (forehead, parotid, and jaw). Differences occurred in some material parameter values between Weickenmeier et al. [START_REF] Weickenmeier | Suction based mechanical characterization of superficial facial soft tissues[END_REF] and both Barbarino et al. [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF] and Rubin and Bodner [START_REF] Rubin | A three-dimensional nonlinear model for dissipative response of soft tissue[END_REF]. These differences are explained by the in vitro samples used in Rubin and Bodner [START_REF] Rubin | A three-dimensional nonlinear model for dissipative response of soft tissue[END_REF] and ignoring dissipative effects in Barbarino et al. [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF].

W = C 10 (I 1 -3) + C 20 (I 1 -3) 2 + 1 D 1 (J -1) + 1 D 2 (J -1) 2 , (3)
The simplified Rubin-Bodner models used in Barbarino et al. [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF] and Weickenmeier et al.

[64] ignored in vivo facial skin anisotropic properties. Flynn et al. [START_REF] Flynn | Simulating the three-dimensional deformation of in vivo facial skin[END_REF] used a force-sensitive micro-robotic device to apply a rich deformation set to facial skin of five volunteers. In all cases, a nonlinear, anisotropic, and viscoelastic force-displacement response was recorded. They proposed a simplified FE model, with a second-order Ogden strain energy equation and a quasilinear viscoelastic relationship representing skin.

W = 2 i=1 µ i α i (λ αi 1 + λ αi 2 + λ αi 3 ) + 1 d i (J -1) 2 , (4)
where µ i is a material parameter with dimensions of stress and α i is a parameter controlling the nonlinearity of the response. d i is the bulk modulus. λ 1 , λ 2 , and λ 3 are the principal stretches.

T(t) = T e (t) + t 0 T e (t -τ ) ∂g R (τ ) ∂τ dτ, (5)
where T(t) is the total Cauchy stress tensor at t, T e is the elastic Cauchy stress tensor,

g R (t) = 1 -ḡP 1 (1 -e -t/τ G 1
) is the reduced relaxation function represented by a Prony series. ḡP 1 is a viscoelastic parameter and τ G 1 is the relaxation time. In-plane pre-stresses in two orthogonal directions were also applied to simulate the tension in living skin. Nonlinear optimisations generated material parameters and in vivo pre-stresses that best-fit the central cheek response of five volunteers and several points of one volunteer.

Recently, Flynn et al. [START_REF] Flynn | An anisotropic viscoelastic model of in vivo facial skin[END_REF] simulated the same data set using an anisotropic constitutive model [START_REF] Gasser | Hyperelastic modelling of arterial layers with distributed collagen fibre orientations[END_REF].

W = µ 2 (I 1 -1) + k 1 k 2 {e k2[ Īn(θ)-1] 2 -1}, (6)
where µ and k 1 control the stiffness of the material at low strains, and k 2 is a dimensionless parameter that controls the stiffness of the material at high strains. Ĩn (θ) = N CN is the fibre stretch squared of the nth family of fibres orientated in the direction N in the reference configuration. C is the deviatoric right Cauchy tensor.

The finite element model in the study was based on a single layer of shell elements. The constitutive relationship has previously been used to simulate the response of in vitro skin under uniaxial tension Annaidh et al. [START_REF] Annaidh | Automated Estimation of Collagen Fibre Dispersion in the Dermis and its Contribution to the Anisotropic Behaviour of Skin[END_REF]. The error between model and experiment was less than the error reported by Flynn et al. [START_REF] Flynn | Simulating the three-dimensional deformation of in vivo facial skin[END_REF]. This is expected with the use of a constitutive relationship that more accurately represents the fibre orientation than an isotropic relationship.

Facial muscle modelling

Facial muscles are striated compared to soft muscles, and activation induces contraction along fibres. Muscle simulations should include fibre directions and a model to generate contractions.

Fibre directions in face models are generated manually using anatomical data or fitting Bsplines on digitized fibre directions through a serial dissection of the muscle [START_REF] Kim | Model-based investigation of control and dynamics in human articulatory motion[END_REF][START_REF] Sifakis | Automatic determination of facial muscle activations from sparse motion capture marker data[END_REF][START_REF] Wu | Modelling facial expressions: A framework for simulating nonlinear soft tissue deformations using embedded 3D muscles[END_REF].

In FE models, muscle fibres can be aligned along element edges. The muscle force is a function of length and velocity difference between element edge nodes [START_REF] Buchaillard | A biomechanical model of cardinal vowel production: Muscle activations and the impact of gravity on tongue positioning[END_REF]. Fibres are also represented using link elements and using temperature to model activation. This approach models passive muscle stiffening [START_REF] Nazari | Shaping by Stiffening: A Modeling Study for Lips[END_REF][START_REF] Nazari | Simulation of dynamic orofacial movements using a constitutive law varying with muscle activation[END_REF].

Transversely isotropic relationships represent muscle passive properties [START_REF] Spencer | Constitutive theory for strongly anisotropic solids[END_REF]. The strain energy derivative with respect to the invariant I 4 = a 0 Ca 0 gives the stress along the fibres. a 0 is the unit vector along the initial fibre direction and C is the right Cauchy-Green strain tensor.

Adding this stress to the active muscle stress gives the total stress in the contracting muscle.

Muscle force generation models are typically of the Hill-type with the muscle force generated, F ce , given by

F ce = F max F (l, v)f A (7)
where F max is the maximum muscle force generated, F (l, v) = f l g(v) (with f l being the tensionlength and g(v) being the tension-velocity muscle characteristics), and f A represents activation dynamics from the central nervous system [START_REF] Zajac | Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control[END_REF].

An alternative to Hill-type models is Feldman's λ model [START_REF] Feldman | Once more on the equilibrium-point hypothesis (λ model) for motor control[END_REF]. Both approaches differ in two ways. Firstly, in a Hill-type model, muscle force is controlled by the maximum force variable.

In the λ model, muscle force is a consequence of the combination of the stretch reflex and the force-length characteristic of the muscle. Secondly, to assign values to the parameters of Hill-type models, experimental data is needed when the muscle is maximally activated. The λ model is based on experimental data from unloading tasks. The generated muscle force in the λ model is

F ce = F max F (A, l, v, d, t)g(v) (8)
where A is the muscle threshold length, d is the reflex delay. The current muscle force is dependent on the length and velocity at time t. This model simulates the nonlinear force-stiffness of muscles (see figure 6.5 in McMahon [START_REF] Mcmahon | Muscles, reflexes, and locomotion[END_REF]).

Ovesy et al. [START_REF] Ovesy | Equivalent linear damping characterization in linear and nonlinear force-stiffness muscle models[END_REF] showed the λ model had a more realistic response rate compared to the Hill-type model. In a study of orofacial gestures, no clear biomechanical differences were found between the two models [START_REF] Nazari | The Distributed Lambda (λ) Model (DLM): A 3-D, Finite-Element Muscle Model Based on Feldman's λ Model; Assessment of Orofacial Gestures[END_REF].

Finite element models

Early physically-based face models considered the face as a mass-spring lumped parameter system [START_REF] Terzopoulos | Physically-based facial modeling, analysis, and animation[END_REF]. Models since then have used FE methods to simulate the nonlinear finite deformations that facial tissues undergo. Many of the FE models described accurately represent the geometric characteristics of the hard and soft tissues. However, several simplifying assumptions are made with respect to the mechanical properties of the soft tissues. Several models assume the skin to be an isotropic material Barbarino et al. [4], Sifakis et al. [START_REF] Sifakis | Automatic determination of facial muscle activations from sparse motion capture marker data[END_REF], Warburton and Maddock [START_REF] Warburton | Physically-based forehead animation including wrinkles[END_REF], and Wu et al. [START_REF] Wu | Generating facial expressions using an anatomically accurate biomechanical model[END_REF].

Others assume skin to have a linear stress-strain response Beldie et al. [START_REF] Beldie | Finite element modelling of maxillofacial surgery and facial expressions -a preliminary study[END_REF], Chabanas et al. [START_REF] Chabanas | Patient specific finite element model of the face soft tissues for computer-assisted maxillofacial surgery[END_REF],

and Gladilin et al. [START_REF] Gladilin | Anatomy-and physics-based facial animation for craniofacial surgery simulations[END_REF]. None of the previously described models consider the tension inherent in living skin, which plays an important role in wrinkling Flynn and McCormack [START_REF] Flynn | Simulating the wrinkling and aging of skin with a multi-layer finite element model[END_REF] and influences the design of surgical incisions Borges [START_REF] Borges | Relaxed Skin Tension Lines (Rstl) Versus Other Skin Lines[END_REF]. To make useful predictions from biomechanical face models, it is important to represent skin as an anisotropic, nonlinear material under tension. The stiffening effect of muscles when under stress is also not accounted for in previous face models.

It is proposed that this effect plays a vital role in lip shaping, which has consequences for speech mechanics.

The remainder of this chapter describes our efforts to develop face models that contains an anisotropic skin model with pre-stress and includes muscles that stiffen under stress.

Face model development

General face model

The first face model was created using a patient's CT scans by generating the skin and skull surfaces respectively using the method described in Chabanas et al. [START_REF] Chabanas | Patient specific finite element model of the face soft tissues for computer-assisted maxillofacial surgery[END_REF]. The volume between, representing the facial tissue, was meshed manually to create three different layers with hexahedral and wedge elements [START_REF] Nazari | Simulation of dynamic orofacial movements using a constitutive law varying with muscle activation[END_REF]. The model was developed within ANSYS (ANSYS, Inc., Cecil Township, PA, USA) and contained 6030 hexahedral elements, 314 wedge elements, and 8736 nodes (figure 24.2). The wedge element formulation is a degenerated formulation from the hexahedral formulation. The mesh was created symmetrically with respect to mid-sagittal plane to reduce the number of degrees of freedom and thus reduce the computational time. We defined contact elements between upper and lower lips and between the lips and a surface simulating teeth. Attachment nodes on the skull were fixed. We defined muscle pathways with the help of anatomical data and CT scans. Muscles were defined as cable elements, which act in tension only. End points of the cable elements correspond to anatomical landmarks located in reference to the skull. This means that their definition is independent of the refinement of the soft tissue mesh. Point to surface contact elements coupled the cable elements with the main mesh. The end points of the cable elements are coupled to the surface of mesh elements whose centroid is closest to the cable extremity. A volume running along each cable element line with a radius equal to the muscle cross-sectional dimension is determined. Elements of the main mesh intersecting this volume is then labelled as a part of the muscle body. Simulations creating different facial gestures were produced. For a detailed discussion see Nazari et al. [START_REF] Nazari | Simulation of dynamic orofacial movements using a constitutive law varying with muscle activation[END_REF]. In Nazari et al. [START_REF] Nazari | Shaping by Stiffening: A Modeling Study for Lips[END_REF] the model's behaviour was verified as a first model producing lip protrusion and lip rounding. This model was used as a basis to create a patient specific model using mesh morphing operations [START_REF] Bucki | A fast and robust patient specific finite element mesh registration technique: application to 60 clinical cases[END_REF][START_REF] Bucki | Finite element speaker-specific face model generation for the study of speech production[END_REF]. The cranium, maxilla, and hyoid are represented as rigid bodies and fixed in space. The jaw is connected to the hyoid and muscles represented as point-to-point Hill-type actuators [START_REF] Stavness | Coupled hard-soft tissue simulation with contact and constraints applied to jaw-tongue-hyoid dynamics[END_REF].

Subject-specific model development

A finite element mesh, similar to that discussed in the previous section, represents soft tissue.

The outer and inner mesh surfaces were fit to corresponding surfaces of CT data of an adult male [START_REF] Bucki | Finite element speaker-specific face model generation for the study of speech production[END_REF]. The mesh has three element layers with the outer element layer representing skin, and inside layers representing the hypodermis.

We implemented the mimetic muscles in the same manner as described in the previous section (figure 24.4). Elements in the path of a muscle are assigned passive and active properties. The total muscle fibre response is based on a transverse-isotropic relationship proposed by Blemker et al. [START_REF] Blemker | A 3D model of muscle reveals the causes of nonuniform strains in the biceps brachii[END_REF]. The passive stress along the fibres increased exponentially with increasing fibre stretch [START_REF] Weiss | Finite element implementation of incompressible, transversely isotropic hyperelasticity RID B-6886-2008[END_REF]. This results in a stress stiffening effect. The same muscle material parameters were used.

λ * = 1.4 is the stretch at which the fibres are straightened, P 3 = 0.05 scales the exponential stresses, and P 4 = 6.6 is the rate of uncrimping of fibres. The maximum active fibre stress was 100 kPa. The zygomatic ligaments are represented by fixing the nodes in the soft tissue mesh closest to the region where these ligaments attach to the maxilla. Soft tissue nodes in the mandibular ligament attachment regions are fixed to the mandible.

Contact between soft tissue and bony structures is modelled, along with contact between upper and lower lip surfaces. Contact is based on interpenetration of the surface nodes. Velocity constraints are formed between interpenetrating nodes and opposing faces. The direction of the constraint is the normal of the opposing face closest to the interpenetrating node. These constraints are added to the system in subsequent time-steps to correct the interpenetration.

Soft-tissue constitutive equations

The hypodermis is represented by a Mooney-Rivlin constitutive equation.

W = C 10 ( Ī1 -3) + C 20 ( Ī1 -3) 2 + κ 2 (ln J) 2 , (9)
where C 10 = 0.4 kPa, C 20 = 1.4 kPa, and κ = 50 kPa are material parameters taken from [START_REF] Barbarino | Experimental and numerical study on the mechanical behavior of the superficial layers of the face[END_REF].

We used a frame-invariant Fung constitutive equation for the skin [START_REF] Ateshian | A frame-invariant formulation of Fung elasticity[END_REF].

W = c 2 (e Q -1) + κ 2 (ln J) 2 , (10)
Q = c -1 3 a=1 2µ a A 0 a : Ē2 + 3 b=1 λ ab (A 0 a : Ē)(A 0 b : Ē) , (11)
where c, µ a , λ ab are material parameters whose values are based on in vivo tests on forearm skin [START_REF] Flynn | Modeling the Mechanical Response of In Vivo Human Skin Under a Rich Set of Deformations[END_REF] (table 1).

Ē = ( FT F -I)
is the distortional Green-Lagrange strain tensor; A 0 a = a 0 a ⊗ a 0 a , where a 0 a defines an initial material axis direction a. For each soft tissue element, a 0 a was set to the relaxed skin tension line direction in that region (figure 24.4(a)) [START_REF] Borges | Relaxed Skin Tension Lines (Rstl) Versus Other Skin Lines[END_REF].

Implementing skin pre-stress

A novel aspect of Flynn et al. [START_REF] Flynn | A finite element model of the face including an orthotropic skin model under in vivo tension[END_REF] is the pre-stress in the soft tissue mesh. Prior to the FE analysis, all the nodal coordinates of the soft tissue mesh were scaled by dividing by a tension scale factor (TSF). At the start of the FE analysis, the globally-scaled soft-tissue mesh corresponded to the stress-free configuration. During the initial analysis step, displacement boundary conditions were imposed on all nodes of the inner surface of the soft-tissue mesh. These boundary conditions transformed these inner nodes back to their original reference positions. Any nodes on the inner surface, which were not attached to bony structures or ligaments, had their displacement boundary conditions removed. In place of these boundary conditions, the corresponding reaction forces were imposed. These reaction forces were reduced linearly to zero in 1 second. This was a more stable procedure than suddenly releasing the nodes. These operations created a pre-stress in the soft tissue. Three different TSFs were analysed. With a TSF=1.05, the pre-stress in the skin mesh averaged 2.5 kPa. A TSF=1.1 resulted in pre-stresses of 5.2 kPa and a TSF=1.15 resulted in pre-stresses of 9.9 kPa. The values of TSF were chosen to give pre-stress levels within estimated in vivo ranges.

Model evaluation

We evaluated the model by simulating several actions and comparing landmark displacements and distances between certain landmark pairs with corresponding in vivo measurements (figure 24.5(a)). Details of the experimental protocols can be found in Coulson et al. [START_REF] Coulson | Quantification of the three-dimensional displacement of normal facial movement[END_REF], Giovanoli et al. [START_REF] Giovanoli | Three-dimensional video-analysis of facial movements in healthy volunteers[END_REF], and Sforza et al. [START_REF] Sforza | The effect of age and sex on facial mimicry: a three-dimensional study in healthy adults[END_REF]. We simulated facial expressions by activating appropriate facial muscle sets (see Table 1 in [START_REF] Flynn | A finite element model of the face including an orthotropic skin model under in vivo tension[END_REF]). These expressions included open-mouth smile, closedmouth smile, pursing of lips, and lips turned downwards. Activation levels were chosen that resulted in realistic expressions. Wide mouth opening was simulated by activating the pterygoid and digastric muscles with 100% activation in each [START_REF] Stavness | Coupled hard-soft tissue simulation with contact and constraints applied to jaw-tongue-hyoid dynamics[END_REF]. We also investigated skin stiffness and pre-stress effects on landmark displacements during facial actions (table 1). These skin properties changes are similar to what is observed in ageing skin [START_REF] Albert | A review of the literature on the aging adult skull and face: Implications for forensic science research and applications[END_REF][START_REF] Batisse | Influence of age on the wrinkling capacities of skin[END_REF].

Results

General face model results

Different

Subject-specific model results

Displacement of inner soft tissue mesh nodes from their scaled positions to the reference subject position created a tension field where the direction of maximum tension in any region was similar to relaxed skin tension lines observed by Borges [START_REF] Borges | Relaxed Skin Tension Lines (Rstl) Versus Other Skin Lines[END_REF]. When the mouth was wide open, the directions of maximum tension changed in-plane by up to 90 o in the chin and anterior mandible regions. For an open-mouth smile, the cheek area experienced the largest tension direction changes [START_REF] Flynn | A finite element model of the face including an orthotropic skin model under in vivo tension[END_REF].

Activating appropriate muscles generated different facial expressions (figure 24.5(b)-(f)). For all facial expressions, the mouth corner displaced the most, as observed experimentally [START_REF] Coulson | Quantification of the three-dimensional displacement of normal facial movement[END_REF][START_REF] Giovanoli | Three-dimensional video-analysis of facial movements in healthy volunteers[END_REF][START_REF] Sforza | The effect of age and sex on facial mimicry: a three-dimensional study in healthy adults[END_REF]. For "normal" skin with a 10% TSF, model landmark displacements were within a standard deviation of experimental measurements (figure 24.7). Ferrario and Sforza [START_REF] Ferrario | Anatomy of emotion: a 3D study of facial mimicry[END_REF] reported errors in the method of Sforza et al. [START_REF] Sforza | The effect of age and sex on facial mimicry: a three-dimensional study in healthy adults[END_REF] tensions with a normal skin stiffness. Standard deviations are reported from [START_REF] Coulson | Quantification of the three-dimensional displacement of normal facial movement[END_REF][START_REF] Giovanoli | Three-dimensional video-analysis of facial movements in healthy volunteers[END_REF][START_REF] Sforza | The effect of age and sex on facial mimicry: a three-dimensional study in healthy adults[END_REF]. From Flynn et al. [START_REF] Flynn | A finite element model of the face including an orthotropic skin model under in vivo tension[END_REF].

Copyright 2015 by Taylor & Francis Ltd. Adapted with permission.

Conclusion

Creating physically-realistic facial simulations requires us to overcome several engineering challenges. There are many complex interactions between hard and soft tissues. Most materials are nonlinear, anisotropic, and time-dependent. Simulating how muscles drive facial expressions is a difficult undertaking. There are diverse benefits to overcoming these challenges, such as improved maxillofacial surgical planning, better insights into speech production, and superior animation quality.

In this chapter, we have outlined several computational tissue, muscle, and full face models in the literature. We have also demonstrated our approach for simulating human facial expressions.

Results from the models have indicated good agreement with experimental measurements. The addition of an orthotropic constitutive equation to represent skin and the implementation of a pre-stress are advancements over previous models. Accounting for the stress-stiffening effect in muscles, which has not been considered in earlier models, results in more realistic deformations, especially in the lip region.

There is much work to do to advance the state of the art. More efficient computational techniques are needed to increase the simulation speed. This would increase their usefulness to predict patient outcome from maxillofacial surgery. Additional soft-tissue characterisation of specific facial regions will improve model behaviour. Measuring mimetic muscle activations during facial expressions and lip movements will inform the face model. A limitation is that the finite element model of the face does not correspond to any of the faces used for the in vivo landmark measurements. Realistic connections between soft-tissue layers and between soft and bony structures needs to be implemented.

Figure 24 . 1 :

 241 Figure 24.1: Face muscles. Note that the buccinator and risorius muscles are not indicated in this sketch. From [40]. Adapted under licence.

  m 4 is a dimensionless constant that controls the nonlinearity of the response to fibre stretching. The McAuley brackets • = (| • | + •)/2 ensure no resistance to compression in fibres.

where C 10 ,

 10 C 20 , D 1 , and D 2 are material parameters, and I 1 = B • I. They chose this model because it is commonly implemented in commercial finite element software. They used a Cutometer and an aspiration device to get measurements in five facial regions -the forehead, zygomatic, parotid, nasolabial, and jaw. The 2 mm diameter aperture on the Cutometer was used to characterise the superficial skin layer. The larger 8 mm aperture on the aspiration device deformed the deeper layers to characterise the SMAS and fat layers. They developed an axisymmetric finite element model consisting of a 1.7 mm thick skin layer and a 3 mm layer representing both the SMAS and fat. They identified material parameters that best-fit the Rubin-Bodner model and the second order reduced polynomial model to the averaged response of the five facial points.

  Sifakis et al.[START_REF] Sifakis | Automatic determination of facial muscle activations from sparse motion capture marker data[END_REF] developed a FE model with a rigid surface mesh for bone, a tetrahedral mesh for soft tissue, and embedded vectors representing muscle fibres. A Mooney-Rivlin equation represented the soft tissue. They simulated facial expressions and narration sequences and determined muscle activations that best-fit model surface-points to corresponding experimentally tracked surface markers for a given action. Gladilin et al.[START_REF] Gladilin | Anatomy-and physics-based facial animation for craniofacial surgery simulations[END_REF] developed a FE model for craniofacial surgery applications. It comprised a linear elastic soft tissue mesh (Young's modulus 2.2 MPa, Poisson's ratio 0.45) and muscles represented virtually by a vector field inside a 3-D ellipsoid. Pre-computed muscle forces simulated happiness and disgust. Beldie et al. [7] simulated facial expressions pre and post maxillofacial surgery. The model contained rigid bone, skin, fat tissue, and 20 facial muscles. The latter were represented using a Hill-type muscle model. A linear elastic model represented skin (Young's modulus 15 kPa, Poisson's ratio 0.49) and a neo-Hookean material represented the fat (C 10 = 420 Pa, Poisson's ratio of 0.495). Chabanas et al. [13] developed several patient-specific models to simulate mandible repositioning. They divided the soft tissue into two layers representing the dermis and hypodermis, which were simulated using a linear-elastic model (Young's modulus 15 kPa, Poisson's ratio 0.49). Barbarino et al. [4] developed a very detailed FE model based on MRI data. The skull and mandible were represented along with retaining ligaments, skin, a combined SMAS and hypodermis layer, and deeper subcutaneous tissue. Eleven passive facial muscles were present. The simplified Rubin-Bodner model (Equations (1) and (2)) was used to represent soft tissues. They modelled the insertion of a rigid ball inside the oral cavity between the teeth and cheek. They also simulated ageing by reducing tissue stiffness. Another detailed face model has been presented by Wu et al. [68]. Their cubic-hermite FE model comprised a combined skin, subcutaneous and SMAS layer, and an underlying deep fascia layer. Twenty transversely-isotropic muscle pairs were embedded in the mesh to simulate facial expressions. Soft tissues and passive muscle properties were represented by a Mooney-Rivlin equation (C 10 = 0.42 kPa -2.50 kPa; C 20 = 0.0 kPa -1.175 kPa). The deep fascia layer was considered as a rigid surface mesh and the superficial layer glided upon it. The authors found more realistic expressions resulted from this glide-plane. They evaluated four simulated facial expressions by comparing surface data with measurements obtained using a 3D structured-light scanner. The largest differences occurred in the lip region. Recently, Warburton and Maddock [63] presented an explicit finite element forehead model comprising stratum corneum, dermis, and hypodermis. The frontalis, procerus, and corrugator supercilli muscles were also present. Soft tissue components were represented by a neo-Hookean model (C 10 = 0.034 MPa -48 MPa depending on the tissue). The active and passive components of the transversely isotropic were summed. The model efficiently simulated soft tissue sliding over the skull and created physically-realistic wrinkles.

Figure 24 . 2 :

 242 Figure 24.2: General face model: wedge elements are yellow and hexahedral elements are blue; From Nazari et al. [45]. Copyright 2011 by Human Kinetics. Adapted with permission.

Finite element model

  Starting from the finite element mesh developed with ANSYS, this model was further developed within Artisynth (www.artisynth.org), an open source biomechanical modelling platform developed at the University of British Columbia (figure24.3)[START_REF] Lloyd | ArtiSynth: a fast interactive biomechanical modeling toolkit combining multibody and finite element simulation[END_REF].

Figure 24 . 3 :

 243 Figure 24.3: Subject-specific face model: (a) soft tissue mesh; (b) bones, jaw muscles, and ligament attachment From Flynn et al. [24]. Copyright 2015 by Taylor & Francis Ltd. Adapted with permission.

Figure 24 . 4 :

 244 Figure 24.4: (a) Orofacial muscles including element muscle fibres; (b) relaxed skin tension line direction assigned to each element; From Flynn et al. [24]. Copyright 2015 by Taylor & Francis Ltd. Adapted with permission.

Figure 24 . 5 :

 245 Figure 24.5: (a) Facial landmarks; (b) Wide open mouth; (c) open-mouth smile; (d) closed-mouth smile; (e) lips turned downwards; (f) lips pursed. From Flynn et al. [24]. Copyright 2015 by Taylor & Francis Ltd. Adapted with permission.

  facial gestures were produced by activating appropriate muscles in the general face model. The resulting gestures are consistent with those produced by the corresponding muscles reported in the anatomical references. The computational time for each simulation was about 35 minutes on a 2.6GHz Core 2 Duo processor.Focusing on lip protrusion and lip rounding, better lip shaping occurs when we stiffen facial tissues[START_REF] Nazari | Simulation of dynamic orofacial movements using a constitutive law varying with muscle activation[END_REF] (figure24.6). This is due to the stress stiffening effect of muscle fibres when they generate contraction forces along their length. In early model versions, fibre numbers were small (maximum two per muscle). The stiffening effect was introduced in the model through the change of passive properties during activation (the coefficients in Mooney-Rivlin hyperelastic model).

Figure 24 . 6 :

 246 Figure 24.6: The effect of orbicularis oris muscle activation: (a) with stress stiffening, a more realsitic effect occurs; (b) with no stress stiffening, the lips protrude excessively.

Figure 24 . 7 :

 247 Figure 24.7: Variation of landmark displacements for skin with different (a) stiffnesses with a TSF of 10%; (b)

Table 1 :

 1 Material parameters for different skin-types. TSF stands for Tension Scale Factor. From[START_REF] Flynn | A finite element model of the face including an orthotropic skin model under in vivo tension[END_REF]. Copyright by Taylor & Francis Ltd. Adapted with permission.Skin type c (kPa) µ 1 (kP a) µ 2 (kPa) µ 3 (kPa) λ 11 (kPa) λ 22 (kPa) TSF

	Normal	21.3	17.8	5.9	5.9	1.0	1.0	1.10
	Stiff	42.6	35.6	11.8	11.8	11.8	2.0	1.05
	Soft	10.7	8.9	3.0	3.0	0.5	0.5	1.15

  ranging from 0.5-3.38 mm. In general, increasing skin stiffness resulted in smaller landmark displacements during facial expression. Increasing tension in the soft tissue had a variable effect on landmark displacements. When smiling, central upper lip displacement increased with increasing tension but decreased with increasing tension when lips pursed or turned downwards.

		20		Open Mouth Smile	Soft			20	Open Mouth Smile	5% TSF
	Landmark Displacement (mm)	5 10 15			Normal Stiff Coulson et al [14] Sforza et al [55] Giovanoli et al [29]	Landmark Displacement (mm)	5 10 15	10% TSF 15% TSF Coulson et al [14] Sforza et al [55] Giovanoli et al [29]
		0						0
		PG	PH	RAN RMC RML RMU ROR	SL		PG	PH	RAN RMC RML RMU ROR	SL
				Landmark					Landmark


Camille Marchet

Lolita Lecompte

Corinne Da Silva

Corinne Cruaud

Jean-Marc Aury

Jacques Nicolas

Pierre Peterlongo

CARNAC-LR : Clustering coefficient-based Acquisition of RNA Communities in Long Reads

come

1 Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA) -Universite de Rennes 1, Institut National de Recherche en Informatique et en Automatique -Avenue du général LeclercCampus de Beaulieu 35042 RENNES CEDEX, France 2 Genoscope-Centre national de séquençage (GENOSCOPE) -CEA -Genoscope-Centre National de Séquençage 2, rue Gaston Crémieux CP5706 91057 EVRY Cedex, France

Résumé Motivation

Lately, long read sequencing technologies, referred to as Third Generation Sequencing, (TGS, Pacific Bioscience [1] and Nanopore [2]) have brought the opportunity to sequence full-length RNA molecules. In doing so they relax the constraint of transcript reconstruction prior to study complete RNA transcripts. By avoiding limitations of previous technologies, [3,4] and giving access to the trancripts structure, they might contribute to complement and improve transcriptomes studies. This is particularly crucial for non model species where assembly was required. Many biological questions (finding gene signatures for a trait, finding expressed variants...) [5,6] are classically addressed using transcriptome sequencing. However, this gain in length is at the cost of a computationally challenging error rate (up to 15%) that disqualifies previous short-reads methods. In this work we propose to support the analysis of RNA long read sequencing with a clustering method that works at the gene level. It enables to group transcripts that emerged from a same gene. From the clusters, the expression of each gene is obtained and related transcripts are identified, even when no reference is available.

Problem statement

Within a long reads data set, our goal is to identify for each gene the whole set of reads that come from its expression without the help of a reference genome or transcriptome. This problem can be computationally formalized as a community detection problem, where a community (also referred to as a cluster) is the population of reads coming from a same gene. Communities are densely connected groups of nodes, although there exists no rigorous shared definition. Our application problem is non trivial and specific for three reasons:

1-in eukaryotes, it is common that alternative spliced and transcriptional variants (called isoforms) which differ in exon content occur for a given gene. In this case we want alternative transcripts to be grouped in a same cluster; * Intervenant † Auteur correspondant: camille.marchet@irisa.fr sciencesconf.org:jobim2018:211641 2-long reads currently suffer from high error rates and computationally challenging error profiles with a majority of indels errors; 3-all genes are not expressed at the same level in the cell, which leads to an heterogeneous coverage in reads of the different genes, then to communities of different sizes including small ones. This can be a hurdle for community detection.

Previous works

The problem in itself is not new, it dates back before the advent of NGS, with Sanger sequencing and the necessity to cluster ESTs. However these methods were tailored to work with lower scalabilty challenges due to the scarcity of data, and a far less important error rate than with current long reads. The concept of community detection is a natural way of depicting our problem. Due to the ambiguity of the community definition, a plethora of methods have been proposed for their detection. Moreover this problem has appeared in many disciplines, taking many slightly different forms according to the application domain. The first approach that brought an important contribution is an algorithm based on modularity. Other methods were then proposed as improvements, in particular methods relaxing the definition of communities as objects that can overlap, such as the Clique Percolation Method (CPM) [7].

Contribution

Roughly speaking, resolution strategies can be classified into two trends according to applications and the community of affiliation: a graph clustering strategy based on the search for minimal cuts in these graphs and a community finding strategy based on the search for dense subgraphs. Our own approach aims to combine the best of both worlds. The first approach generally searches for a partition into a fixed number of clusters by deleting a minimum number of links that are supposed to be incorrect in the graph. The second approach frequently uses a modularity criterion to measure the link density and decide whether overlapping clusters exist, without a priori regarding the number of clusters. Given that it is difficult to decide on the right number of clusters and to form them solely on the basis of minimizing potentially erroneous links, the main findings and recent developments are based on the community finding strategy and we will focus our review on this approach. Our algorithm is based on the concept of clustering coefficient and we formalize our problem as finding communities such that a community is a connected component in the graph of similarity having a clustering coefficient above a fixed cutoff, and such that communities are disjoint sets. An optimal clustering in k communities is a minimal k-cut of the graph, that is, a set of k disjoint subsets of reads such that the set of edges between two different subsets has minimal size. We then implement heuristics that approximate a result of this problem. They are implemented in a tool dubbed CARNAC-LR (Clustering coefficient-based Acquisition of RNA Communities in Long Reads), integrated into a pipeline. The input is a set of long reads and the output is a file with reads indexes grouped in one line per cluster. Our approach is compared to state of the art algorithms to detect communities. We then show its relevance on a real data set issued from mouse brain transcriptome.

Results

We use a real mouse dataset sequenced using a MinION platform at the Genoscope to compare our solution to other algorithms used in the context of biological clustering and demonstrate its is better-suited for transcriptomics long reads. We build "ground truth" clusters using mapping routine that are compared to de novo clustering results. We use them to benchmark classic community detection algorithms and state of the art sequence clustering tools such as CD-HIT [8] and show we perform better on ONT reads from mouse. When a reference is available thus mapping possible, we show that it stands as an alternative