domain: spi phys

text:
Raphaël Perrier

Loïc Tadrist

Jean-Marc Linares

Damage resilience of manufactured and biological actuators

Keywords: Resilience, Actuator, Bio-inspiration, Damaged actuator, Bio-inspired muscle, Bio-inspired pulvinus

Designing resilient actuators is a challenge for industry, in part because an index for resilience has yet to be established. In this work, several definitions of resilience are analysed and, on the basis of this, an index quantifying resilience for actuators is proposed. This index does indeed allow for the resilience computation of a wide range of manufactured and biological actuators to be compared. The two manufactured actuators chosen as iconic models are a hydraulic cylinder and a bio-inspired McKibben muscle, and these are shown not to be resilient by design. In addition, two biological actuators likely to be resilient were also analysed. The pulvinus resilience index shows that it is partly resilient depending on damage location. But the most promising is the skeletal muscle, which has been shown to be highly resilient. Finally, the bio-inspired roots of resilience are discussed: resilience may originate from multi-scale structural design.

Introduction

In industrial machines, input motion is created by an actuator, which converts non-mechanical energy (chemical, electric, hydraulic, thermal, etc.) into mechanical power. Thus, when an actuator fails in an industrial setting, this jeopardises the functioning of the mechanism. To secure critical motions, the current industrial standard is to duplicate crucial parts to avoid catastrophic failure. This is particularly true for drones and helicopters where flying commands that orient blades are duplicated [START_REF] Zammali | Communication integrity for future helicopters flight control systems[END_REF]. This strategy is called redundancy and comes with drawbacks. Redundant systems are heavier, more expensive, less compact, and consume resources and energy. In the context of sustainable development, redundancy is no longer a gold standard and new design strategies are emerging. One of the most recent is the digital twin that helps predict risks of catastrophic failure [START_REF] Lu | From bim towards digital twin: strategy and future development for smart asset management[END_REF].

Another option to secure critical motions is to rely on resilient-by-design actuators [START_REF] Malshe | Biological strategies from natural structures for resilience in manufacturing[END_REF]. Resilience is a trendy concept that is not well defined in the context of industrial systems. It conveys the idea that the system may resist disturbances without catastrophic loss of performance. To define resilience of an industrial system, one may leverage existing definitions of resilience from different research fields. Table 1 summarises a number of definitions found in science and engineering, together with their different nuances. Some of the definitions focus on resistance to, mitigation of, external disturbances whereas others deal with recovery after a disturbance. The latter definition matches the concept of self-healing material in engineering [START_REF] Terryn | A review on self-healing polymers for soft robotics[END_REF] which is not considered in this analysis. Here, the focus is set on resilience as resistance to, mitigation of, external disturbances. Despite numerous definitions found in literature, none of them was given specifically for industrial systems or actuators. The definition given in computer sciences (Table 1, line 2) may be adapted to define mechanical resilience as the faculty of an actuator to maintain its motion capabilities when disturbed or damaged (and respectively a system for its function).

With regards to resilience, this definition may be used to optimise the design of a new class of actuators. Design for resilience has not been investigated much by industry. To create resilient actuators, an efficient method is to leverage those that can be found in Nature. Among the most promising is the skeletal muscle. It allows the motion of mammals, fish and birds (vertebrates) even when it is slightly damaged. In fact, most athletes notice that during training, muscle damage is barely felt and performance remain constant. It is after a period of rest, however, that muscle soreness then appears. This delayed onset of muscle soreness corresponds to micro-damage within the muscle (Myofibril breaks and accumulation of lactic acid) [START_REF] Nosaka | Muscle soreness and damage and the repeated-bout effect[END_REF]. In

Field

Definition

Ref.

Oxford dictionary

(1) The ability of people or things to recover quickly after something unpleasant, such as shock, injury etc.

(2) The ability of a substance to return to its original shape after it has been bent, stretched or pressed.

Computer sciences

(1) Continue to operate in presence of compromised nodes, and endure and overcome internal attacks.

[7]

(2) Maintain functions and structure in the face of internal and external change.

[2]

Resilience science

The extent to which a system can maintain its structure, function, and identity in the face of disturbance.

[17]

Engineering resilience Ability of systems to anticipate and adapt to the potential for surprise and failure.

[14]

Civil engineering

Capability to recover from external stresses to an equilibrium.

[3]

Manufacturing systems

The intrinsic property of a system to resist, and/or recover from, and/or adapt to a new and improved state in a time much smaller than the overall lifetime of the system, under known and unknown stochastic or random perturbation(s).

[20]

Biology

The ability to quickly and completely recover after deviation from normal physiological state or damage caused by a stressor or an adverse health event.

[30]

Economy

The ability of an economy or a society to minimize welfare losses for a disaster of a given magnitude.

[11]

Supply Chain

The adaptive capability of a supply chain to prepare for and/or respond to disruptions, to make a timely and cost effective recovery, and therefore progress to a post-disruption state of operations -ideally, a better state than prior to the disruption.

[29]

Table 1: Definitions of resilience in science and engineering.

2

this sense, muscle is resilient given that even when damaged an athlete is able to continue training without a catastrophic loss of performance. In a similar vein, plants also exhibit versatile actuators likely to be resilient. Pulvini move plants leaves to follow the sun's path and avoid herbivores. Early observations show that even when a significant part of the pulvinus is excised, the leaf motion still occurs [START_REF] Millet | The rhythmic leaf movements after regeneration of partially excised pulvinus in phaseolus vulgaris l[END_REF]. This shows that pulvinus is also a resilient actuator.

In these two biological examples, resilience should emerge from a combination of structural and control properties. Resilience by command (software) has been successfully conceptualized and mimicked numerically [START_REF] Al-Shuka | Active impedance control of bioinspired motion robotic manipulators: An overview[END_REF][START_REF] Cully | Robots that can adapt like animals[END_REF][START_REF] Koos | Fast damage recovery in robotics with the t-resilience algorithm[END_REF][START_REF] Zhao | Reliable state feedback control system design against actuator failures[END_REF][START_REF] Zykov | Evolving dynamic gaits on a physical robot[END_REF]. Bio-inspired algorithms based on artificial intelligence, have been implemented for multiactuator systems [START_REF] Cully | Robots that can adapt like animals[END_REF].

However, little attention has been paid to the structural properties of these biological actuators that may favour resilience. To address this, firstly an index quantifying resilience to damage for actuators is proposed and discussed. Secondly, this index is then applied to a hydraulic cylinder, a skeletal muscle, a pulvinus and a McKibben bio-inspired muscle. These applications result in efficient descriptions of the resilience of manufactured and biological actuators. The index further shows that loss of performance is dependent on where the damage is located. This allows for a definition of the global resilience of a system or an actuator to be given. Lastly, practical considerations are discussed, specifically bio-inspired design laws to create a resilient industrial actuator. We aim to define resilience to damage, χ, with respect to the motion capability of an actuator for a given level of damage. Moreover, such a definition must suit a large variety of actuators and systems, from biology to industry.

An index to quantify resilience

The motion of an actuator or function of a system may be described by different quantities, such as maximal stroke, velocity of actuation, maximal strength, maximal instantaneous power, yield, etc. In fact, a different measure of motion capability may be given depending on the targeted motion. For the sake of generality, the power produced by an actuator during a typical cycle, cycle f (t) v(t) dt is considered (where f (t) and v(t) are the time variations of force and velocity during the actuation cycle).

The definition of resilience must also take into account damage level. Here, again, the definition of damage level must be clarified. The damage of a system or an actuator may be described by different quantities such as crack depth, volume of plastic deformation, loss of material, loss of internal fluid, corrosion, fatigue, wear, etc. Although, an index of damage may be created for each of these quantities, it could not be applied to every type of actuator. For instance, material corrosion may be useful to describe the ageing of the piston of a hydraulic cylinder, but it cannot be applied to skeletal muscle made of liquid and soft solid. For the sake of generality, no specific damage was considered in particular, but rather all types of damage were gathered within a single quantity: the damage volume V * . Specifically, the ratio between the damage volume, V * , and the total volume, V , is considered to represent the damage level, η = V * /V . With these two conditions, one may propose the following expression for resilience to damage of a single actuator,

χ = cycle f * (t)v * (t)dt cycle f (t)v(t)dt 1 1 -η . (1)
For an undisturbed actuator, χ is 1. When damaged, the resilience of an actuator may be greater than one, indicating that the damage volume has little influence on the actuator's capabilities. Indeed, this case would represent oversized actuators. Conversely, minor damage involving catastrophic loss of performance will generate extremely low scores of resilience (χ → 0). χ = 0 indicating an out-of-order actuator.

This definition stands for a single actuator and may not be applied directly for a system combining a number of actuators. Specifically, to define resilience for systems made of redundant actuators, one may first define redundancy,

R = N n , ( 2
)
where n is the minimum number of actuators needed to perform the motion and N the total number of actuators in the system. With this definition, the resilience of a system reads,

χ s = Rχ . (3)
The definitions above of resilience directly show that redundancy is an efficient way to enhance the resilience of a system. Indeed, χ s may be much larger than χ if the redundancy is large.

To show the broad spectrum of this definition of resilience, four reference cases are presented: two manufactured actuators -a hydraulic cylinder (conventional industrial actuator), and the McKibben muscle (bio-inspired actuator); and two biological actuators, -the biceps (skeletal muscle) and the pulvinus (plant muscle), see figure 1.

Computing resilience of four reference actuators

The hydraulic cylinder (industrial actuator)

The index of resilience is computed on an industrial actuator, a simple effect hydraulic cylinder, figure 1c. This actuation technology is widespread in industry, for instance for hydraulic bike brakes, glasshouse window and trunk openers, lifting operations, etc. Now, a simple hydraulic car jack is considered, figure 2. This car jack is composed of a steel body of thickness w, length L and diameter d, a piston fitted with a toric seal, and a pressure source feeding through an input pipe (length l, diameter e). Experience shows that such hydraulic cylinders may deteriorate due to wear of the toric seal [START_REF] Shanbhag | Failure monitoring and predictive maintenance of hydraulic cylinder-state-of-the-art review[END_REF]. This triggers a leak from the seal, which is idealized as a cylinder of diameter a, cross-section πa 2 and depth w. The volume of this small cylinder represents the damaged volume V * = πa 2 w. The power of the hydraulic cylinder f v (force times velocity) is given by the power of the hydraulic feeding P Q (pressure times flow rate),

f v = P Q . (4)
An oil flow rate, Q, is fed from the constant pressure source through the input pipe and a flow rate Q l is lost by the leak. Thus both should flow in a viscous regime (low Reynolds number, Re < 1),

Q = πP e 4 128νl and Q l = πP a 4 128νw . (5)
The damage level η may be computed with reference to the total volume of the hydraulic cylinder, V = πd 2 L. This leads to the expression of the typical leak dimension,

η = a 2 w d 2 L leading to a 2 = η d 2 L w . (6)
One can now express the resilience of the hydraulic cylinder, The force-velocity relationship of the hydraulic cylinder is plotted in figure 2a. In log-log scale, the damaged force-velocity curve shifts down (black to red curve). For a given damage level, the available power is changed by the same factor for any force.

χ = f * v * f v 1 1 -η = P (Q -Q l ) P Q 1 1 -η = 1 1 -η 1 -η 2 d 4 L 2 l e 4 w 3 . ( 7
)
When the hydraulic resistance of the leak is similar to the one of the input pipe, the resilience of the hydraulic cylinder collapses and reaches 0 (figure 2a). Interestingly, resilience is independent of the force or velocity of the motion. This is a small artefact of this model, where the pressure source is considered ideal and the fluid incompressible. Here, with P = 10 5 Pa, d =30 mm, e =3 mm, l =10 mm, ν = 1, w =5 mm and L =140 mm, χ = 0 corresponds to a damage level η = 0.0013. This poor resilience to damage shows that hydraulic cylinders are sensitive to wear. Therefore this situation requires monitoring and maintenance operations. Prudently, to avoid such catastrophic failures, most hydraulic cylinders are designed with security margins. The hydraulic cylinder has an extremely low resilience (null when damage is larger than 0.0013). For visualisation purposes, the damage level is shown between 0 and 0.002.

Biological actuators 3.2.1. The biceps (skeletal muscle)

Mammals, birds and fish (phylogenetically distant animals) rely on skeletal muscles to move, hunt or escape predators. The muscle structure for these three animal classes is quite similar [START_REF] Luther | Evolution of myosin filament arrangements in vertebrate skeletal muscle[END_REF], indicating that it has been preserved through evolution. This evolutionary history suggests that skeletal muscle may be resilient. Furthermore, intense athletic training induces delayed onset of muscle soreness (DOMS) although muscle performances are barely altered during training [START_REF] Nosaka | Muscle soreness and damage and the repeated-bout effect[END_REF]. Muscle soreness corresponds to microtrauma of muscle fibres, specifically on the Z-layer (junction between 2 parts of the sarcomere) of sarcomeres and conjunctive tissues. This performance preservation illustrates the resilience of skeletal muscles.

The skeletal muscle is a highly-structured actuator with complex feedback-loop commands [START_REF] Gillies | Structure and function of the skeletal muscle extracellular matrix[END_REF]. It is composed of tendons and fascicules at the macro-scale. Fascicules gather parallel muscle fibres (diameter 10 -100 µm). Muscle fibres are made of myofibril, which are, in turn, made of a series of elementary micro-scale bricks: the sarcomeres (length, at rest: 3.5 µm; contracted: 2.5 µm). Thus, from an engineering point of view, muscle is idealized as a sequence of series and parallel sarcomeres, Figure 1c.

Hill's seminal work [START_REF] Hill | The heat of shortening and the dynamic constants of muscle[END_REF], which gathered experimental data on frog legs, made it possible to establish the force-velocity relationship to be read as [START_REF] Cohen | Weightlifting and the actomyosin cycle[END_REF],

f f 0 = 1 -|v|/v 0 1 + α|v|/v 0 or |v| v 0 = 1 -f /f 0 1 + αf /f 0 . ( 8
)
f 0 is the maximal force the muscle can produce. Its micro-mechanical meaning is the sum of parallel forces of single sarcomeres, considering the number of myofibres in parallel N p , and the force generated by a sarcomere s, f 0 = N p s. v 0 is the maximal velocity the muscle can actuate. Its micro-mechanical meaning is the sum of series of velocities of single sarcomeres within the fibers, considering the number of sarcomeres in series N s , and the intrinsic velocity of a sarcomere u, v 0 = N s u. The parameter α is the ratio between bio-chemical constants related to actin-myosin bound, independent of the number of sarcomeres.

Little work has been done to link muscle damage to motion performance. Leveraging on Hill's equation, Eq [START_REF] Furnémont | Novel control strategy for the+ spea: a redundant actuator with reconfigurable parallel elements[END_REF], and fatigue (chemical inhibition) data from Greenberg and Moore [START_REF] Greenberg | The molecular basis of frictional loads in the in vitro motility assay with applications to the study of the loaded mechanochemistry of molecular motors[END_REF], one can compute the forcevelocity curve in standard and inhibited conditions, see figure 3a. The chemical inhibition is, here, considered as a model for damage.

It is assumed that only the number of active sarcomeres is changed by the chemical inhibition (N p → N * p , N s → N * s ), and that the unitary force s and velocity u are unchanged

f * 0 = N * p s = f 0 N * p /N p and v * 0 = N * s u = v 0 N * s /N s .
f * f 0 = N * p N p 1 -(|v * |/v 0 )(N s /N * s ) 1 + α(|v * |/v 0 )(N s /N * s ) . (9)
a.

b. It is then assumed that the damage rates are equal for series and parallel sarcomeres N * p /N p = N * s /N s . Hill's equation with damage reads,

f * f 0 = (N * s /N s ) -(|v * |/v 0 ) 1 + α(|v * |/v 0 )(N s /N * s ) or |v * | v 0 = N * s /N s -f * /f 0 1 + α(f * /f )(N s /N * s ) . ( 10
)
This equation is fitted (least mean squares) on Greenberg data varying N * s /N s and α. The parameters were found to be N * s /N s = 0.47 and α = 2.04. The fitted Hill's curves are plotted in figure 3a. From this modelling and fitted values, we consider a simple actuation cycle: (i) zero velocity and increasing force from 0 to f 1 ; isometric contraction, (ii) constant force f 1 and increasing velocity to v * , eccentric contraction; (iii) constant velocity -v * and decreasing force to 0, concentric contraction; and (iv) zero force and decreasing velocity to 0, with concentric contraction. The dimensionless powers P/(f 0 v 0 ) and P * /(f 0 v 0 ) of the cycle read,

P f 0 v 0 = cycle f 1 f 0 |v 1 | v 0 dt = 1 -(f 1 /f 0 ) 1 + α(f 1 /f 0 ) f 1 f 0 and P * f 0 v 0 = cycle f 1 f 0 |v * 1 | v 0 dt = (N * s /N s ) -(f 1 /f 0 ) 1 + α(f 1 /f 0 )(N s /N * s ) f 1 f 0 . (11)
From the powers over a cycle and the damage volume η = N * s /N s , the resilience of the muscle reads,

χ = 1 1 -η η -f 1 /f 0 1 -f 1 /f 0 1 + αf 1 /f 0 1 + αηf 1 /f 0 . ( 12
)
This expression of skeletal muscle resilience depends on the level of damage η and the required level of force f . The variations of resilience depending on these two parameters is drawn in figure 3b. Resilience is close to one for low forces and low velocities. We also notice that for large forces or large velocities, resilience decreases strongly with level of damage.

This model shows that the damage rate at the sarcomere level changes the macroscopic behaviour of the muscle and its resilience.

The pulvinus (plant muscle)

A pulvinus is a thickening found at the base of leaves of many plants from the Fabaceae and Oxilaceae families. It allows reversible motions of leaves, for instance closure at night, or response to touch (Mimosa Pudica), figure 1a. Its functions vary from species to species which include orienting leaves toward the sun, protecting from herbivore attacks (Mimosa Pudica) and allowing the rain to reach the ground (Samanea Saman). Similarly to skeletal muscle, it is found in distant species on the phylogenetical tree, which indicates its robustness [START_REF] Tadrist | Deformation and mechanics of a pulvinusinspired material[END_REF]. Since Dutrochet (1837) it is also known that ablating half of the pulvinus does not prevent the leaves from moving [START_REF] Dutrochet | Mémoires pour servir à l'histoire anatomique et physiologique des végétaux et des animaux[END_REF], which point to resilient behaviour.

The pulvinus from Mimosa Pudica is composed of 3 main structural elements organised in a cylindrical geometry, figure 1a (inset). At the centre are the vascular tissues (xylem and phloem) that provide water and extract sap. These hard tissues are surrounded by thin-walled parenchyma cells also called motor cells. A thin cuticle protects the organ from evapo-transpiration. The motor cells (diameter 25 µm) are able to expand (extensor) or to shrink (flexor) when turgor pressure is modified. This local deformation creates motion. a.

b. No theoretical force-velocity relationship exists for pulvinus in literature. However, leveraging the work of Millet et al. [START_REF] Millet | The rhythmic leaf movements after regeneration of partially excised pulvinus in phaseolus vulgaris l[END_REF] which reports on the evolution of the angle of inclination θ of Phaseolus vulgaris leaves for sound and ablated pulvinus, one can compute pulvinus resilience, see figure 4. In the figure, black points correspond to the control experiment: a sound pulvinus that follows a circadian motion, figure 4a andb. Red points correspond to the tests, excising around 30% of the pulvinus (Excised flexor, figure 4a. Excised extensor, figure 4b). The pulvinus is excised on the first day (black arrow) and its evolution over two days after excision is shown.

In the following, the first day after excision (grey area) is studied to compute resilience to damage. The second day is not studied not to consider self-healing and regeneration processes.

From this data, the evolution of the angle of inclination θ(t) on day one (circadian cycle) is fitted with two half of Gaussians for smoothing purposes, figure 5a (Excised flexor) and c (Excised extensor). The evolution of force is reconstructed considering that the force is proportional to f ∝ θ -θ 0 , where θ 0 is the minimal angle on the circadian cycle. This corresponds to acting on the lever arm to move the leaf. The velocity is also reconstructed as v ∝ dθ/dt. Those two quantities are plotted in figure 5b (Excised flexor) and d (Excised extensor) and form quasicycles (some growth occurs).

Based on these quantities, the resilience index, equation ( 1), is computed. The resilience of each excision experiment can be estimated, by comparing black to red areas which correspond to the power over a cycle. The damage level is estimated at 30 % of the total volume. On the first day, the resilience of the extensor (excised flexor) and the flexor (excised extensor) are, χ extensor = 0.25 and χ f lexor = 0.05 [START_REF] Hill | The heat of shortening and the dynamic constants of muscle[END_REF] The resilience of the extensor is much larger than that of the flexor. This shows that the proposed index of resilience quantifies the biological observation found in literature (Dutrochet [6], Millet et al. [START_REF] Millet | The rhythmic leaf movements after regeneration of partially excised pulvinus in phaseolus vulgaris l[END_REF]). The self-healing effect (regeneration) can be observed on the second day with an increasing resilience index (χ extensor = 0.84 and χ f lexor = 0.16).

The "McKibben" muscle (bio-inspired actuator)

McKibben muscles are manufactured pneumatic actuators inspired from the skeletal muscle [START_REF] Tondu | Modelling of the mckibben artificial muscle: A review[END_REF]. McKibben muscles are widely used for robotic arms, exoskeletons, and other bio-inspired devices. These types of muscles are made of an elongated cavity that becomes spherical when pressurised. This shortens the muscle and creates motion. Similarly to the previous actuators, McKibben resilience may be evaluated using equation [START_REF] Al-Shuka | Active impedance control of bioinspired motion robotic manipulators: An overview[END_REF] and some minimal modelling of its functioning. The McKibben muscle is idealised as an ellipsoid cavity of diameter d and major axis L (volume V = πd 2 L/6) with skin thickness w. The muscle is connected to a pressure tank P , with an input pipe of diameter e and length l. For an intact actuator, the pressure in the cavity matches the pressure of the tank. Now, if the actuator is damaged, some air may flow through this damage of typical size a (damage volume V * = wπa 2 ) which may reduce the pressure inside the cavity and thus the available power of the actuator.

Similarly to the hydraulic cylinder, the power of the McKibben muscle is given by the power of the pneumatic feeding f v = P Q. With a similar reasoning, one obtains the resilience of the McKibben muscle,

χ = 1 1 -η   1 - 2 3 η d 2 L √ l e 2 w 2/3 2   . (14)
The force-velocity relationship is plotted in figure 6a ( r 0 = 50 mm , L 0 =100 mm, L =10 mm, r =5 mm and l =10 mm). In log-log scale, the curve shifts down when damage occurs (from black to red). The resilience of the McKibben muscle is plotted in figure 6b. When the damage level reaches η = 0.0015, the resilience collapses. The McKibben muscle is as sensitive to damage as the hydraulic cylinder. In fact, McKibben muscles mimic the geometry and actuation shape of skeletal muscle, showing a Hill-like force velocity relationship. However, muscle resilience has not been mimicked making it highly sensitive to minor damage.

To avoid catastrophic failures, McKibben muscle may be strengthened with reinforced self-healing fibres [START_REF] Terryn | A pneumatic artificial muscle manufactured out of self-healing polymers that can repair macroscopic damages[END_REF]. Such self-healing strategies are behind the scope of the present resilience analysis.

Resilience sensitivity to damage location : global actuator resilience

The proposed definition of resilience is not expansive. Depending on the location of the damage, different levels of resilience may be computed for an actuator. For instance, the pulvinus may be considered very resilient when excising the flexor, but poorly resilient when the extensor is damaged. For the sake of generality, one may, for instance, consider damage in every location in the actuator and compute global resilience χ. The global resilience of the actuator would be given by the most harmful damage, where χ is minimal. The global resilience reads χ =min

Ω (χ), ( 15
)
where Ω is the set of possible damages of fixed volume. For instance, in the case of the hydraulic cylinder, a crack on top of the cylinder, figure 7, may not change the actuator's capabilities unless used at maximal stroke. In this case, the resilience is larger than 1.

To show the importance of damage location, the resilience of the hydraulic cylinder is computed along the cylinder body and in the seal by displacing the leak position (numbers from 1 to 6 ). Dimensions are given in section 3.1. A cycle from the bottom to the top of the cylinder is considered as a reference cycle. When the leak is in position 1 , at the top position of the stroke, the fluid never leaks and resilience is just above 1. Conversely, for positions 2 and 3 , a leak occurs during the part of the cycle where the piston is above the leak position, and there is no leak elsewhere. This reduces the efficiency of the hydraulic cylinder and thus its resilience. Finally the most critical leak positions are 4 , 5 and 6 where the leak occurs during the whole cycle. Those results are gathered in figure 7 that shows that resilience breaks down differently depending on the leak position. If the damage occurs on the seal or the bottom of the cylinder ( 4 , 5 or 6 ) resilience breaks down independently of the stroke. Error bars represent variations of ∆a =2.5% of the damage typical size. This corresponds to a variation of damage level ∆η = 5%.

To give another example where damage location is important, a magnetic actuator made of a simple rotor and rotating stator is considered [START_REF] Xu | On reciprocally rotating magnetic actuation of a robotic capsule in unknown tubular environments[END_REF]. This magnetic actuator may appear resilient, because when a part of the magnetic rotor is damaged, its torque then decreases only by the damage volume. There is no catastrophic loss of performance due to this specific damage. However, the critical part of this actuator is not in the magnetic rotor, but rather in the electrical motor, the performance of which may collapse if the solenoid winding is damaged. Thus, the global resilience of the actuator needs to be studied and analysed in order to determine its critical part.

Discussion

As shown on the hydraulic cylinder modelling, most traditional industrial actuators are not resilient to damage. For decades, redundancy and over-sizing were the gold standards in industry. However industry 4.0 challenges to create downsized compact actuators with less raw material consumption, less energy and less weight. A resilient actuator may be key to overcoming this challenges. Fortunately, resilient biological actuators may be mimicked to create industrial actuators. However, the direct copy may have drawbacks mainly because most biological actuators have evolved under multiple pressures of selection towards a specific motion as shown with pulvini. Pulvinus is more resilient on the extensor side than on the flexor side, likely because the extensor actuates critical protective motion.

Moreover, the direct mimicry of the shape, the motion capability and the force-velocity relationship, as done in the McKibben muscle, is not sufficient to produce resilience to damage. Resilience emerges from a more subtle structural design.

From a structural point of view, the skeletal muscle and the pulvinus are the aggregation of many microactuators in a structured (skeletal muscle) or an unstructured manner (pulvinus). This study suggests that a multi-scale structure, the structural properties shared by pulvini and biceps, is effective to create resilience. This resilience may be enhanced by feedback loops and micro-scale control. Bio-inspired series and parallel elastic actuators with their multi-scale structure may offer a promising option to create resilience [START_REF] Furnémont | Novel control strategy for the+ spea: a redundant actuator with reconfigurable parallel elements[END_REF][START_REF] Higueras-Ruiz | What is an artificial muscle? a comparison of soft actuators to biological muscles[END_REF][START_REF] Verstraten | Improving the performance of industrial machines with variable stiffness springs[END_REF] Pulvinus control is a complex cell-centred self-regulation, still under active research [START_REF] Tran | Mechanosensitive ion channels contribute to mechanically evoked rapid leaflet movement in mimosa pudica[END_REF]. The skeletal muscle is controlled by multi-levelled feedback loops and the central nervous system [START_REF] Johnston | Muscles and movement: Muscles, reflexes, and locomotion. thomas a. mcmahon[END_REF]. The feedback loops avoid shaking (control-induced oscillations) and allow precise, smooth and fast movement (these problems are recurrent in robotics). These multi-levelled control loops allow for the presence of damage to be efficiently managed. These loops are not present for the pulvinus which may explain why it is only resilient when the flexor is excised. The multi-scale structure of biological actuators, featuring series and parallel actuators, creates the necessary large number of degrees of freedom to apply bio-inspired resilience algorithms [START_REF] Cully | Robots that can adapt like animals[END_REF].

Biological actuators demonstrate self-healing capacities, which have not been considered in the present paper. In fact, for the pulvinus, self-healing is efficient to produce resilience as it is shown in figure 4a, where oscillation amplitudes are totally recovered after one day. The present analysis focuses on the first day after excision, where cells are not yet healed. Analysing self-healing properties of the skeletal muscle has not been carried out, and Hill's force-velocity relationship, equation [START_REF] Furnémont | Novel control strategy for the+ spea: a redundant actuator with reconfigurable parallel elements[END_REF], does not take self-healing into account. However we know that self-healing also occurs with skeletal muscle since athletes recover from muscle soreness within days of intense training. To create an advanced resilient actuator, a combination of these properties are required: multi-scale structure from cells to tissues, multi-levelled feedback control and self-healing.

In summary, based on definitions of resilience found in different fields, an index of resilience has been proposed. The resilience of two manufactured and two biological actuators were analysed: the hydraulic cylinder, the McKibben bio-inspired muscle, the skeletal muscle and the pulvinus. With the proposed definition, the skeletal muscle is shown to be fully resilient, the pulvinus partly resilient, and manufactured actuators not resilient at all.

The proposed index of resilience to damage has been designed to be the most general. Several other definitions of damage level would have applied. For instance, the damaged mass ratio m * /m may stand for the damage level as opposed to the damage volume ratio. For biological actuators, these two ratios would be almost similar given that densities of biological material are close to one (bone: 1.1 -1.8 10 3 kg/m 3 , muscle: 1.1 10 3 kg/m 3 , adipose tissue: 0.92 10 3 kg/m 3 ; dry wood: 0.14 -1.3 10 3 kg/m 3 -living wood density is the mean between dry wood and water densities). However, when an industrial actuator is considered, there may be more significant variations in material densities. For instance, the seal of the hydraulic cylinder is made of rubber (density: 0.92 -2.2 10 3 kg/m 3 ) whereas the body is made of steel (density: 8 10 3 kg/m 3 ). Thus major damage in the seal may be underestimated when using the mass ratio as a measure of damage.

Conclusion

By clarifying resilience for actuators, equation ( 1), an index was computed on a hydraulic cylinder, a McKibben bio-inspired muscle, a pulvinus, and a biceps. Manufactured actuators are shown not to be resilient, whereas biological ones are. Depending on the location of damage, actuator resilience may be strongly affected.

This tool, equation [START_REF] Al-Shuka | Active impedance control of bioinspired motion robotic manipulators: An overview[END_REF], would allow to quantify in practice resilience when designing new bio-inspired actuators. This would allow for an optimisation of actuator designs with regards to resilience. Depending on the application, the proposed definition of resilience may be adapted by changing the target of equation ( 1) in terms of performance.

Figure 1 :

 1 Figure 1: Actuators of reference. a. A leaf of (Mimosa Pudica), adapted from Tadrist et al. [24]. Pulvinus locations are encircled in red. aInset: Cross-section of pulvinus with motor cells in light green. b. Diagram of a skeletal muscle showing highly structured muscle fibres (at multiple scales), adapted from Gillies [9]. c. Car jack hydraulic cylinder. Inset: Piston and rubber seal (red circle) where damage may occur. d. Pneumatic McKibben muscle.

Figure 2 :

 2 Figure 2: a. Force-velocity relationship for the hydraulic cylinder (log-log scale). Black: intact hydraulic cylinder. Red: damaged hydraulic cylinder (η = 0.009). The dotted lines represent a cycle of motion of the hydraulic cylinder. b. Colour map of the hydraulic cylinder resilience for different forces, f and damage level, η.The hydraulic cylinder has an extremely low resilience (null when damage is larger than 0.0013). For visualisation purposes, the damage level is shown between 0 and 0.002.

Figure 3 :

 3 Figure 3: a. Data points are from Greenberg et al. [10]. The intact (black) and damaged (red) curves are fitted on Greenberg data varying α and N * s /Ns. The dotted lines represent the muscle actuation cycle for a fixed force. b. Colour map of muscle resilience for different forces (f /f 0 ) and damages (η). All the data is scaled between 0 and 1 for clarity.

Figure 4 :

 4 Figure 4: Angle of inclination with an intact pulvinus (black) and with the flexor excised (a. in red) or with the extensor excised (b. in red). Grey areas correspond to the first day after excision, indicated with the black arrow

Figure 5 :

 5 Figure 5: Angle of inclination θ for a single cycle from day 2 to day 3. The force-velocity relationship is obtained numerically after fitting the displacement curve, deriving the displacement to obtain the speed. The force is considered to be the moment of the weight of the plant. It is then proportional of the displacement.

Figure 6 :

 6 Figure 6: a. Force-velocity relationship for the McKibben muscle (log-log scale). Black: intact McKibben muscle. Red: damaged McKibben muscle (η = 0.001). The dotted lines represent a cycle of motion of the McKibben muscle. b. Colour map of McKibben muscle resilience for different forces, f and damage level, η. The McKibben muscle has an extremely low resilience (null when damages are larger than 0.0015). For visualisation purposes, the damage level is shown between 0 and 0.002

Figure 7 :

 7 Figure 7: a. Scheme of a simple effect hydraulic cylinder. Damage is here represented by numbers, to show which damage is the most critical. b. Different level of resilience χ are plotted for each level of resilience in a. Errors were computed considering a range of variation of the leak diameter a of 2.5% around the mean value. This corresponds to a variation of damage level ∆η = 5%

Table 2 :

 2 Definition of parameters. ( * ) indicates quantities relative to a damaged actuator or system.

	Actuation and resilience	Pulvinus
	f (t) Force during a cycle	θ	Leaf inclination angle
	v(t) Velocity during a cycle	Hydraulic cylinder/McKibben muscle
	V	Initial volume of material	P	pressure
	η	Relative damaged volume V * /V	Q	Feeding flow rate
	χ	Resilience	Q l Leak flow rate
	R	Redundancy of the system	a	Leak typical diameter
	χ s	Resilience of a system	e	Diameter of the input pipe
	Muscle	l	Length of the pipe
	f 0	Skeletal muscle maximum force	w	Housing thickness
	v 0	Skeletal muscle maximum velocity	L	Housing length
	α	Actin-Myosin kinetical parameter	d	Housing diameter


Hadi Harb
email: hadi.harb@ec-lyon.fr

Liming Chen
email: liming.chen@ec-lyon.fr

Voice-Based

Keywords: Gender Identification in multimedia applications content-based audio indexing, Piecewise Gaussian Modeling, mixture of neural networks

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Automatically detecting the gender of a speaker has several potential applications. In the context of Automatic Speech Recognition, gender dependent models are more accurate than gender independent ones. Hence, gender recognition is needed prior to the application of one gender dependent model [START_REF] Acero | Speaker and Gender Normalization for Continuous-Density Hidden Markov Models[END_REF][START_REF] Neti | Phone-Context Specific Gender-Dependent Acoustic-Models for Continuous Speech Recognition[END_REF]. In the context of speaker recognition, perfect gender detection can improve the performance by limiting the search space to speakers from the same gender. In content based multimedia indexing, the speaker's gender is a cue used in the annotation. Also, Gender dependent speech coders are more accurate than gender independent ones [START_REF] Marston | Gender Adapted Speech Coding[END_REF][START_REF] Potamitis | Gender-Dependent and Speaker-Dependent Speech Enhancement[END_REF]. Therefore, automatic gender detection can be an important tool in multimedia signal analysis systems.

Several acoustic conditions exist in audio-visual data: compressed speech, telephone quality speech, noisy speech, speech over background music, studio quality speech, different languages, and so on. Clearly, in this context, a gender identification system must be able to process this variety of speech conditions with acceptable performance.

In this paper we propose a gender identification system based on a general audio classifier. The proposed technique doesn't assume any constraint on the speech quality or segment lengths, in contrary to the existing techniques.

Related work

Motivated by different applications, several works have focused on voice based gender detection in the literature. [START_REF] Konig | GDNN a Gender Dependent Neural Network for Continuous Speech Recognition[END_REF] extracted 12 Linear Prediction coding Coefficients (LPC) and the energy features every 500 ms and used a Multi Layer Perceptron as a classifier for gender detection. They reported an 84% of frame based accuracy on the DARPA resource management database [START_REF] Price | The DARPA 1000-Word Resource Management Database for Continuous Speech Recognition[END_REF]. The database is a collection of clean speech recorded from 160 speakers in American English. Vergin and Farhat (1996) used the first two formants estimated from vowels to classify gender based on a 7 seconds sentences reporting 85% of classification accuracy on the Air Travel Information System (ATIS) corpus (Hemphill Charles et al., 1990) containing specifically recorded clean speech. [START_REF] Neti | Phone-Context Specific Gender-Dependent Acoustic-Models for Continuous Speech Recognition[END_REF] used a simple pattern matching approach where the acoustic observation of a speech utterance is first decoded into phonemes and the Euclidian distance is calculated between the observation and the recognized male and female phoneme models. The model with the lowest distance is chosen to detect the gender of the speaker. The results on the ATIS corpus are 62% of accuracy for sentences from 3 to 8 seconds. However, when using a general GMM approach to model each gender's acoustic vectors, Neti et al. report in the same paper classification results of 95% precision rate on the same sentences of 3 to 8 seconds. In order to deal with the problem of gender normalization of speech [START_REF] Jung | Implementation of Real-Time AMDF Pitch-Detection for Voice Gender Normalization[END_REF] used pitch detection based on the simple Average Magnitude Difference Function (AMDF) in gender identification on the DARPA TIMIT speech corpus. [START_REF] Tzanetakis | Musical Genre Classification of Audio Signals IEEE Transactions on Speech and Audio Processing[END_REF] followed a general audio classifier approach using Mel Frequency Cepstral Coefficients (MFCC) features and Gaussian Mixture Models (GMM) as a classifier. When applied to gender identification in a multimedia indexing context, the results are 74% of classification accuracy with three classes, male, female and sports announcement. [START_REF] Slomka | Automatic Gender Identification Optimised For Language Independence[END_REF] used a combination of a pitch-based approach and a general audio classifier approach using GMM. The reported results of 94% are based on 7 seconds files after silence removal on the OGI and the Switchboard telephone speech databases [START_REF] Muthusama | The OGI Multi-Language Telephone Speech Corpus[END_REF][START_REF] Godfrey | SWITCHBOARD Telephone Speech Corpus for Research and Development[END_REF].

Hidden Markov Models were also used for gender identification. For each gender, one HMM speech recognition engine is trained. The gender dependent models are used to decode a test speech signal. The model with higher likelihood is chosen as a cue for the gender of the speaker [START_REF] Huang | Improved Acoustic Modeling with the SPHINX Speech Recognition System[END_REF]. [START_REF] Parris | Language Independent Gender Identification In Proc[END_REF] combined pitch and HMM for gender identification reporting results of 97.3%. Their experiments have been carried out on sentences of 5 seconds from the OGI database. Some studies on the behavior of specific speech units, such as phonemes, for each gender were carried out [START_REF] Martland | Analysis of Ten Vowel Sounds Across Gender and Regional Cultural Accent[END_REF].

This overview of the existing techniques for gender identification shows that the reported accuracies are generally based on sentences from 3 to 7 seconds obtained manually. Moreover, some papers use sentences from the same speaker. This assumption holds for speech data where speaker boundaries are known in advance, such as the case of telephone speech. Although the automatic segmentation of a continuous stream of speech, such as the one existing in multimedia documents, is feasible, it still constitutes an important problem for the multimedia indexing community [START_REF] Viswanathan | TranSegId: A system for Concurrent Speech Transcription, Speaker Segmentation and Speaker Identification[END_REF][START_REF] Delacourt | DISTBIC: A Speaker-Based Segmentation for Audio Data Indexing[END_REF]. Generally Equal Error Rates (EER) are of the order of 15% depending on the nature of the documents (broadcast news are "easier" to segment than meeting recordings for example).

In several studies, some preprocessing of speech is also done, such as silence removal or phoneme recognition. While silence can be efficiently detected for clean speech, it becomes problematic for noisy speech. Phoneme recognition on the other hand adds an additional complexity layer to gender identification systems by the need of training phoneme recognizers on a specific speech corpus.

Gender identification in multimedia applications

In the context of multimedia applications gender identification has specific characteristics that make it different from the gender identification in other applications.

(1) The speech signal in multimedia data is recorded from a variety of different sources.

For instance, indoor, outdoor, and telephone speech are frequent in multimedia data. Therefore, a gender identifier for multimedia applications must be robust to channel and acoustic condition changes. In the systems proposed in [START_REF] Parris | Language Independent Gender Identification In Proc[END_REF] and [START_REF] Slomka | Automatic Gender Identification Optimised For Language Independence[END_REF] it is supposed that the speech is recorded from the telephone network and hence restricting the use. (2) Several audio coding and compressing techniques are used and hence implying the robustness to audio compression techniques.

(3) The speech is Multilanguage implying that a gender identifier in this context must be language independent. This assumption makes the use of HMM-based gender identification systems not effective since they are phoneme-related. (4) The speech is a continuous stream with no a priori known boundaries at each speaker turn. That is, it is not guaranteed that relatively long term audio segments contain speech from the same speaker or gender. On the other hand the time precision in multimedia applications is crucial, it is considered that 2 seconds is a maximum delay in such applications.

The system described in this paper makes use of several features and classifiers and fulfills the above requirements for gender identification.

The general audio classifier approach

Gender identification is a classic pattern recognition problem with two classes. In this section we describe the main features used for gender identification and we propose a new set of perceptually motivated features arguing that they are a better alternative to the classically used acoustic features. Moreover, several classifiers are described and a comparison is carried out.

The features

Pitch features.

The pitch feature is perceptually and biologically proved as a good discriminator between males' and females' voices. However, the estimation of the pitch from the signal is not an easy task. A good estimate of the pitch can only be obtained for voiced portions of a clean non-noisy signal [START_REF] Hess | Pitch Determination of Speech Signals[END_REF][START_REF] Shimamura | Weighted Autocorrelation for Pitch Extraction of Noisy Speech[END_REF][START_REF] Ross | Average Magnitude Difference Function Pitch Extractor[END_REF]. Moreover, an overlap of the pitch values between male's and female's voices naturally exists, hence intrinsically limiting the capacity of the pitch feature in the case of gender identification, figure 1.

In our experiments we use a standard algorithm for pitch estimation based on the autocorrelation function in the time domain for windows lengths of 100 ms. The algorithm estimates the pitch for the voiced portions of the signal. Voiced portions are detected using the Zero Crossing Rate and the accepted pitch values are between 20 and 600 Hz.

In the presented system the gender identification is performed on frames of 1s, called Integration Time Windows (ITW). Therefore, in each ITW window 10 pitch values can be extracted, called All Pitch values (AP). Consequently, the mean (Mean Pitch, MP) and the minimum of the pitch (minP) values are obtained for each ITW.

Acoustic features.

Short term acoustic features describe the spectral components of the audio signal and are generally extracted at 10 ms rate. Fast Fourier Transform can be used to extract the spectral components of the signal. Further filtering based on the perceptually motivated MEL scale is usually carried out. However, such features which are extracted at a short term basis (several ms) have a great variability for the male and female speech and captures phoneme like characteristics which is not required. For the problem of gender identification, we actually need features that do not capture the linguistic information such as words or phonemes.

In this work we have made use of long term acoustic features called Piecewise Gaussian Modeling (PGM) features which we initially proposed for speech/music classification [START_REF] Harb | Robust Speech Music Discrimination Using Spectrum's First Order Statistics and Neural Networks[END_REF]. The aim of PGM features for gender identification is twofold: (1) capturing long term spectral features that are discriminating between male's and female's voices, 2-modeling some aspects of the human perception of the sound, namely the short term memory aspect.

The Piecewise Gaussian Modeling (PGM).

The PGM features are inspired by some aspects of the human perception and classification of sound signal. We propose that several features are essential for a human subject to classify a stimulus into audio classes, such as voice, music, applause, etc.

Consider the task of male/female classification by a human subject. For this task, classifying a short term audio segment, 10 to 20 ms, seems to be hard for human subjects. However, long term segments, 200 ms for instance, are easier to classify. Hence, we can state that long term features are essential for the human classification of speech by gender.

On the other hand, the insertion of short time segments of male speech, respectively female speech, into a long term time segment of female speech, respectively male speech, cannot be easily detected by human subjects. This supposes that human subjects classify audio segments by gender in a global manner. We hypothesise therefore that, human subjects when classifying a stimulus by gender base their decision on their actual auditory memory. That is, they classify a short term segment based on its correlation with the past auditory events.

For an offline mode, the model can be seen as a sliding window, the ITW window, on the signal where the Gaussian parameters are estimated, with a time step ITW. This is what we call the Piecewise Gaussian Modelling (PGM) of the signal. The time precision of the PGM modelling is equal to delta ITW (Typical values for multimedia applications are of the order of a few seconds). In this study, we use 1 second windows.

Formally, let s(t) be the audio signal and t the time index. The short term spectral vectors, such as the Mel Frequency Spectral Vectors (MFSC), are: X t , t = 1 ...N * T where N and T are two constants. T refers to the number of short term spectral vectors contained in a ITW window. For instance if every 10 ms one vector is obtained and the ITW window is 1 second, then T = 100. N refers to the number of ITW windows.

The PGM consists of modeling a set of T consecutive short term spectral vectors by one Gaussian model. That is, N * T short term spectral vectors will be modeled by N Gaussians.

{ X 1 , X 2 ,..., X N .T }→{M 1 ( μ 1 , σ 1 ), M 2 ( μ 2 , σ 2 ),...,M N ( μ N , σ N )} With M i ( μ i , σ i )isthei-th
Gaussian expressed by its mean vector μ i and its variance vector

σ i μ i = 1 T iN t=(i-1)N +1 X t σ i = 1 T iN t=(i-1)N +1 ( X t -μ i ) • ( X t -μ i ) T
The PGM features are the concatenation of the mean and variance normalized by their respective maximum. This modeling scheme has several advantages over the use of short term spectral vectors.

The PGM features do capture neither phoneme-like features nor word-like features. They capture the dynamics of the speech and the distribution of the energy in each frequency channel contained in a long term window which may better convey gender related attributes.

Complexity analysis

We studied the complexity of the male/female classification problem for each of the features presented above. Such a study can provide valuable information about the discrimination power of each feature for this classification problem.

We studied the short term spectral features and the PGM features. The complexity measures that we used are the Fisher's Discriminant Ratio, the Volume of Overlap Region, and the classification accuracy of a decision tree algorithm.

The data used for the complexity measures is a subset of the train F dataset (see Section 6) containing 1000 seconds of male speech and 1000 seconds of female speech.

Fisher's Discriminant ratio.

The Fisher's Discriminant ratio permits the estimation of the discrimination capability in each feature dimension. It is given by:

f (d) = (μ 1d -μ 2d ) 2 σ 2 1d + σ 2 2d
Where f (d) is the Fisher's Discriminant ratio for the feature dimension "d", and μ 1d ,μ 2d σ 2 1d ,σ 2 2d are respectively the means and variances of classes "1"and "2" in the feature dimension "d".

As suggested in [START_REF] Ho | Complexity Measures of Supervised Classification Problems[END_REF] we use the maximum of f (d) over all dimensions. The higher the fisher discriminate ratio is the better are the features for the given classification problem. Clearly, as shown in Table 1, the PGM-features show higher Fisher's discriminant ratio than classic short term spectral features for this specific classification problem. These results suggest that PGM-features are more suitable than the short term spectral features for gender identification.

Volume of Overlap Region. The Volume of Overlap

Region is another measure to analyze the complexity of a classification problem; it calculates the overlap between the classes in a selected feature space [START_REF] Ho | Complexity Measures of Supervised Classification Problems[END_REF].

This can be measured by calculating the maximum and the minimum of the feature values in each feature dimension and then calculating the length of overlap for each dimension. The volume of overlap will be the product of the overlap lengths for all dimensions.

It is given by the following equation:

VOR = MIN(max( f i , c 1 ), max( f i , c 2 )) -MAX(min( f i , c 1 ), min( f i , c 2 )) MAX(max( f i , c 1 ) max( f i , c 2 )) -MIN(min( f i , c 1 ), min( f i , c 2 ))
Where max( f i , c 1 ) and min( f i , c 1 ) are respectively the maximum and the minimum values of the feature f i for the class c 1 (resp. c 2 ).i = 1,...,d for a d-dimensional feature space.

The VOR is zero (if the overlap is negative then it is set to zero) if there is at least a feature dimension in which the two classes do not overlap.

With no contradiction to the Fisher's Discriminant ratio results presented in the previous section, the PGM-features show less overlap between male and female classes than short term spectral features as illustrated by Table 2. From this point of view PGM-features are better discriminator between males' and females' voices than the short term spectral features.

Decision trees. Decision trees are well known techniques in data mining domain.

They can be used as classifiers as they aim at building rules in an IF THEN fashion permitting the decision about the class of a sample given its different attributes, or features.

We used the SIPINA one, a decision tree algorithm [START_REF] Zighed | SIPINA: Méthode et logiciel[END_REF]. A decision tree was built to discriminate between the males and females classes given the short term spectral attributes and another tree was built given the PGM-features. The decision trees were used to classify the training data and the classification error rate was used as an evaluation of each of the features, namely short term spectral features and PGM-features.

As shown in the Table 3, the PGM-features perform clearly better for this classification problem hence motivating their use as an alternative of the classical short term spectral features for this problem of gender identification.

The study on the complexity measures suggests that the PGM-features are more discriminating than the short term spectral features. Moreover, the compactness of the features needed to describe the data using PGM-features motivates their use for gender identification. Thus, PGM-features have been retained as acoustic features beside the pitch related ones.

The classifier

The choice of a classifier for the gender detection problem in multimedia applications basically depends on the classification accuracy. Other considerations, such as the time needed for training and for the classification can also affect the choice since the volume of the data to be analyzed is huge. We investigated Gaussian Mixture Models (GMM), Multi Layer Perceptron (MLP), and Decision Tree classifiers. We do not include experiments about simple K -Nearest Neighbors classifiers since they are limited by low classification accuracy and high computation complexity.

Gaussian mixture models.

Given a set of feature vectors, the GMM suppose that their probability distribution function is a combination of several Gaussians. Therefore GMM are a compact representation for a given classification problem since the information is embedded in the Gaussian parameters. GMM are also fast in both the classification and the training processes.

In our experiments the GMM were trained using the Expectation Maximization algorithm and initialized using the k-means clustering algorithm.

Since we use the highly correlated PGM features as the basic acoustic features we expect that the GMM performs poorly.

Multi Layer Perceptron.

MLP imposes no hypothesis on the distribution of the feature vectors. It tries to find a decision boundary, almost arbitrary, which is optimal for the discrimination between the feature vectors. The main drawback for MLPs is that the training time can be very long. However, we assume that if the features are good discriminators between the classes and if their values are well normalized the training process will be fast enough. In all our experiments we use a Multi Layer Perceptron with one hidden layer, 80 hidden neurons, and 2 output neurons. The Error Back-Propagation algorithm is used for the training.

The MLP classifiers are suitable for the PGM features since the MLPs are not strongly affected by the correlation of the input features as the statistical classifiers are.

Decision Trees. Decision

Trees are well known in the data mining literature. They are special kind of trees where each node is a question on a parameter and the road from the root to one leaf is a rule permitting the classification of a feature vector, or a set of parameters.

All the classifiers were trained on 200 seconds of male speech and 200 seconds of female speech extracted from the train F dataset and tested on the entire train F dataset, that is 2200 seconds, Section 6. The classifiers were trained using the PGM-features. The classification results are shown in Table 4.

The classification accuracy of the MLP is noticeably higher than that of GMM and Decision Tree classifiers as shown in Table 4. Notice that, as it is expected the GMM classifiers perform poorly on highly correlated features while the MLP are more suitable. The PGM features also suppose a diagonal hypothesis on the covariance matrix which may contribute to the poor performance of the GMM.

GMM and MLP would probably provide similar performance if a decorrelated version of the features is used. Nevertheless, it seams that the MLP treats reasonably well the correlation in the PGM features and hence it was retained as the basic classifier for the gender identification problem.

Information fusion

It can be argued that the pitch and the PGM features do not capture the same characteristics of the signal. The experiments in the Section 6.2 support this assumption. We hypothesize for instance that the combination of the different features can improve the performance of a gender classifier. An in depth analysis, which is out of the scope of this paper due to space limits, of the correlation between PGM and pitch features would be useful in this case to enable an optimal combination of classifiers and features.

Several ways can be used for the combination of the features. Combining the features at the feature vector level, figure 3, is one generally used technique especially in speech recognition [START_REF] Rabiner | Fundamentals of Speech Recognition[END_REF]). An alternative of the combination at the feature level is the combination at the classifiers' output level [START_REF] Kirchoff | Dynamic Classifier Combination in Hybrid Speech Recognition Systems Using Utterance-level Confidence Values[END_REF][START_REF] Wu | Incorporating Information From Syllable-length Time Scales Into Automatic Speech Recognition[END_REF], figure 4. That is, for each feature type one classifier is used and the combination of the classifiers' outputs constitutes the features combined output. In this work we incorporated the combinations at the two levels, the features' level and the classifiers' output level. This combination as described in 7.1.2 showed improvements in the performance over one single combination technique.

Practical issues

Although the evaluation of a gender identifier is generally based on the frame classification accuracy, real world applications imply some improvements over the frame-based accuracy.

They are mainly motivated by the following two considerations.

Combining experts

The major drawback of the gradient descent training technique for Multi Layer Perceptron is the time needed for training when the number of training samples is large.

In the case of gender identification, the more acoustic conditions exist the more training data to describe such conditions is needed. Therefore, a general gender identification system with good accuracy must be trained on thousands of training samples making the scalability of the training process a critical issue.

In this work we investigated the splitting of the training data into manageable subsets of the training set, say 2000 training samples per subset. The subsets are obtained by sequentially segmenting the training data. One classifier, or expert, is trained on each subset using the standard training algorithm. All experts trained on different subsets are then combined into one single expert. The combination is made by summing the outputs of the individual experts in order to obtain a single output layer for the resulting expert. Other functions such as multiplication or vote can also be used [START_REF] Kittler | Sum Versus Vote Fusion in Multiple Classifier Systems[END_REF]. However, in a previous work, our experimental results have shown that the performance obtained by multiplying or summing the outputs are similar while being particularly better than that of the majority vote [START_REF] Harb | Mixture of Experts for Audio Classification: An Application to Male/Female Classification and Musical Genre Recognition[END_REF]. Therefore, the sum function is retained for the combination of the individual experts.

The output " j", male or female in this work, of the Mixture Expert "ME" is given by:

o ( j) ME = N i=1 α i .o ( j) Ei
where "E i " are the individual experts, "N " is the number of individual experts, and α i is a weight associated to each individual expert. For the sake of simplicity the α i s are equal and they sum to one. The obtained general expert is an expert trained on the complete training dataset while each individual expert is specialized on a subset of the data. Such a model can be trained on a complex classification problem since the complexity is treated by the mixture of the experts and each individual expert will be trained on a softer version of the classification problem. However, if the individual experts are trained on insufficient data, they possibly will become unstable and biased to the training data [START_REF] Skurichina | Bagging for Linear Classifiers[END_REF].

A similar technique was applied for speech recognition [START_REF] Mirghafori | Parallel Training of MLP Probability Estimators for Speech Recognition a Gender-Based Approach[END_REF]) applications with different experts trained on the speech from different speakers are trained.

Smoothing the classification results

The speech contained in audio-visual programs is continuous. This assumption leads us to incorporate the results of neighboring frames to smooth the classification results. We segment the speech signal using a metric-based approach and the KullBack-Leibler distance as measure of similarity between neighboring windows [START_REF] Seigler | Automatic Segmentation, Classification, and Clustering of Broadcast News Audio[END_REF]. Each segment is assumed to contain one acoustic condition, in our case speech from the same gender. The labels of the frames in each segment are smoothed based on the average classification result for the entire segment. It is supposed that each segment contains speech from the same gender. However, we minimize the risk of mixing different genders in the same segment by decreasing the threshold used to segment the speech signal. Generally the classification results are slightly improved using such a smoothing technique, and in the worst cases the results do not change. In our system the threshold is set automatically to obtain a mean segments length of 3 seconds. That is, the local maxima in a sliding window of 3 seconds duration are selected as segment boundaries.

The used distance, the KullBack-Leibler (KL) distance, originates from the information theory (Cover and Thomas, 1991). It is a distance between two random variables. The original KL distance doesn't have the properties of a distance, but the symmetric KL is a distance. In the case of Gaussian distribution of the random variables the symmetric KL distance is computed by:

KL2(X, Y ) = σ 2 X σ 2 Y + σ 2 Y σ 2 X + (μ X -μ Y ) 2 1 σ 2 X + 1 σ 2 Y With σ X ,σ Y ,μ X ,μ Y are
respectively the standard deviation of X and Y and the mean of X and Y .

In the case of audio segmentation, X and Y are the set of spectral vectors obtained from the ITW window X at time t, and the ITW window Y at time t + T (T is the duration of the ITW window).

Notice that the variables used to compute the KL distance are the same features used in the PGM modeling; hence no new feature extraction is needed for the segmentation.

Experiments

Several experiments were carried out to evaluate the classifier for several classification conditions. We evaluated a "single Expert" and a "two Experts" gender identifiers with and without smoothing. The database used to evaluate the system consists of recordings from four French radio stations and one English radio station. Training data, Train F, was extracted from the recording of a French radio station; it consists of speech from news programs and meetings. The data from other radio stations was used as the French test data, Test F. Test F data was also compressed with MPEG layer-3 coder at a 16 Kbps rate to obtain Test F mp3. Furthermore, 1600s were selected from the English radio station containing telephone speech, outdoor speech and studio speech constituting the Test E dataset. A subset of the Switchboard dataset was also used for the evaluation. Table 5 shows the composition of the datasets used in the experiments. All the data was manually classified by gender. The amounts of time for male speech and female speech for each dataset were intentionally made equal.

System's architecture

An overview of our system for gender identification is presented in figure 5. It contains mainly two general modules: the feature extraction module and the classifier module. Each MLP has 80 hidden neurons and 2 output neurons with output values ranging from 0 to 1 and corresponding to the probability of a feature vector to be female or male. The outputs of all the MLPs are summed to obtain two outputs for the classifier as it was presented in Section 5.1.

Each MLP is trained with no relation to the others using the Back Propagation algorithm. Once the MLPs are trained the individual expert is created and is already trained.

Classification accuracy for the single expert case

In this experiment the data form Train F was used to train the system, and the data from Test F was used to evaluate it. The system with one individual expert was used. The amount The experimental results shown in figure 7 prove that indeed the classification accuracy increases by augmenting the size of the training data although the function is not monotonous. The accuracy of the classifier is about 75% when it is trained on 80 seconds of speech and it attains 90% when the classifier is trained on 2200 seconds of speech.

The overall classification results with a time precision of 1 second are 90.2% as shown in Table 6. These results are comparable to the reported results in the literature. A comparison with the results reported in [START_REF] Tzanetakis | Musical Genre Classification of Audio Signals IEEE Transactions on Speech and Audio Processing[END_REF] (73%) shows the effectiveness of the proposed system over a straightforward general audio classifier approach based on MFCC features and Gaussian Mixture Models. Moreover, when the smoothing is performed, Section 6.2, the classification results are improved (92.7%).

Table 6 shows the results of each MLP contained in the expert and the overall accuracy of the expert. It is clear, that the expert performs better than its best MLP proving that the information fusion is motivated. Notice however, that the pitch information when included in the feature vector did not improve the performance. This may be due to the fact that the estimated pitch is not very reliable and that the pitch values overlap between male speech and female speech. Nevertheless, the overall accuracy can be improved, although slightly, when using pitch and acoustic information over the use of the PGM features alone.

We have carried out another experiment to observe the effectiveness of the proposed approach for language change and when the test data contains telephone and outdoor speech. The system with one individual expert was trained on 2200s from Train F and tested on 1600s from Test E data. The results shown in Table 7 demonstrate that the proposed approach is language independent though the performance is slightly degraded. Notice that the system was faced to language and channel changes.

We can also notice that in these two experiments, the feature vector which performs better is not the same. While MLP-PGM performs the best in Table 6, it is MLP-PGM-MP that performs the best in Table 7. However, in both cases the individual expert performs better than its best MLP. This is important since for different test conditions the best MLP is surpassed automatically by the individual expert.

The overall accuracy obtained by one single expert from the two datasets is thus 87%. This accuracy attains 90.7% when the smoothing is performed.

The last experiment for the individual expert case that we have carried out was to test the effectiveness of the gender identifier when the speech is compressed at low compression rates. The Test F mp3 was used as test set and Train F was used for training. The classification accuracy for the individual expert did not degrade as compared to the accuracy obtained for the non-compressed data. This experiment tends to prove that the proposed system is robust to severe compression techniques as shown in Table 8

Classification accuracy for two experts case

In this experiment the system includes two individual experts each trained on half the training data, which is 1100 seconds of speech from Train F dataset. The two experts are combined as described in Section 5.1. The system was tested on Test F and Test E datasets. The classification results are presented in Table 9. As we can see, the classification results for the two datasets are improved from 87% for the single expert case to 89.4% when two individual experts are used instead. Moreover, in contrast to the case of one single expert, there is no bias toward any dataset or any gender when two individual experts are used. This is of extreme importance in multimedia applications since no information about the statistics of the data is given in advance. For instance one radio station may include English female speech only.

However, when the smoothing is performed the classification results attain an accuracy rate up to 93% on both datasets.

Classification accuracy for telephone speech

The speech material used in the previous experiments originates from noisy radio recordings, making the classification results look poor. The figure 8 shows a segment of speech from the Test F database and one segment from the Switchboard database; we can see the nature of the data used in the previous experiments. It is important to evaluate the performance of the gender classifier using experimental conditions that are close to those used in the literature. Generally gender identification systems in the literature are evaluated on speech material obtained from the telephone data with a time precision larger than 5 seconds. Therefore, we used a subset of the switchboard database [START_REF] Godfrey | SWITCHBOARD Telephone Speech Corpus for Research and Development[END_REF]. 19 telephone conversations were used, 9 conversations for training the gender classifier and the remaining 10 conversations were used for testing the classifier. In average, the training data consists of 1000 seconds of male speech and 1000 seconds of female speech from 18 speakers, 9 male speakers and 9 female speakers. The test data corresponds to 1000 seconds of male speech and 1000 seconds of female speech from 10 male speakers and 10 female speakers different than those used for the training. One expert was used for this experiment and the results for 1 second and 5 seconds time precisions are presented in Table 10. As it is clearly shown in the table, the performance is particularly better than the performance for the case of the database used in the previous experiments. Gender identification for speech data obtained from multimedia sources seems to be harder than on speech data obtained from the telephone network. This is may be due to the existence of outdoor and noisy speech in the multimedia context. Moreover, as it is expected, by decreasing the time precision to 5 seconds, the classification accuracy increases significantly.

Conclusion

This paper presented a voice-based gender identification system using a general audio classifier. Several classifiers and features were studied. A combination of Piecewise Gaussian Modeling features and pitch-related features with a set of Neural Networks was shown to perform better than any individual classifier. The system was tested on adverse conditions of compression, channel mismatch and language change. It was shown how smoothing the classification results can improve the accuracy. When applied to telephone speech, the classification accuracy of the gender classifier was shown to be considerably better than when applied to speech from unrestricted radio sources.

Figure 1 .

 1 Figure 1. Pitch Histogram for 1000 seconds of males (lower values) and 1000 seconds of females' speech (higher values). We can see the overlap between two classes.

Figure 2 .

 2 Figure 2. Piecewise Gaussian Modeling (PGM).

Figure 3 .

 3 Figure 3. Fusion at the feature vector level.

Figure 4 .

 4 Figure 4. Fusion at the output level.

Figure 5 .

 5 Figure 5. General overview of the gender identification system's architecture.

Figure 6 .

 6 Figure 6. Architecture of the classifier used in the system.

Figure 7 .

 7 Figure 7. Classification accuracy rate as a function of the duration of the training data.

Figure 8 .

 8 Figure 8. Example of the spectrogram of one speech segment from the Test F database (on the left), and one speech segment from the Switchboard database (on the right).

Table 1 .

 1 Fisher's discriminant ratio for 1000 s of male speech and 1000 s of female speech with PGM-features and short term spectral features.

		Short term spectral features	PGM-features
	F	15.3	70.9

Table 2 .

 2 Volume of Overlap Region for 1000 s of male speech and 1000 s of female speech with PGM-features and short term spectral features.

		Short term spectral features	PGM-features
	VOR	7.473 E-3	1.05673 E-17

Table 3 .

 3 Error Rate of a Decision Tree trained on 1000 s of male speech and 1000 s of female speech and tested on the training data for PGM-features and short term spectral features.

		Short term spectral features	PGM-features
	Error rate %	20	15.8

Table 4 .

 4 Classification accuracy for several classifiers trained on the same training data and tested on the same test data.

		GMM (14 Gaussians)	Decision Tree	MLP (80 Hidden neurons)
	Accuracy %	72.5	73.4	82.2

Table 5 .

 5 Evaluative dataset durations.

	Dataset	duration (s)	Male speakers	Female speakers
	Train F	2200	12	7
	Test F	2000	10	7
	Test F mp3	2000	10	7
	Test E	1600	9	7
	Subset switchboard	4000	19	19

Table 6 .

 6 Classification accuracy for radio data with and without smoothing.

		Male accuracy	Female accuracy	Total accuracy
	Smoothing (3s mean segment length)	95.2	90.1	92.7
	PGM	91.3	87.2	89.3
	PGM-MinP	87.0	89.0	88.0
	PGM-MP	88.6	86.3	87.5
	PGM-AP	86.6	89.8	88.2
	Average	88.4	88.1	88.3
	Expert	90.3	90.1	90.2

Table 7 .

 7 Classification accuracy for English data containing telephone and outdoor speech with and without smoothing.

		Male	Female	Average
	Smoothing (3s mean segment length)	93.2	84.2	88.7
	PGM	88.9	76.8	82.9
	PGM-MinP	89.5	76.0	82.8
	PGM-MP	88.5	79.9	84.2
	PGM-AP	84.8	74.0	79.4
	Average	87.9	76.7	82.3
	Expert	91.4	78.6	85.0

Table 8 .

 8 Classification accuracy of the individual expert when applied to compressed audio data.

		Male	Female	Total	Smoothing (3 s mean segment length)
	Test F mp3	89.1	88.7	88.9	90.0

Table 9 .

 9 Classification accuracy when two individual experts are used.

		Male	Female	Average	Average with Smoothing (3s mean segment length)
	Test F	88.5	89.6	89.1	91.2
	Test E	90.4	88.7	89.6	94.6
	Total	89.5	89.2	89.4	92.9

Table 10 .

 10 Classification accuracy for one individual expert on a subset of the Switchboard database.

		Male Accuracy	Female Accuracy	Average
	1 s	88.2	98.9	93.5
	5 s	97.0	100	98.5

Acknowledgments

This work has been partially supported by the Cyrano project within the French RNRT program. The first author was partially supported by Grant number 3691-2001 from the French ministry of research and technology. The authors would like to thank the anonymous reviewers whose critiques greatly improved the paper.