M Bentefrit

S Grondel

C Soyer

A Fannir

E Cattan
email: eric.cattan@univ-valenciennes.fr

J D Madden

M G Nguyen

C Plesse

F Vidal

Linear finite-difference bond graph model of an ionic polymer actuator

Keywords: ionic EAP, finite difference method, bond graph

With the recent growing interest for soft actuation, many new types of ionic polymers working in air have been developed. Due to the interrelated mechanical, electrical, and chemical properties which greatly influence the characteristics of such actuators, their behavior is complex and difficult to understand, predict and optimize. In light of this challenge, an original linear multiphysics finite difference bond graph model was derived to characterize this ionic actuation. This finite difference scheme was divided into two coupled subparts, each related to a specific physical, electrochemical or mechanical domain, and then converted into a bond graph model as this language is particularly suited for systems from multiple energy domains. Simulations were then conducted and a good agreement with the experimental results was obtained. Furthermore, an analysis of the power efficiency of such actuators as a function of space and time was proposed and allowed to evaluate their performance.

Introduction

Over the last two decades, electroactive polymers have been extensively studied [START_REF] Smela | Conducting polymers as artificial muscles: challenges and possibilities[END_REF][START_REF] Inganäs | Electrochemical muscles: bending strips built from conjugated polymers Synth[END_REF][START_REF] Pei | Bending bilayer strips built from polyaniline for artificial electrochemical muscles Smart[END_REF] and developed due to their huge potential in terms of actuators, soft sensors, and for applications in robotics [START_REF] Jager | Microrobots for micrometer-size objects in aqueous media: potential tools for single-cell manipulation[END_REF], biomedical science [START_REF] Shoa | Analytical modeling of a conducting polymer-driven catheter[END_REF][START_REF] Jager | The cell clinic: closable microvials for single cell studies[END_REF][START_REF] Low | Microactuators toward microvalves for responsive controlled drug delivery[END_REF], and artificial muscles [START_REF] Martinez | Mechanical awareness from sensing artificial muscles: experiments and modeling[END_REF]. Electroactive polymers can be divided into two families: dielectric polymers and ionic polymers. The first one based on electrostatic actuation is the most predominant in the literature thanks to suited properties in terms of large displacements and the ability to work in air, but requires high voltage for activation. The second family based on a redox process causes the insertion or expulsion of ions supplied by an electrolyte into or from an active layer of the conducting polymer which in turn creates a change in volume in the same layer, and therefore a global mechanical displacement.

With the recent development of ionic actuators that work in air and host their own ionic liquid [START_REF] Plesse | Synthesis and characterization of conducting interpenetrating polymer networks for new actuators[END_REF][START_REF] Alici | Conducting polymer microactuators operating in air[END_REF][START_REF] Gaihre | Synthesis and performance evaluation of thin film PPy-PVDF multilayer electroactive polymer actuators[END_REF][START_REF] Jeon | Bacterial cellulose actuator with electrically driven bending deformation in hydrated condition[END_REF][START_REF] Panwar | New ionic polymer-metal composite actuators based on PVDF/PSSA/ PVP polymer blend membrane[END_REF][START_REF] Kim | Electro-active hybrid actuators based on freeze-dried bacterial cellulose and PEDOT: PSS Smart[END_REF], new opportunities [START_REF] Martinez | Mechanical awareness from sensing artificial muscles: experiments and modeling[END_REF] have been opened up. In particular, a new type of ionic actuator working in air composed of a three-layer interpenetrated architecture has been recently developed (figure 1) at the LPPI together with IEMN [START_REF] Goujon | Flexible solid polymer electrolytes based on nitrile butadiene rubber/poly(ethylene oxide) interpenetrating polymer networks containing either LiTFSI or EMITFSI[END_REF]. This specific architecture improves mechanical behavior, since there is no delamination between each layer and increases ionic conductivity within the actuator thickness [16].

For such a configuration, the central layer acts as an ionic host matrix for the electrolyte. It is formed from an interpenetrated network of poly(ethylene oxide)( PEO) and nitrile butadiene rubber (NBR) and is filled with the ionic liquid 1-Ethyl-3-methylimidazolium bis-trifluoromethylsulfonyl-imide (EMITFSI). The two remaining layers behave as working electrodes and are made of a conducting polymer-poly(3,4ethylenedioxythiophene)( PEDOT)-semi-interpenetrated with the central matrix. The actuation mechanism relies on the volume variation of the PEDOT electrodes. Indeed, when a voltage is applied between those two electrodes, one of them is reduced while the other one is oxidized. This results in a migration of ion, in or from the host matrix, which in turns causes the expansion of the electrode being reduced and the contraction of the electrode being oxidized. Those combined phenomena produce the global bending actuation of the device.

As far as we know, few models have been developed with such a family of actuators. Several physical domains are involved during actuation: electrochemical and mechanical, which are coupled. From the literature review, methods of modeling ionic actuators can be divided non-exhaustively into three categories. The first category called white box, contains analytical methods that require perfect knowledge of all the phenomena occurring, but in return provide an exact solution to the problem. For instance, Martinez et al [START_REF] Martinez | Mechanical awareness from sensing artificial muscles: experiments and modeling[END_REF] propose a theoretical model which describes electrochemical and mechanical phenomena since it relates mechanical work to flowing current and elements involved in the chemical reaction. This model is well adapted for actuators working at low frequencies but requires many physical parameters to measure and identify. The second category, gray box, only involves partial knowledge of the physical phenomena that occur and provides an approximated solution. For example, Nishida Gou et al who work on IPMC actuators [START_REF] Nishida | Multiscale distributed parameter modeling of ionic polymer-metal composite soft actuator[END_REF], use successfully distributed port-Hamiltonian systems on multiple spatial scales and are therefore able to analyze influence of nonuniform and local properties. In the same category, we can also quote the work of Moghadam et al [START_REF] Moghadam | Nonlinear dynamic modeling of ionic polymer conductive network composite actuators using rigid finite element method Sensors[END_REF] who takes into account the geometric nonlinear displacement with rigid finite elements, whereas Moeinkhah et al [START_REF] Moavenian | Takagi-Sugeno fuzzy modelling and parallel distributed compensation control of conducting polymer actuators[END_REF] propose a distributed equivalent electrical circuit in order to better represent the electrochemical part. The last category, black box, also gives an approximated solution but does not require any knowledge of the mechanisms occurring. This method is often used in control applications since it prioritizes the use of simple models adapted to a given sample [20,[START_REF] John | Inversion-based feedforward control of polypyrrole trilayer bender actuators[END_REF].

The main issue of this paper is to understand and predict the behavior of the new ionic polymer composed of three PEDOT/PEO-NBR layers, as well as to quantify its power efficiency. Given the partial knowledge we have of electrochemical phenomena occurring and in order to cope with the challenge of space description and power analysis, we decided to develop an original and simple gray box model based on a linear multiphysics finite difference bond graph representation. The primary benefits of such a representation are first the balance between accuracy and complexity supplied by the finite difference scheme [START_REF] Spiteri | Méthode des Différences Finies Pour les EDP Stationnaires[END_REF] as it gives a spatial solution for both electrochemical and mechanical domains and second the schematized and simple representation of energy exchanges and storage within the system when using the Bond Graph language.

This article is organized as follows. In section 2, physical equations are briefly recalled for each domain and finite difference scheme are then developed in Bond Graph language. In section 3, physical parameters are estimated experimentally and injected into the model. Finally, the built model is used to estimate the actuator efficiency.

Presentation of the model

As usual, Bond Graph modeling requires first to establish the system Word Bond Graph. As this representation is the first step toward describing the energy map and the composition of the system, a physical analysis of the system and its power interactions are sufficient to determine the Word Bond Graph. As shown in figure 2, it is divided into three parts in our case. The first part is dedicated to the electrochemical behavior representation, the last one describes the mechanical behavior, and the middle one enables coupling between the first two. Contrary to conventional block diagrams, the inputs and outputs of each subsystem here define power variables represented by a conjugated effort-flow pair (e, f ) indicated by a half arrow.

In this case, the effort and flow variables are (F, v) and (sA,  ) for the mechanical domain, and (E, I) and (u, i) for the electrochemical domain, respectively. Variables (F,  v) and (sA,  ) represent mechanical force and velocity or strain rate, respectively, while (E, I) and (u, i) correspond to electrical voltage and current, respectively.

Each Word Bond Graph can be developed using elements that can either produce or store and dissipate energy, as shown figure 2. Elements relating to storage are usually defined by C for potential energy and I for kinetic energy, while dissipative elements are described by R. Effort and flow sources for energy creation are represented by Se and Sf elements, respectively. Finally, elements such as '0' and '1' junctions or TF and GY elements can be used for transmitting energy without loss [START_REF] Borutzky | Bond Graph Modelling of Engineering Systems[END_REF].

Electrochemical aspect

As related in [START_REF] Shoa | Analytical modeling of a conducting polymer-driven catheter[END_REF], when driven at low frequencies by a voltage () ux t ,, conducting polymers act as electrical capacitors. The three layers' actuator can be thus electrically represented by an RC circuit with resistances that translate the electronic and ionic transport. The resistance part will be provided by surface and thickness resistance of the PEDOT on one hand, and by the thickness resistance of the NBR-PEO hosting matrix on a second hand. We will also consider in some case a third resistance when leakage in current ( ) ix t , is observed, this shunt resistance will describe thus local short circuits that can occur between the two PEDOT electrodes. The resistance per unit length of PEDOT is R e , the resistance per unit length of NBR-PEO is R EHM , the shunt resistance per unit length is R cc , and the equivalent capacitance per unit length is C eq .W e assume that PEDOT is electronically isotropic so we can then express the same resistance per unit length along the surface or the thickness.

From the electrical diagram, figure 3, we can then determine the partial differential equations (PDEs) governing current and voltage behavior in the circuit. Using voltage divider law we can express + () ux x t d, as function of ( ) ux t , and thereby we find: In the same way, by applying Kirchhoff's voltage law, we end up with a similar relation with voltage:

* ¶ ¶ =- ++ ++ + + w ww ( ) ( ) () () () xt x ux t RR RR R R R i, , 2 . 2.
* ¶ ¶ =- () () () ux t x ix t R , , . 2 . 2
e By combining (2.1) with (2.2) and assuming that  x d0 , we can then express the two PDEs that govern current and voltage in an elementary equivalent circuit:

* ¶ ¶ -= () () () ix t x ix t Z Z , ,0 , 2 . 3 2 2 2 1 * ¶ ¶ -= () () () ux t x ux t Z Z , ,0 . 2 . 4 2 2 2 1
By approximating (2.3) and (2.4) using the forward finite difference method, the Bond Graph scheme of an electrochemical actuator part can be rewritten.

As illustrated in figure 4, the schematic established, provides a micro-scale visual representation of the power transfer between the resistive and capacitive parts of the electrochemical element. Replication and serial linking of the same element enable the rapid and easy visualization of the same power transfer on a macro-scale.

Mechanical aspect

From a mechanical point of view, the polymer actuator is assimilated to a beam with a rectangular cross section.

In order to simplify the study and with regards to the electrochemical phenomena involved, it is assumed that the actuator only generates bending deformations. We will also put forth the hypothesis that in our field of study the material remains viscoelastic and linear, i.e. that no creeping occurs, and that it can be seen as a homogenous, isotropic material with an equivalent Young's modulus E and an area moment of inertia I along the x axis. Let us name rx FA v ,,,,, y Mk ,,, and G, the external applied force, the beam density, the cross-sectional area, the transverse displacement, the damping, the external applied momentum, the angular displacement, the numerical factor related to the cross section shape, and the shear modulus, respectively. The equations of motion can then be written as follows:

rx ¶ ¶ = ¶ ¶ + ¶ ¶ () () () () Fx t x A vx t t vx t t ,, , ,2 . 5 2
2 r y + ¶ ¶ = ¶ ¶ () () () () Fx t Mx t x I xt t , ,, ,2 . 6 2 2 y = ¶ ¶ - ⎜ ⎟ ⎛ ⎝ ⎞ ⎠ () () () () Fx t k G A vx t x xt , , , , 2 . 7 y
= ¶ ¶ () () () Mx t E I xt x , , .2 . 8
With the additional assumptions that there is no inertial rotation of the cross sections and no shear force (Euler-Bernoulli assumptions), equation (2.7) disappears and equation (2.6) is simplified as follows:

+ ¶ ¶ = () () () Fx t Mx t x , , 0. 2.9
Obtained PDEs can be approximated with the forward finite difference method:

rx +D - D = ¶ ¶ + ¶ ¶ ( ) () () () () Fx x t Fx t x A vx t t vx t t ,, , , , 2.10 2
2 +D + +D - D = () () ( ) () Fx x t Mx x t Mx t x , , , 0, 2.
11

yy +D = +D - D () () ( ) () Mx x t E I xx t x t x , ,, .2 . 1 2
The Bond graph schematic of the mechanical element can be then developed from the previous approximations, as shown figure 5.

Assembly of the global model

The expulsion of ions from the ionic liquid and insertion into the two PEDOT conducting polymer layers causes one layer to expand and the other to contract, resulting in mechanicalmomentum and the displacement of the actuator. It then appears, as reported by Tina Shoa et al [START_REF] Shoa | Analytical modeling of a conducting polymer-driven catheter[END_REF], that the displacement of the actuator (or the momentum generated) is directly related to the charges flowing inside the conducting polymer. Equation (2.8) can be rewritten as follows:

 y = ¶ ¶ +D () () () ( ) Mx t E I xt x EA x x t , , , . 2.13
Equation (2.13) is called the electro-chemo-mechanical coupling equation, where  ( ) xt , is the strain caused by the insertion (expulsion) of ions. It can be expressed as follows:

 ar = () () ( ) xt xt ,, , 2 . 1 4 c
where r c is the charge transfer per unit volume and a is an empirically determined ratio that expresses the strain generated per volumetric charge transfer.

From equations (2.13) and (2.14) we can then transform the global model to a Bond Graph for a given elementary actuator of length dx, see figure 6.

Estimation of the physical parameters

Beams synthesis and preparation

In order to simulate the ionic actuator, all the physical parameters of the finite difference Bond graph model must be fully determined. In this way, a beam measuring 9375 μm×2000 μm×200 μm was initially synthesized [START_REF] Plesse | Polyethylene oxidepolytetrahydrofurane-PEDOT conducting interpenetrating polymer networks for high speed actuators Smart[END_REF] and shaped from a bulk PEDOT-PEO/NBR material as follows:

-The bulk material was as described in [START_REF] Plesse | Polyethylene oxidepolytetrahydrofurane-PEDOT conducting interpenetrating polymer networks for high speed actuators Smart[END_REF] and was spincoated onto a rigid silicon wafer with a polyvinyl alcohol (PVA) acting as an adhesive and a sacrificial layer. -The beam was then cut directly on the silicon wafer using an infrared nano-pulsed laser (E-Series, OXFORD Laser Ltd). With this cutting system, the beam can be shaped to the required dimensions with a resolution of around one micrometer. The luminous power of the laser beam was adjusted so that the etching did not go through the silicon wafer. -The beam was then released from the wafer by dissolving the PVA layer in deionized water for 10 min. -Once released, the beam was dried by blowing nitrogen over it and swollen with ionic liquid 1-Ethyl-3-methylimidazolium bis (trifluoromethylesulfonyl) imide (SOL-VIONIC) at ambient temperature for over two weeks.

The aim is now to extract the physical characteristics i.e. both mechanical and electrochemical from this sample.

Electrochemical parameters

All the electrochemical parameters can be determined from a simple current reading with an applied square voltage of ±1.5 V swept at a frequency of 0.1 Hz. Notice here that it was decided to clamp the entire sample between two glass wafers covered with a 200 nm thin gold layer in order to eliminate the surface resistance effect.

The readings, figure 7(a), show a charging and discharging phase, during positive and negative square voltage. For each phase, the current goes back to zero, which means that there is no current leakage due to the large thickness of the material and low voltage used. As main assumption, we can now neglect the R cc element for this experiment and therefore the equivalent circuit can be simplified as in figure 7(b).

According with that equivalent circuit, the current during charging ( ) ixt , 1 can be related to voltage as follows:

tt =-⎜ ⎟ ⎛ ⎝ ⎞ ⎠ () () () ixt Cu xt t , , exp , 3.1 1 eq
where t is the time constant of the circuit and ( ) ux t , is the applied voltage.

+ R R e EHM can then be estimated from the maximal current magnitude during the charging phase whereas C eq can be determined from the slope of the current curve during the same phase. It must then be verified that the current reading matches an RC charging or discharging circuit.

Figure 7(a) shows that the current signal follows an exponential behavior and hence can be approximated by equation (3.1). We can then extract R , e R , EHM and C , eq we find for the resistive part a total value of 645 W and for the capacitive part a value of 2.09 mF.

Coupling coefficient

The coupling coefficient a, also called charge-to-strain ratio, is expressed as the ratio of the peak-to-peak magnitude of the strain DS and the peak-to-peak magnitude of the charge The bending strain  b is expressed in per cent and cal- culated from the displacement as follows [START_REF] Sugino | Actuator properties of the complexes composed by carbon nanotube and ionic liquid: the effects of additives[END_REF]:

 = + () hv lv 100 . 3.3 b 22
With v being the displacement, w the width and l the length.

From the previous charge curve and displacement curve under ±1.5 V, see figures 8(a) and (b),w efind for the strain to charge ratio a value of 4.56 × 10 -10 m 3 C -1 .

Mechanical characterization

In the case of mechanical characterization, Young's modulus can be measured in several ways, each introducing a different level of accuracy and information. The traditional method is tensile testing, which also determines the strain hardening property of the test material. Young's modulus can also be estimated by vibrating the beam at its resonance frequency or acoustically by measuring the celerity of a known acoustic wave travelling inside the material. As our material is highly damped, the latter is not appropriate. The tensile test could not be considered either as it is difficult to adapt small-geometry samples for the tensile test bench.

The most suitable method we have found so far is the flexure test. Indeed, when a stress is applied to the beam that is below the yield stress of the material, according to Hooke's law, the force applied is proportional to displacement as follows:

= () () () Fx t k vx t ,, , 3 . 4
where k is the stiffness of the material and () vx t , the resulting displacement under the applied force F. We can then relate Young's modulus E to stiffness k through the width w and length l of the beam, through equation (3.5) which stays valid only for cantilever beams. Measurements were performed with a FemtoTools FT-S microforce probe that can detect forces down to 1 μN, see figure 9. The probe is mounted on an x, y, z piezo-actuated platform and driven by software that allows us to run automated measurements. The probe, which can be positioned with extremely high precision, is initially placed in contact with the beam and moved 100 μm along  y in 10 μm increments. Every 10 μm, the probe reads the force and the values are then used to plot the force-displacement curve, see figure 9(b). The curve shows that the data are perfectly spaced along  y and that the points are well aligned. This implies that the resulting stress was way below the yield stress.

In order to obtain the damping ratio of the material, the beam was clamped to a mechanical shaker (TIR-Vibration Test System TV 500018) and excited by an oscillating force. Once the excitation stopped, the tip displacement was determined using a laser system Keyence LKG32, see figure 10(a). From the logarithmic decrement, see figure 10(b), the damping ratio can be determined.

Indeed, in a spring-damper system the displacement () vt can be described as shown below:

gw f =- + () ( ) ( ) ( ) vt C t t . exp . . cos , 3.6 a
where C and f depend on the initial conditions, and g and w a are the damping ratio and the pseudo pulsation, respectively.

The damping ratio can then be expressed as follows:

g =-() () T ln .3 . 7 A A a 2 1
With T a being the pseudo period, and A 1 and A 2 two displacement points away from T .

a For a beam with dimensions of 9375 μm×2000 μm× 200 μm, applying the previously described techniques we find a young modulus of 200 MPa ± 20 MPa, and a damping equal to 0.38 N m -1 s -1 for a beam density of 1600 kg m -3 .

All physical parameters for the specimen with dimensions of 9375 μm×2000 μm×200 μm are recalled in table 1. In order to focus on the uncertainties that may occur between two specimens from a same batch, another actuator with a different width was also characterized whereas its physical parameters were deduced from the first actuator by taking into account this width difference. Results show a slight deviation on the electrochemical part between measurements and estimated values. This means that the electrochemical parameters contrarily to the mechanical parameters seem sensitive to ambient conditions. This phenomenon was already observed by the authors who analyzed physical parameters variations as function of temperature in [START_REF] Maziz | Demonstrating kHz frequency actuation for conducting polymer microactuators[END_REF]27]. In the next part, the simulation will be performed using measured parameters on the first actuator.

Results

Time domain responses

The aim of this section is to compare the results of the simulated time domain responses with those of the experiments in order to assess the performance of the proposed modeling approach. To simulate the actuator behavior, it has been chosen to use ten finite difference elements in the bond graph model to achieve a balance between accuracy and ease of calculation. Moreover, the simulated actuator was driven in the same conditions than in the experiments one, i.e. by a ±1.5 V square voltage at a frequency of 0.1 Hz in air and at ambient temperature.

Both electrochemical parts and mechanical parts are analyzed in this study. Hence, figure 11 actuator length whereas figure 11(d) shows a comparison between the simulated and experimental transverse beam displacement () vx t , at the tip position, i.e. for = x 9375 μm. First of all, results show a close matching between the proposed model and the experimental data. Hence, it can be seen from figure 11(d) that the obtained simulated and experimental displacements at the ionic beam extremity for a ±1.5 V square voltage excitation and a frequency of 0.1 Hz are in good agreement. Furthermore, it can also be noticed in figure 11(b) that the simulated current slope fits well with the experimental one, despite a discard of 1.8 mA noticed on the maximum value. This discrepancy in the current pic values is mainly due to the time discretization in our numerical model, but is not sufficient to affect the deflection results of the ionic actuator beam.

To estimate the validity of the proposed model, another simulation is proposed on a similar sample but with different dimensions (length of 6400 μm), voltages amplitude and frequency i.e. with an entry square voltage of ±0.56 V, ±1.22 V, ±1.47 V and ±2.06 V and a frequency of 0.05 Hz. Correlation between the displacements and the currents on figure 12 shows that the model is able to make a good prediction for samples with different geometries at different voltages and frequencies.

Additionally, one of the advantages of our modeling approach using finite difference element is to allow the direct simulation of the ionic actuator behavior as function of the space. Therefore, it can be noticed from figure 11(a) and as expected that the simulated electrical current was not absorbed in the same way along the ionic actuator length due to the surface resistance. Furthermore, figure 11(c) illustrates the linear deflection of the ionic actuator. It must be mentioned here that despite the fact that our model is not developed for the case of large deflection, it could be adapted for such a case using similar modeling than in [START_REF] Moghadam | Nonlinear dynamic modeling of ionic polymer conductive network composite actuators using rigid finite element method Sensors[END_REF] but this issue requires at the same time to experimentally characterize the nonlinearity of the polymer behavior in case of high voltage. As this part is out of the scope of this paper since we concentrate us on the power efficiency and exchanges inside the ionic actuator, it will not be treated here.

Power and efficiency study

The results exposed in figures 11 and 12 validate the multiphysics model for small deflections of the ionic actuator. This section is now devoted to a power analysis both in electrochemical and mechanical parts, in order to determine the global actuator efficiency.

As illustrated in figure 13(a), the power electrically conducted through PEDOT electrodes is non-uniformly transmitted along the ionic actuator length due to the electrical surface resistance. A large decrease is observed which means that for a better efficiency of the actuator, the electrical electrodes need to be deposited all along the ionic actuator. Then, as seen figure 13(c), a part of power is electrochemically stored inside the equivalent capacitance C eq near location m = x 0 m and time = t 0s, the stored electrochemical power reaching a minimum of -1.11 mW. At that location, this means that only a part of the injected power takes places in the redox process, the other part being electrochemically dissipated through electrical and ionic resistance as seen figure 13(e). This dissipated power achieves a maximum value of 3.27 mW at x=0 μm and t=0 s, which demonstrates that the electrochemical system is highly dissipative. Then a part of the stored electrochemical power is converted through the coupling (2.13) and used to generate mechanical displacement. Again, a part of the injected mechanical power figure 13(b) is stored figure 13(d) whereas the other part is dissipated figure 13(f). Notice that negative electrochemical power represents here the release of energy previously stored in the electrochemical parts, whereas the negative mechanical power is related to sign reversal of inertial forces and energy storage in elastic part of actuator for a short period.

Figures 14(a) and (b) illustrate the total powers, i.e. the input, dissipated and stored powers as function of the time of the electrochemical and mechanical parts of the ionic actuator respectively. From these curves, it was also possible to determine the input, dissipated and stored energies as function of time for both the electrochemical (figure 14(c)) and mechanical (figure 14(d)) domains.

The obtained responses reflect the behavior to a transient excitation of a multiple RC circuit and a multiple RLC circuit for the electrochemical domain and for the mechanical domain respectively. Indeed figure 14(a) shows that the stored electrochemical power increases first between 0 and 5 s (half a period) which means that electronics charge are getting accumulated inside the PEDOT layers, while input and dissipated powers are getting lower because of the decrease in current due to the charging capacitor phenomena. When the input voltage switches sign at = t 5s,the stored power drops below zero at -2.8 mW and therefore the stored energy is released to the system, the capacitors being discharging. Furthermore, we notice that the overcurrent is totally dissipated until the stored power goes above zero, at = t 6.3 s. Figure 14(b) shows that mechanical powers behave the same way except for the dissipative mechanical power which is really low since we are exciting the actuator at a frequency far from the mechanical eigenfrequency: the damping has here only a minimal effect at that frequency. From electrochemical and mechanical energies in figures 14(c) and (d), we come to the conclusions that mean stored energies are constant. This is because the energies are released back to the system in time with the difference that in mechanical domain the stored energy is greater due to the small dissipative part.

Figure 15(c) illustrates global efficiency i.e. the ratio between stored mechanical energies and input electrochemical energies, whereas figures 15(a) and (b) show electrochemical and mechanical efficiencies respectively, i.e. the ratio between stored and input energies. From the study of these figures, it can be concluded that the overall maximal efficiency is at about 0.01% and that energy dissipation mainly occurs in the electrochemical part. In fact, maximal electrochemical efficiency is equal to 7% (this is much lower than a classical RC circuit at 50%) whereas maximal mechanical efficiency is much greater at about 75% due to the minimal effect of mechanical damping at this frequency.

Conclusion

In this paper, a simple one-dimensional electro-chemomechanical model using a bond graph finite difference method, was developed in order to simulate the behavior of a new kind of ionic actuator. It is distinguished from existing modeling work of ionic polymer actuators in that it is amenable to time responses and energy exchange simulations while capturing all the multi-physics domains. The simulated times responses of the linear model were verified by comparing them with experimental results.

In summary, the built model enables us thus to predict behavior and understand phenomena occurring in the ionic actuator while visualizing the power and energy efficiencies along the actuator length and over time. It is our aim that the model presented in this paper may be used and developed further for the use and design of ionic polymer actuators.

Future work will be focused on two aspects. First, the proposed actuation model will be extended to incorporate electrochemical and mechanical nonlinearities that become pronounced at large actuations levels. The nonlinearities include geometric nonlinearity, nonlinear elasticity and the dependence of electrochemical parameters on the curvature output. The second direction of future work is the application of the modeling approach for the sensing case. There the physics of the model has to be extended to account for force interaction measurements in the presence of external objects.

Figure 1 .

 1 Figure 1. Ionic actuator structure. The blue layers are the PEDOT polymer electrodes whereas the yellow layer corresponds to the ionic PEO-NBR host matrix, and the gold layers to the Au electrodes required for electrical contact.

Figure 3 .

 3 Figure 3. Equivalent circuit of an elementary actuator of length x d.

Figure 4 .

 4 Figure 4. Bond graph of an electrochemical cell.

Figure 2 .

 2 Figure 2. Word bond graph of the global system. Boxes represent physical domains. Parameters above and below arrows represent efforts and flows provided or received by those domains.

Figure 5 .

 5 Figure 5. Bond graph of a mechanical cell.

Figure 6 .

 6 Figure 6. The electro-chemo-mechanical element of length dx. Mechanical and electrochemical elements are coupled through a modulated source element which is the translation of equation (2.13).

Figure 7 .

 7 Figure 7. (a) Current flowing through the thickness of the three layers actuator PEDOT/PEO-NBR under ±1.5 V squared voltage driven at 0.1 Hz, (b) simplified equivalent circuit.

Figure 8 .

 8 Figure 8. (a) Transferred charge to the three layers of the PEDOT/ PEO-NBR actuator with a square voltage of ±1.5 V driven at 0.1 Hz, (b) produced strain under ±1.5 V squared voltage swept at 0.1 Hz.

Figure 9 .

 9 Figure 9. (a) bench, (b) beam stiffness.

Figure 10 .

 10 Figure 10. (a) Displacement measurement bench, (b) free tip motion after sinusoidal excitation. The zoom-in-box shows a pseudo period T a of the signal.

  (a) represents the simulated electrical current () ix t , along the ionic actuator length whereas figure 11(b) shows a comparison between the simulated and experimental current () it 0, at the clamping point, i.e. where the gold electrodes are located. In a same way, figure 11(c) illustrates the simulated transverse beam displacement () vx t , at different locations along the ionic

-Figure 11 .

 11 Figure 11. (a) Simulated current ( ) ix t , along the ionic beam location, (b) simulated current versus experimental current at clamping point = () ix t 0, , (c) simulated transverse beam displacement () vx t , as function of time, (d) simulated displacement versus experimental displacement at free end tip m = () vx t 9375 m, . Entry voltage is set at a ±1.5 V swept at 0.1 Hz.

Figure 12 .

 12 Figure 12. Top: simulated current versus experimental current at clamping point = () ix t 0, under various voltages. Bottom: maximum simulated displacement versus maximum experimental displacement at free end tip m = () vx t 6400 m, , under various voltages. Entry voltages are squared and set at ±0.56 V, ±1.22 V, ±1.47 V and ±2.06 V and swept at 0.05 Hz.

Figure 13 .

 13 Figure 13. (a) Injected electrical power = () () () Px t ux t ix t ,, . , .

i

  (b) Injected mechanical power. (c) Stored electrochemical power. (d) Stored mechanical power. (e) Dissipated electrochemical power. (f) Dissipated mechanical power.

Figure 14 .

 14 Figure 14. Evolution of total injected power and energy (in blue), total stored power and energy (in green) and total dissipated power and energy (in red), for the electrochemical domain (a), (c) and mechanical domain (b), (d).

Figure 15 .

 15 Figure 15. Instantaneous efficiencies in (a) electrochemical system, (b) mechanical system, (c) global system.

  [START_REF] Smela | Conducting polymers as artificial muscles: challenges and possibilities[END_REF]

					1
			cc	EHM	jC	eq
	1						1
	e jC	eq	EHM	cc	cc	EHM	jC	eq

Table 1 .

 1 Measured parameters on 9375×2000×200 beam, measured versus estimated parameters on 10 000×3000×200 beam.

	Measured	Estimated

Acknowledgments

This project received funding from the French Government through the National Research Agency ANR-15-CE08-0032-01 MICRO-TIP, under program PIA EQUIPEX LEAF (ANR-11-EQPX-0025) and RENATECH.


Auriane Boudin

Roxane Bertrand

Stéphane Rauzy

Magalie Ochs

Philippe Blache

A Multimodal Model for Predicting Conversational Feedbacks

We propose in this paper a statistical model in the perspective of predicting listener's feedbacks in a conversation. The first contribution of the paper is a study of the prediction of all feedbacks, including those in overlap with the speaker with a good accuracy. Existing model are good at predicting feedbacks during a pause, but reach a very low success level for all feedbacks. We give in this paper a first step towards this complex problem. The second contribution is a model predicting precisely the type of the feedback (generic vs. specific) as well as other specific features (valence expectation) useful in particular for generating feedbacks in dialogue systems. This work relies on an original corpus.

Introduction

Conversational interactions are characterized by different phenomena showing participant's engagement. Among them, the most important are undoubtedly conversational feedbacks [START_REF] Schegloff | Discourse as an interactional achievement: Some uses of "uh huh" and other things that come between sentences[END_REF] which usually consist in brief signals produced by the interlocutor during the main speaker's speech and can be verbal (e.g. yes), vocal (e.g. mhm), and/or gestural (head movements, smiles). All linguistic interaction theories underline their crucial role during an interaction [START_REF] Clark | Using language[END_REF][START_REF] Horton | Theories and Approaches to the Study of Conversation and Interactive Discourse[END_REF]. We know in particular that they are mandatory to update the shared knowledge (common ground ) and promote the alignment between participants which is necessary for mutual comprehension and success of the interaction [START_REF] Bavelas | Listeners as co-narrators[END_REF][START_REF] Pickering | Understanding dialogue: Language use and social interaction[END_REF]. Understanding their behaviors and the conditions of their realization in a natural context is then of deep importance both for theoretical reasons, but also for applications in the perspective of human-machine interaction.

Feedbacks predictive cues have been studied in different modalities: prosodic [START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF][START_REF] Poppe | Backchannel strategies for artificial listeners[END_REF][START_REF] Ward | Prosodic features which cue back-channel responses in english and japanese[END_REF], syntactic [START_REF] Blache | Two-level classification for dialogue act recognition in task-oriented dialogues[END_REF], gestural [START_REF] Ferré | Unimodal and bimodal backchannels in conversational english[END_REF] semantic and pragmatic [START_REF] Allwood | The mumin coding scheme for the annotation of feedback, turn management and sequencing phenomena[END_REF]. However, to date, no global model proposing a multimodal account of feedback realization and a prediction of feedback types exists. One of the reasons is that there exist only few corpora providing such information. The vast majority of existing resources focus on audio only, which explains the fact that most of the works rely on prosody, taking other modalities into account only to a certain extent. Corpora providing both video and audio are limited and generally do not bear annotation of the gestures involved in the description of these phenomena. Moreover, the processing of these multimodal data is still an open question. Overall, a multimodal model of feedbacks in natural interaction, describing and predicting their realization, has yet to be established.

We present in this paper an overview of the different feedback models and propose a new model involving a large set of multimodal features. Moreover, we propose to join time and type prediction, making it possible to explore the prediction of feedback occurrences concerning not only the site of realization for the feedbacks, but also their type. This work is based on an original corpus annotated at different levels. We propose a modeling approach rendering possible a clear interpretation of the model. The model and its evaluation are finally discussed.

Related works

In this section, we propose an overview of the main feedback modeling works by focusing on three main information: the features they rely on, the time span of the observation window (in other words the segmentation) and the results.

Most of the works study feedbacks as a specific case of turn-taking. Among them, a seminal study has been proposed in [START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF] for predicting three different situations: turn changes (smooth switches) vs. turn retentions (holds) and backchannels (a feedback subtype). The signal is segmented into inter-pausal units (IPU) and at each pause longer than 50ms, a set of features is extracted from the preceding IPU. Seven features are used in turn-yielding prediction: a falling or high-rising intonation at the end of the IPU; a reduced lengthening of IPU-final words; a lower intensity level; a lower pitch level; a point of textual completion; a higher value of three voice quality features: jitter, shimmer, and NHR; and a longer IPU duration. Six features are identified more precisely for backchannels: a rising intonation at the end of the IPU; a higher intensity level; a higher pitch level; a final POS bigram (Det-N, Adj-N, N-N); a lower value of noise-to-harmonics ratio (NHR); and a longer IPU duration. They fit multiple logistic regression to assess the relative importance of the different cues. This study show that the likelihood of the occurrence of a feedback is 30% when all 6 cues are present (the percentage of turn-taking being 65% with the 7 relevant cues). A cross-lingual study based on 3 languages has been proposed in [START_REF] Brusco | A cross-linguistic analysis of the temporal dynamics of turn-taking cues using machine learning as a descriptive tool[END_REF].

This approach is adapted by [START_REF] Meena | Data-driven models for timing feedback responses in a map task dialogue system[END_REF] for detecting "response relevant places" (a response being feedbacks, short answers, clarification questions, etc. organized into 8 classes). The task called Response Location Detection consists in classifying into 3 main classes: hold (a response would be inappropriate), expected response, optional response. They completed the model of [START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF] by taking into account three different types of features: prosodic (pitch, intensity), contextual (turn and IPU length, last system dialogue act, pause duration) and lexico-syntactic (word form, POS, semantic classes). The observations are also based on an IPU segmentation (2,272 IPUs). Different learning algorithms and feature combinations have been applied. The model based on lexico-syntactic features obtains a 84.42% score of accuracy for the voted perceptron model, only slightly improved when adding prosodic features (84.64% accuracy with naive bayes).

In his work, [START_REF] Skantze | Towards a general, continuous model of turn-taking in spoken dialogue using lstm recurrent neural networks[END_REF] proposes a continuous model of turn-taking, predicting turn-shifts and their types (short backchannel or a longer utterance). It uses a set acoustic features (voice activity, pitch, intensity,spectral stability), completed by POS. Feature vectors are extracted from the signal each 50ms. The model learned by a RNN reaches an F-Score of 0.762, using all features for the turn-shift prediction.

When focusing on feedback prediction independently from turn-taking, we find several rule-based approaches. In their reference work, [START_REF] Ward | Prosodic features which cue back-channel responses in english and japanese[END_REF] propose a famous rule based on prosodic cues (pitch region, its level, duration and localization and previous feedback realization). This rule has been completed by other algorithms in [START_REF] Poppe | Backchannel strategies for artificial listeners[END_REF] aiming at determining the placement of feedbacks. These rules are based on prosodic and gaze features. They are implemented in a virtual listener. Note that this work also provide interesting information about human listener behavior and has been refined by a corpus study in [START_REF] Truong | A multimodal analysis of vocal and visual backchannels in spontaneous dialogs[END_REF]. In their work, [START_REF] Morency | A probabilistic multimodal approach for predicting listener backchannels[END_REF] also propose to use several lexical and prosodic cues as well as the indication of speaker's gaze. Speaker features are sampled at a rate of 30Hz. Different probabilistic sequential model are trained for different encoding templates. The results show an F-score of 0.2562 (outperforming the Ward & Tsukahara method). On their side, [START_REF] Cathcart | A shallow model of backchannel continuers in spoken dialogue[END_REF] present an approach for predicting a specific type of feedbacks (backchannel continuers) on the basis of pause duration and POS. Their results show an F-measure of 35%.

To summarize, the different works usually focus on a subset of predictive cues, usually coming from a unique modality. None of them tries to give a global picture involving a multimodal set of features. Second, these works aim at predicting the site or the time for the feedback realization, but not the feedback type. Finally, as for the modeling aspects, no clear indication are usually given on the relative importance of the cues and more generally on the model's interpretation. The great variability and the scores obtained by the different techniques show the difficulty of the task.

Feedbacks: typology, cues

We distinguish between two feedback types: generic (displaying attention to the speaker) and specific (expressing responses to the content of the speaker's production) [START_REF] Bavelas | Listeners as co-narrators[END_REF][START_REF] Tolins | Addressee backchannels steer narrative development[END_REF][START_REF] Bertrand | Co-narration in french conversation storytelling: A quantitative insight[END_REF]. Generic feedbacks are often vocal item such as mh or gestural signals like nods. They express interest and understanding while specific feedbacks (e.g. brief verbal utterances, a particular tone of voice, etc.) correspond to an evaluation or comment. Some feedbacks are prototypical of one type (e.g. mhm is typically generic whereas oh no is specific), some others not. Feedbacks can be expressed in different modalities: verbal, visual or multimodal. Besides verbal feedbacks, which are in most of the cases brief lexical expressions, visual feedbacks can be realized in different ways: head movement (nod, jerk, shake, tilt, turn, waggle), facial expressions (smile, laugh) or eyebrow (frowning, raising). Bimodal feedbacks (involving both visual and verbal productions) are also very frequent and play an important role [START_REF] Ferré | Unimodal and bimodal backchannels in conversational english[END_REF]. In some cases, bimodality can reinforce the function: for example, bimodal BCs show a stronger agreement than unimodal ones. Table 1 give some examples of the main feedback types and their modalities associated to different possible subtypes (note that our work focuses on French, but can be directly generalized to other languages).

Typology: As shown in the table, specific feedbacks are described according to two levels of analysis: polarity (positive or negative) [START_REF] Allwood | A study of gestural feedback expressions[END_REF][START_REF] Chovil | Discourse-oriented facial displays in conversation[END_REF] and their expected/unexpected character [START_REF] Prévot | A cup of cofee -a large collection of feedback utterances provided with communicative function annotations[END_REF]. This question of feedback subtype is crucial [START_REF] Schegloff | Discourse as an interactional achievement: Some uses of "uh huh" and other things that come between sentences[END_REF][START_REF] Bavelas | Listeners as co-narrators[END_REF] and could explain the various and sometimes controversial cues found in the literature (yes for example can be either generic or specific).

Feedback cues: Based on the literature as well as our experience, we propose to explore the role of a large set of features, from three modalities: speech (prosodic features, [ [START_REF] Meena | Data-driven models for timing feedback responses in a map task dialogue system[END_REF][START_REF] Blache | integrated model for predicting backchannel feedbacks[END_REF][START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF]) and visual (gestures, expressions, attitudes, [START_REF] Allwood | A study of gestural feedback expressions[END_REF][START_REF] Ferré | Unimodal and bimodal backchannels in conversational english[END_REF][START_REF] Morency | Predicting listener backchannels: A probabilistic multimodal approach[END_REF][START_REF] Poppe | Backchannel strategies for artificial listeners[END_REF]). Besides widely used features (e.g. POS, pauses, etc.), we also propose to involve less studied features. Before entering into feature description, it is necessary to specify the frame unit into which the predictive features will be analyzed. In most of the cases, inter-pausal units (IPUs) are chosen, the features of the end of the main speaker's IPU are the input variable of the model. The problem in this case is that we miss many feedback produced in overlap with the speaker (representing 40% of feedbacks in our corpus). It is then necessary to chose another segmentation. One solution consists in segmenting the input by means of a rolling window. In some works, an arbitrary frame size of 30-50ms is chosen [START_REF] Morency | A probabilistic multimodal approach for predicting listener backchannels[END_REF][START_REF] Skantze | Towards a general, continuous model of turn-taking in spoken dialogue using lstm recurrent neural networks[END_REF]. However, this type of segmentation entails a huge problem of imbalanced classes when trying to predict feedback vs. no-feedback, including during speakers speech. We propose instead to segment on the basis of events, at each word, or during no-speech segments at nod or laugh. Such events form the right boundary of a predicting window of 2 seconds. This duration is arbitrary, but usually correspond to segments larger enough to contain complete units (in terms of syntax and pragmatic contents). Table 2 present the complete set of features involved in our model.

At the prosodic level, besides pauses, we examine the intonation pattern given by a sequence of tone. Some specific tone n-grams before a pause followed by a feedback could correspond to a final intonation pattern which is often correlated with the introduction of a new information [START_REF] Chafe | Discourse, consciousness and time[END_REF] and carries an important part of the interactional meaning [START_REF] Portes | Some cues about the interactional value of the continuation contour in french[END_REF]. Several studies have shown that the final intonation contour could be a good predict or for feedback occurrence. Tones also represent an intermediate level between low-level acoustic features such as pitch and phonological interpretations. Taking into account tone n-grams makes it possible to compare the influence of tones taken separately or by sequence.

Concerning lexico-syntactic features, we propose to include POS and semantic information about word polarity (positive, negative) and aspect (concreteness). These information can be associated to specific listener's reaction concerning a certain level of emotion, but also the use of a discourse referent associated to concrete words. On their side, discourse markers are the sign of discourse organization, often associated to transition between discourse units, that can be associated to reactions.

Finally, introducing visual features complete the multimodal description. Nods, laughs and smiles, that can by themselves constituting feedbacks, are also to be taken into account in the prediction, not only because of mimicry (a laugh can trigger a laugh feedback) , but also due to the importance of their communicative function [START_REF] Bertrand | Backchannels revisited from a multimodal perspective[END_REF].

The dataset

This work focuses on French. Our dataset is built upon an existing corpus of natural conversations that we have completed with the different annotations. The corpus, called Cheese-Paco [START_REF] Priego-Valverde | cheese!": a corpus of face-to-face french interactions. a case study for analyzing smiling and conversational humor[END_REF][START_REF] Amoyal | PACO : A corpus to analyze the impact of common ground in spontaneous face-to-face interaction[END_REF], contains 7 hours of audio-video recording. The participants, in dyads, were installed face-to-face. They first had to read a short story before having a free conversation. Cheese-Paco is composed of 27 interactions, each lasting an average of 17 minutes. A manual transcription has been done, including different information such as noise, laugh, pause, elision, unexpected events. This transcription has been automatically aligned onto the signal thanks to the SPPAS system [START_REF] Bigi | Sppas: a tool for the phonetic segmentations of speech[END_REF] that returns the list of phonemes, syllables and IPUS. The MarsaTag analyzer [START_REF] Rauzy | Marsatag, a tagger for french written texts and speech transcriptions[END_REF] has been applied to extract the lemmas and POS. Moreover, smiles and nods were annotated semi-automatically using the HMAD toolkit [START_REF] Rauzy | Automatic annotation of facial actions from a video record: The case of eyebrows raising and frowning[END_REF][START_REF] Rauzy | Smad : A tool for automatically annotating the smile intensity along a videorecord[END_REF]. In this work, we used a subset of data composed of 4 dyads for a total of about one hour. The corpus contains 769 feedbacks, 2,739 IPUs and 15,215 words.

Feedback annotation: Feedbacks have been manually annotated by 3 annotators. The annotation guide considers the distinction between specific and generic feedbacks to which we added the two subtypes (valence +/-positive and +/-expected). In order to facilitate the annotation, a pre-processing has been done for identifying automatically the possible feedbacks on the basis of different signals: laughs, smiles shorter than 200ms, repetitions, interjections. Annotators had to check whether these suggestions was correct and when necessary add feedbacks non identified during pre-processing. The second annotation step consists in identifying the feedback type. Five categories are possible: one for generic feedbacks and four for specific feedback sub-types (+/-positive, +/-expected). Finally, annotators were asked to determine the feedback boundaries.

Lexico-syntactic features:

Besides POS, we annotated lexical-semantic information (concreteness, valence) on the basis of word lists given in [START_REF] Bonin | Concreteness norms for 1,659 french words: Relationships with other psycholinguistic variables and word recognition times[END_REF]. We also identified discourse markers. Concerning POS, we kept only bigrams and trigrams with a frequency higher than 40.

Prosodic features: We used the automatic pitch modeling tool MOMEL-INTSINT [START_REF] Rossi | L'intonation: de l'acoustique à la sémantique[END_REF] which consists in two steps. The first consists in modeling the f0 based on a sequence of transitions between successive points on the curve (anchor points). The procedure of calculation of MOMEL is based on the relationship between the median, minimum and maximum values of each speaker's pitch range. The Octave-Median Scale used by the authors allows to compare speakers with different pitch ranges (typically males versus females) In a second step, the anchor points obtained from MOMEL are automatically coded by an alphabet of tonal symbols T(op), B(ottom), M(id) referring to absolute values and Higher, Lower, Same, Upstepped, Downstepped (referring relative values) and give rise to an intonation pattern represented by the key/midpoint and the span of the speaker's pitch range. We also encode, as it is the case in different studies, the length of the pause (with a threshold of 1,2 seconds between short and long pauses).

Gestures, expressions: Nods has been annotated semi-automatically. The automatic step relies on the HMAD system [START_REF] Rauzy | Automatic annotation of facial actions from a video record: The case of eyebrows raising and frowning[END_REF] which returns the time interval of the nod. A manual correction is then done, identifying when necessary missing nods and correcting time boundaries. The annotation of smiles follow the same procedure, and distinguishes 2 levels of smile (noted S3 and S4 in the following) [START_REF] Rauzy | Smad : A tool for automatically annotating the smile intensity along a videorecord[END_REF].

Data analysis

We give in this section a brief overview of the main statistical characteristics of our dataset. These statistics are in line with the literature [START_REF] Poppe | Backchannel strategies for artificial listeners[END_REF]: feedback are frequent and their production is a consistent phenomenon. Our data revealed a frequency of 9.3 feedback per minute (see Table 3). Specific feedback are slightly more frequent than generic. Regarding the sub-type of specific feedbacks, there is more often positive than negative feedback. The most frequent is the positive-unexpected feedback, the less is negative-unexpected. The most frequent feedback realizations are verbalization, laughs [START_REF] Bertrand | Co-narration in french conversation storytelling: A quantitative insight[END_REF] and nods [START_REF] Stivers | Stance, alignment, and affiliation during storytelling: When nodding is a token of affiliation[END_REF][START_REF] Allwood | A study of gestural feedback expressions[END_REF]. We find interesting to see how nods, laughs, verbalization and smiles are used according to the generic/specific function of the feedback. Figure 1 reveal the sum of feedback produced with at least one of these items (for a total of 769 feedbacks). Even if all feedbacks are mainly produced with verbal items (interjections, repetitions and other short lexical elements), this tendency is stronger for specific feedbacks. This can be explained by the fact that specific feedbacks need more details and context-dependent interventions. Verbal items for generic feedbacks are mostly interjections and plays the role of continuers. As expected, nods are significantly more used for generic than specific feedback [START_REF] Bavelas | Listeners as co-narrators[END_REF]. Conversely, laughs and smiles are widely used as specific feedbacks (and are very rare for generic ones). Finally, generic feedbacks are mainly produced with nods and/or verbalization, whereas specific feedbacks are primarily produced with verbalization/laugh and/or smile. We also noticed that generic

The model

We applied a statistical processing, logistic regression, that fits with our dataset specificities. At the difference with most of other predicting models, our goal is to predict both the position and the type of feedback to be produced during a conversation. We use for that a two-stage approach based on two models: one predicting the realization of a feedback, the second predicting its type (generic vs. specific). In this approach, logit is then used as a hierarchical classification technique. One interest of using logit also lies in the fact that it returns a probability for the classification. This is interesting in the perspective of implementing the model in a human-machine communication system, offering the possibility to introduce a fine variability in the feedback production. We also chose logistic regression in order to take into consideration different questions about the dataset (the size is rather small and learning techniques are prone to overfitting) as well as dimensionality and interpretability of the model.

The probability to produce a given type of feedback (or the probability that the feedback is produced at a given time location) is modeled by the logit equation:

logit(p) = ln p 1 -p = a 0 + a 1 x 1 (t) + ... + a j x j (t) + ... (1)
where x j (t) are the predictors which depend on the time location t and which can adopt binary, categorical or continuous types. In a first step, the parameters of the logit model (i.e. the a i coefficients) are estimated on the training sample. An inspection of the result allows to decide which predictor contributes significantly to the prediction. The model is finally formed with the subset of relevant predictors. A probability p is thus attributed to any combined values of the predictors. To convert this probability into a binary response, we apply the following classifier equation:

if (p > p threshold ) {response = 1} else {response = 0} (2)
where p threshold is for example the averaged probability to produce a feedback. This threshold value is estimated on the training sample. The model thus provides us with a binary prediction which depends on the predictor values in input.

The performance of the model is afterwards evaluated by using standard metrics based on the confusion matrix of the predictions versus the observations. A cross-validation strategy is applied in order to quantify the potential presence of overfitting problems. This evaluation procedure will be also applied in order to measure the relative contribution of a given subset of predictors (e.g. grouped by modalities).

Predicting the occurrence of a feedback The problem of predicting the time location of the feedbacks is difficult since it requires to include in the training table events where feedback does not occur. In our strategy, we considered all time locations corresponding to the end of an event (e.g. end of a word, of a pause, of a laugh or smile, etc.) as an entry of the table and we encoded the value of the binary dependent variable as 0 (i.e. absence of feedback). In addition we inserted in the table the list of observed feedbacks (i.e. with the binary dependent variable at 1). This strategy comes to the standard binary classifier model. In our case, the difficulty lies in the imbalanced character of the binary distribution (i.e. the no feedback events are massively dominant).

Note that dynamical models based on HMM, CRF, LSTM RNN have been proposed for predicting occurrences of feedbacks or related situations (see for example [START_REF] Morency | A probabilistic multimodal approach for predicting listener backchannels[END_REF][START_REF] Skantze | Towards a general, continuous model of turn-taking in spoken dialogue using lstm recurrent neural networks[END_REF]). These models integrate the information produced by the main speaker as well as information on the current state of the listener. However, they reach very low F-score results unless they restrict the prediction of feedbacks during pauses only.

Predicting the type of feedback For the task of predicting the type of feedback the training sample is a table containing for each observed feedback:

• The binary dependent variable (encoded as 1 if the feedback has the desired type an 0 otherwise)

• The predictors of binary type (e.g. absence or presence of a given POS trigram, absence of presence of a pause, ...) or continuous type (e.g. number of tokens).

The logit model takes this training table as input, the relevant predictors are identified and the final model is adopted which allows to compute for any combination of predictor values the predicted type of feedback in output through equation 2.

Results

We propose in the following to discuss the results obtained for different feature combinations and clustered by modality. For each combination of coefficients, the accuracy and the Cohen's kappa scores have been obtained by running a Monte Carlo cross-validation (on 100 trials with a ratio 80%-20% for the training versus the evaluation sample). Quoted errors are the 1σ standard deviation on the estimates of these parameters.

Feedback occurrence: Concerning the prediction of feedback occurrence, let's note first that our modeling leads to a good accuracy whatever the feature combination. Also note that in their work, [START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF] mention that the likelihood of occurrence of a feedback is 30% when involving all their predicting cues. As underlined previously, the main difficulty in this task is that classes are imbalanced and most of the input samples concern no-feedbacks, explaining the rather low kappa value. But also note that this value is significantly higher than random. Table 4 shows the emergence of 12 features: pauses, laughs, smiles (S3 and S4), 4 tone bigrams, 3 POS bigrams and number of tokens. Interestingly, each modality contribute to the prediction. Expression and visual features (laughs and smile) are unsurprisingly correlated with feedback realization, confirming the literature. In terms of prosody, the presence of pauses also confirm existing results. More precisely, we observe 4 tonal bigrams: DH, SS, SD and MD. The DH pattern could correspond to a rise intonation contour (to be compared with the L-H% contour in [START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF]). On its side, the MD bigram corresponds to a fall. This pattern can be find in several works, even though with different consequences: [START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF] indicate that this intonation contour is likely to result in turn change or a continuation by the same speaker (with no feedback) where [START_REF] Ward | Prosodic features which cue back-channel responses in english and japanese[END_REF] indicate that a region of low pitch lasting at least 110 milliseconds is considered as a feedback-inviting cue. The SD and SS bigrams could also refer to such feedback-inviting-cues. In terms of lexico-syntactic cues, the most important feature is the number of tokens. This information is in fact complex: it can be related to speech rate, but also to the syntactic structure (grammatical words are small, their number increase when the syntactic structure is more complex). In both cases, this information could be in relation with "completude": more tokens could be associated to a more complete unit. Generic/specific prediction: Table 5 shows the accuracy for predicting the different feedback subtypes. In this case, the classes are more balanced, leading to a better kappa. We obtain, in spite of the task difficulty, interesting results.

In this table, we present the different models for each subtype : generic vs. specific, and for the specific feedbacks positive vs. negative and expected vs. unexpected. In the +/-generic classification, the main features are visual (laughs, smiles and nods), which is in line with the literature. As for prosody, three patterns occur: DHL, DH, TL. The T tone refers to a maximum in the speaker's pitch range and DH refers to a rise. We know that high values of f0 are often considered more salient (focus, emphatic style) which could be more associated with specific feedbacks (conveying for example stance, emotion, etc.). When comparing the different combinations, the all features gives the best results. The POS bigrams tend to show the opening of a new structure, which could play in favor of generic Concerning the prosody, a short pause just before the feedback is the most salient cue for positive feedbacks. It is difficult to interpret this result, unless considering that positive responses are preferred and preferentially occur in a short delay. Note that surprisingly, the feature PositiveToken does not play a role in this task. Results concerning +/-expected information are less reliable. Prosody features show two tonal bigrams, LH and LU, corresponding to a rise that could be associated with a new information (that could be for example associated with a surprise feedback).

Conclusion

We have presented in this paper different models addressing for the first time both the prediction of feedbacks and of their types. Our approach shows that a multimodal combination of predictive features can lead to a good accuracy level and represent, to the best of our knowledge, a state of the art for this double classification task. This statistical modeling constitutes the first step for future systematic studies based on different machine learning techniques. In terms of application, this model has been implemented in an automatic dialogue system and is currently under evaluation.

Figure 1 :

 1 Figure 1: Description of feedback production

Table 1 :

 1 [START_REF] Gravano | Turn-taking cues in task-oriented dialogue[END_REF] Feedback Types

	Type	Modality	Feedback
	Generic Specific / agreement Specific / disagreement	Verbal Visual Bimodal Verbal Visual Bimodal Verbal Visual	oui, mh nod, smile nod+yeah, smile+ok oui, d'accord, ok nod, smile nod+yeah, nod+ooh non shake	Type Specific surprise Specific / fear	Modality Verbal Visual Bimodal Verbal Visual Bimodal	Feedback ah bon raising, frowning, tilt raising+no non, oh non frowning, raising, shake shake+no, frowning+non
		Bimodal	shake+no, shake+mh
	Speech	Short silent pause lasting at least 200ms and maximum 1200ms
			Long silent pause lasting at least 1200ms
			Bigrams and trigrams of tones
	Verbal	POS bigrams and trigams
			Discourse markers
			Positive, Negative, Concrete words
	Visual	Laughs, Smiles
			Nods

Table 2 :

 2

Predictive features 5, 20, 34, 22, 12, 38]), verbal (lexico-syntactic features

Table 3 :

 3 Feedback frequency per minute and per type

Table 4 :

 4 Prediction of feedback occurrences

Table 5 :

 5 Prediction of feedbacks subtypes feedbacks (continuers).

	Feat. combination	Type	Formula	Accuracy	Kappa
	All	generic	Laugh + Smile + Nod + DHL + Adv-Clit +	0.62 ±0.0369	0.255 ±0.067
			Det-N + N-Prep + DH + TL + Dem-V
		positive	Clit-V-Det + Pause(¡ 1s2) + Smile + Pd.Vm.	0.61 ±0.046	0.093 ±0.081
			+ Disc.Mark + SD + MS + Aux-V
		expected	V-Adv + LH + LU + N-Adj + Det-N + LUD	0.51 ±0.048	0.037 ±0.098
			+ PositiveToken
	Visual	generic	Laugh + Smile + Nod	0.59 ±0.035	0.229 ±0.054
		positive	Laugh	0.41 ±0.046	0.007 ±0.053
		expected	x	x	x
	Prosody	generic	DHL + DH + TL	0.50 ±0.034	0.069 ±0.049
		positive	Pause(¡ 1s2) + SD + MS	0.67 ±0.16	0.054 ±0.1
		expected	LH + LU + LUD	0.48 ±0.048	0.048 ±0.073
	Lexico-synt	generic	Adv-Clit + Det-N + N-Prep + Dem-V	0.57 ±0.036	0.134 ±0.073
		positive	Clit-V-Det + Dem-V + Disc.Mark + Aux-V	0.57 0.068	0.047 ±0.087
		expected	V-Adv + N-Adj + Det-N + PositiveToken	0.51 ±0.05	0.045 ±0.094