Salma Jamoussi
email: jamoussi@loria.fr

Kamel Sma¨ıli
email: smaili@loria.fr

Jean-P Aul Haton

From sp eec h to SQL queries : a sp eec h understanding system

Keywords: Speech understanding, seman tic measures, automatic extraction of concepts, word vector represen tation, Bayesian net works, AutoClass

In this pap er, we describ e our sp eech understanding system and we test it on two diff eren t applications. The prop osed system is a task sp ecific one and it concern especially oral database consultation tasks. In this work, we consider that the automatic sp eech understanding problem could be seen as an association problem between two diff eren t languages. At the entry , the request expressed in natural language and at the end, just before the interpretation stage, the same request is expressed in term of concepts. A concept represen ts a given meaning, it is defined by a set of words sharing the same seman tic prop erties. In this pap er, we prop ose a new Bayesian net work based metho d to automatically extract the underlined concepts. We also prop ose and compare three approac hes for the vector represen tation of words. We finish this pap er by a description of the post-pro cessing step during which we generate corresp onding SQL queries to the pronounced sentences and we connect our understanding system to a sp eech recognition engine. This step allows us to validate our sp eech understanding approac h by obtaining with the two treated applications the rates of 78% and 81% of well formed SQL requests.

interactiv e applications must be able to pro cess users spoken queries. It means they have to recognize what has been uttered, extract its meaning and give suitable answers or execute right corresp onding commands. In such applications, the speech understanding component constitutes a key step. Several metho ds were prop osed in the literature to clean up this problem and the ma jorit y of them is based on sto chastic approac hes. These metho ds allow to reduce the need of human expertise, however they require a sup ervised learning step which means a former stage of manual annotation of the training corpus [START_REF] Bousquet-V Ernhettes | Con text use to impro ve the sp eech understanding pro cessing[END_REF][START_REF] Lefèvre | Issues in the developmen t of a sto chastic sp eech understanding system[END_REF][START_REF] Pieraccini | Learning how to understand language[END_REF].

The data annotation step consists in segmenting the data into conceptual segments where each segment represen ts an underlined meaning [START_REF] Bousquet-V Ernhettes | Con text use to impro ve the sp eech understanding pro cessing[END_REF]. Within this step, we have to find first of all the list of concepts which are related to the considered corpus. Then, we can use these concepts to lab el the segments of each sentence in the corpus and finally, we can launch the training step. Doing all this in a manual way constitutes a tiresome and an expensive phase. Moreover, the manual extraction is prone to sub jectivit y and to human errors. Automating this task will th us reduce the human interv ention and will especially allow us to use the same pro cess when context changes. Our purp ose in this pap er is to fully automate the understanding pro cess from the input signal until the SQL request generation step.

In this pap er, we start by giving a brief description of the statistical approac h which constitutes the most used metho d for resolving the speech understanding problem. We present then the detailed architecture of our understanding system where we prop ose a new approac h to automatically extract the semantic concepts of the considered application. For this, we use a Bayesian net work for unsup ervised classification, called AutoClass and we expose three metho ds for the vector represen tation of words, these represen tations aim to help the Bayesian net work to build up effi cien t concepts. We test this metho d on two applications data and we compare the Bayesian net work performances with those obtained by the Kohonen maps and the K-means algorithm. Then, we will describ e the last stage of our understanding pro cess, in which we lab el the user requests and we generate the associated SQL queries. Finally, we use a speech recognition system to be able to treat sentences given in their signal forms. Tow kinds of results are given in this pap er. The first results are obtained when the system input is speech and the second ones concern the textual entry form.

The statistical approac h for the sp eec h understanding problem

A speech understanding system could be considered as a machine that produces an action as the result of an input sentence. Th us, the understanding problem could be seen as a translation pro cess, it translates a signal (represented by a sequence of words) into a special form that represen ts the meaning convoyed by the sentence. In a first time, the sentence is lab elled by a list of conceptual entities (often called concepts), these lab els constitute a useful intermediate represen tation which must be simple and represen tativ e. In a second time, this represen tation will be used to interpret semantically the sentence.

The speech understanding problem can be seen then as an association problem, where we have to associate inputs (e.g. speech or text) to their resp ective meanings represen ted by a list of concepts. A concept is related to a given meaning, it is given by a set of words expressing the same idea and sharing the same semantic prop erties. For example, the words plane, train, boat, bus can all corresp ond to the concept "transport means " in a tra vel application.

The step of interpretation consists in converting the obtained concepts to an action to be done as a final resp onse to the user. In order to achieve such a goal, we have to convert these concepts into a target formal command (e.g. an SQL query, a shell command, etc.). The figure 1 illustrates the general architecture of such speech understanding system, this model was given in [START_REF] Pieraccini | Learning how to understand language[END_REF] and it was included in several other works because of its eff ectiv eness and its simplicit y [START_REF] Bousquet-V Ernhettes | Con text use to impro ve the sp eech understanding pro cessing[END_REF][START_REF] Lefèvre | Issues in the developmen t of a sto chastic sp eech understanding system[END_REF]. The semantic transducer component is considered as the main module in a speech understanding system while it translates a given sentence to its conceptual form. In the literature, this step is often achieved by using the Hidden Markov Models technique. The aim is to find the concept list maximizing the likelihood P (C |A) where C represen ts a set of concepts and A is the acoustic represen tation of the pronounced sentence S :

Ĉ = arg max C P (C |A)
Using the Bayes formula we can transform this last equation into :

Ĉ = arg max S,C P (A|S )P (S |C )P (C )
The terms of this equation represen t three particular models :

• The acoustic model represen ted by the probabilit y of the acoustic observation A given the sequence of the sentence words S : P (A|S ) • The syntaxic model which is given by P (S |C ) : the probabilit y that we have a word sequence S given some meaning C . C represen ts a certain sequence of concepts. • The semantic model given by the probabilit y P (C ) of a concept sequence C .

The acoustic model is often maximized using some speech recognition techniques. For the understanding task only the tow last models are considered.

In this case we appro ximate these models to :

P (S |C ) i P (w i |w i-1 , . . . , w i-n +1 , c i ) P (C ) i P (c i |c i-1 , . . . , c i-m +1 )
Where the w i are the words of the sentence S and the c i are the concepts composing C . To simplify these equations, we often consider that n = 1 and m = 2 to obtain :

P (P h|C ) i P (w i |c i ) P (C ) i P (c i |c i-1 )
In this case, the HMM states will corresp ond to the concepts and P (c i |c i-1 ) is the transition probabilit y between the two states c i and c i-1 . P (w i |c i ) represents the probabilit y of observing the word w i at the HMM state c i . A very simple case of this modelisation problem is given by the next figure (figure 2). The problem with this metho d is the needed data for the training step. In fact, to compute our model probabilities we have to provide huge corpus of annotated data. The annotation step is achieved manually and it is considered as the most diffi cult task. In this pap er, we try to automate all the understanding pro cess especially to avoid these very heavy manual steps.

I C C F i-1 i
3 Our understanding system arc hitecture

In our work, we adopt the same general architecture as given in the figure 1 but we prop ose new techniques within each component. Moreover, we try to extract automatically our concepts in a preliminary step.

Our detailed system architecture is shown in the figure 3. It is composed of three principal components. The first one is a corpus pro cessing module, where we try to automatically extract the appropriate list of concepts by using a Bayesian net work. This step is the more crucial one, because we will use its output in all the other steps. The second and the third ones are those already defined in the figure 1. In our case the "Seman tic Transducer" is a sentence lab elling module where we associate to each word its semantic class. The "Represen tation Converter" is divided into two steps. The first one is the "Generic query pro duction" stage, it uses the sentence concepts to build up a generic query for this sentence. The second one is the "SQL query generation" stage, it uses the initial sentence and the generic query to set concepts and to provide the final corresp onding SQL query.

Ba yesian approac h for automatic concept extraction

The aim of this step is to identify the semantic concepts related to our application. The manual determination of these concepts is a very heavy task, so we should find an automatic metho d to achieve such a work. The metho d to be used must be able to gather the words of the corpus in various classes in order to build up the list of the appropriate concepts.

To reach our goal we used an unsup ervised classification technique. Among the unsup ervised classification metho ds, we tried the Kohonen maps and the K-means metho d. The obtained concepts were quite significant, but contained some "noise", it means that we found many words which did not have their place in the meaning expressed by these concepts. To solve this problem, we explored other metho ds and adopted the Bayesian net work technique because of its mathematical base and its powerful inference mechanism [START_REF] Cheeseman | Ba yesian classification (AutoClass): theory and results[END_REF]. A comparison between these metho ds performances will be given in the section 6.2. In this section, we describ e the Bayesian theory principle used for the clustering problem and we detail some calculation stages allowing us to find the concepts related to our training corpus.

In this pap er, we use a Bayesian net work conceived for the clustering problem and called "AutoClass". It accepts real and discrete values as input. As result, it provides for each input, its membership probabilities in all the found classes. AutoClass supp oses that there is a hidden multinomial variable which represents the various classes of the input data. It is based on the Bayes theorem expressed by :

p(H |D ) = p(H ) p(D |H ) p(D ) (1)
In our case, D represen ts the observed data, that means, the words to be classified. H is a hypothesis concerning the number of classes and their descriptions in term of probabilities. AutoClass tries to maximize the probabilit y p(H |D ), i.e. given D (the words of the corpus), we must select H (the set of concepts) which maximizes this probabilit y.

In our Bayesian net work, a word x i is given by a vector of K attributes, x ik , k ∈ {1...K }. A concept C j is also describ ed by K attributes, each one is modeled by a normal Gaussian distribution.

-→ θ j k is a vector parameter describing the attribute number k of the concept number j , C j . It contains two elements, the distribution mean µ j k and the variance σ j k . For the whole concept, this vector is noted -→ θ j and it contains the -→ θ j k of all the attributes of the concept C j . The probabilit y that a word x i belongs to the concept C j , called the class probability is noted π j and it also constitutes a descriptiv e parameter of the concept C j .

Th us, we defined our net work parameters, the data D represen ts the words as a vector -→ x with I elements including all the x i . The hypothesis H corresp onds to the description of the concepts and it is represen ted by three elements, the number of concepts J and the two vectors -→ π and -→ θ which contains resp ectively π j and -→ θ j of all the concepts. AutoClass divides the concept identification problem into two parts : the determination of the classification parameters ( -→ π and -→ θ ) for a given number of concepts and the determination of the number of concepts J . This last problem requires several appro ximations which are explained in [START_REF] Cheeseman | Ba yesian classification (AutoClass): theory and results[END_REF]. In what follows, H will only represen t the vectors -→ π and -→ θ . Replacing D and H by their values in the equation 1 we obtain :

p( -→ θ , - → π | -→ x ) = p( -→ θ , -→ π ) p( -→ x | -→ θ , - → π ) p( -→ x ) (2)
Where p( -→ θ , -→ π ) is the prior distribution of the classification parameters, its calculation is well describ ed in [START_REF] Cheeseman | Ba yesian classification (AutoClass): theory and results[END_REF]. The prior probabilit y of words, p( -→ x ) can be computed directly, it is simply considered as a normalizing constan t. Here, we are interested in the calculus of the probabilit y p( -→ x | -→ θ , -→ π ) which represen ts the likelihood function of the data.

It is known that -→ x is a vector represen ting all the words of the training data, the probabilit y of this vector is obtained by the pro duct of the probabilities of all the words separately as shown in the following equation :

p( -→ x | -→ θ , -→ π ) = I i=1 p(x i | -→ θ , -→ π ) (3) p(x i | -→ θ , -→ π )
is the probabilit y of observing the word x i indep endently of the concept to which it belongs. It is given by the sum of the probabilities that this word belongs to each concept separately , weighted by the class probabilities as indicated by the following equation :

p(x i | -→ θ , -→ π ) = J j =1 π j p(x i |x i ∈ C j , -→ θ j ) (4)
Since the word x i is describ ed by K attributes, with the strong assumption that these attributes are indep endent, the probabilit y p(x i |x i ∈ C j , -→ θ j ) can th us be written in the following form :

p(x i |x i ∈ C j , -→ θ j ) = K k=1 p(x ik |x i ∈ C j , - → θ j k ) (5)
AutoClass models each real attribute by a normal Gaussian distribution represented by the vector -→ θ j k which contains two parameters µ j k and σ j k . In this case, the class distribution p(x ik |x i ∈ C j , -→ θ j k ), can be written like this :

p(x ik |x i ∈ C j , µ j k , σ j k ) = 1 √ 2π σ j k exp - 1 2 x ik -µ j k σ j k 2 (6)
Once this class distribution is determined, we only have to seek for the concept parameters which maximize the starting probabilit y p( -→ θ , -→ π | -→ x ) and find the optimal concepts related to our data [START_REF] Cheeseman | Ba yesian classification (AutoClass): theory and results[END_REF].

Vector represen tations of words

In this section we present three diff eren t approac hes to represen t words in vectorial asp ect. This represen tation, which must be semantically significant, constitutes a key stage in the understanding pro cess. In fact, according to this represen tation, the Bayesian net work will decide of words to group in the same class in order to build up the needed list of concepts.

Word context

One word can have several features but only few of them are relevant for a good semantic represen tation. In a first step, we decided to associate to each word its diff eren t contexts. We consider that if two words have the same contexts then they are semantically similar. In this approac h, a word will be represen ted by a vector of 2 × N elements containing the N left context words and the N right context words. Figure 4 shows how we associate for each word its left and right bigram contextual represen tation. Using this vector represen tation of words with our Bayesian net work, we obtain many classes represen ting good semantic concepts, but an imp ortan t overlapping has been noticed. Moreover, we had diffi culties in controlling the number of concepts.

Similarity vector representation

find more homogeneous concepts, we completely changed the vector structure of each word. We used the average mutual information measure which tries to find contextual similarities between words.

In this approac h, we associate to each word a vector with M elements, where M is the size of the lexicon. The j th element of this vector represen ts the average mutual information between the word number j of the lexicon and the word to be represen ted (equation 7). W i = [I (w 1 : w i ), I (w 2 : w i ), . . . , I (w j : w i ), . . . , I (w M : w i )] (

This vector expresses the similarit y degree between the word to represen t and all the other words of the corpus. The formula of the average mutual information between two words w a and w b is given by :

I
Where P (w a , w b ) is the probabilit y to find w a and w b in the same sentence, P (w a | w b ) is the probabilit y to find w a knowing that we already met w b , P (w a ) is the probabilit y of the word w a and P (w a ) is the probabilit y of any other word except w a .

By using this vector represen tation, the Bayesian net work homogeneous semantic classes. A class is made up of words sharing the same semantic prop erties. The number of classes is very coherent with our application. This represen tation also enables us to solve the problem of the overlapping between concepts. However some imp are still present and we will try avoid them with next prop osed word represen tation.

Combinaison : context and

In this approac h we combined the two preceding represen tations in order to improve results. In the first approac h we work on the occurrence level where we directly exploit information related to the word context. In the second one, we use a measure to seek for similarities between words. We can easily notice that the information used in these two metho ds is diff eren t but complementary .

To combine these two metho ds, we decided to represen t each word by a matrix M × 3 of average mutual information measures. The first column of this matrix corresp onds to the preceding of average mutual information (see section 5.2), the second column ts the average mutual information measures between the vocabulary words and the left context of the word to be represen ted. The third column determined by the same manner but it concerns the right context. The j th value of the second column is the weighted average mutual information between the j th word of the vocabulary and the vector constituting the left context of the word W i . It is calculated as follows:

I M M j (C i l ) = w l ∈ L W i I (w j : w l ) × K wl w l ∈ L W i K wl (9)
Where I M M j (C i l ) is the average mutual information between the word w j of the lexicon and the left context of the word W i . I (w j : w l ) represen ts the average mutual information between the word number j of the lexicon and the word w l which belongs to the left context of the word W i and K wl is the number of times where the word w l is found in the left context of the word W i . The word W i th us represen ted by the matrix shown in the figure 5. The matrix used to represen t a word in the corpus exploits a maximum number of information that can be related to this word. It considers its context and its similarit y with all the other words of the lexicon. a representation could help the Bayesian net work to classify the words and allows to considerably improve results. We obtain a coherent list of concepts. We decided to keep these ones for the rest of the understanding treatmen t.

Exp erimen tal conditions

In our work, we are interested in two kinds of applications. The first one is a bookmark consultation application where we use the corpus of the Europ ean pro ject MIAMM. The aim of this pro ject is to build up a platform of an oral multimo dal dialogue. The corpus contains 71287 diff eren queries expressed in French. Each query expresses a particular manner to request the database. Some examples of these queries are given in the table 1.

The second application concerns the purse and it is related to a french pro ject called IVOMOB that aims to industrialize the techniques of speech recognition and speech understanding. The training corpus that we used 51864 queries written in French and expressing many manners to request the purse database. Some examples of these queries are also given in table 2. Each training corpus contains almost one hundred vocabulary words after eliminating all the to ol words as well as the words having weak frequency. Our aim in this step is to cluster all these vocabulary words to form the semantic concepts of the treated application.

The evaluation method for the concept extraction step

Making an objective evaluation of semantic concepts is a very hard task. In fact, if we want to decide if a concept is correct or not, we can found as much resp onses as asked people. This kind of question dep ends on many factors like the application context, the asked people, the words nature, etc.

In our case, we use an evaluation measure well known and used in some similar tasks which is the effi cacit measure. This latter aims to find a kind of percentage of correct words in each found concept. Consequen tly, we need to dra w up manually reference list of concepts that we consider perfect where the words are very well classified. In our work, we define reference concepts for the MIAMM application and 11 reference concepts for the IVOMOB application. We assume that each reference concept is noted C i . To compute the effi cacit y of an obtained list of n concepts (C j, 1≤ j ≤ n ), we first need to define the recall and the precision measures as :

r ecall(i, j ) = n ij N i pr ecision (i, j ) = n ij N j
Where n ij is the number of words present in both concepts C i and C j . N i is the total number of words of the concept C i and resp ectively for N j .

Th us the effi cacit y F (i, j ) will be defined as :

F (i, j ) = 2 × pr ecision (i, j ) × r ecall(i, j ) pr ecision (i, j ) + r ecall(i, j )

This effi cacit y measure concerns only two concepts C i and C j . To compute the effi cacit y related to the whole reference concept C i we just seek for the maximum effi cacit y value obtained with this latter :

F (i) = max j F (i, j )
For the whole list of the reference concepts the final value of effi cacit y is computed as a weighted mean :

F = i p i × F (i)
Where p i represen ts the weight of the concept C i and it is given by :

p i = N i k N k 6.

Results

As said ab ove, we consider that the task of extracting the semantic concepts of an application consists in clustering its vocabulary words to build up various classes which represen t the concepts. We then test two clustering metho ds and we compare their performances with those achieved by the Bayesian net work.

The tested metho ds are the Kohonen maps and the K-means algorithm.

To evaluate and compare the concepts obtained by each clustering metho d we use the effi cacit y measure that we defined in the preceding section.

We also compare the three vector represen tations of words that we prop osed in th e section 5. In the next table (table 3), we only give results obtained by the similarit y vector represen tation and the combined matrix represen tation. In fact, context word represen tation can be only used by the Bayesian net work and we can not determine its results with the Kohonen and the K-means metho ds. We just notice that when using these kind of represen tation with AutoClass we obtain many overlapping concepts some times not semantically represen tativ es. The achieved effi cacit y values are 76% and 74.5% resp ectively with the applications MIAMM and IVOMOB. 3 Effi cacities values obtained with the concepts found by each clustering metho d and each vector represen tation with MIAMM and IVOMOB applications.

Finally, we notice that using the combined matrix represen tation of words, the Bayesian net work the best results and finds a coherent lists of concepts which are perfectly related to the considered applications. In fact, using this metho d we achieve good results and we obtain 86.3% and 89.3% as effi cacit y rates resp ectively the MIAMM and the IVOMOB applications.

P ost-pro cessing step

The last step consists in providing the SQL queries associated with the input textual requests. During phase, we start by the request interpretation. In fact, if we have all the concepts which govern our application, we can aff ect to each request its suitable list of concepts by associating each word its corresp onding semantic class. Since our concepts do not overlap, lab elling the requests does not present any risk of ambiguit y.

Generic query representation

Our goal is to provide at the end the corresp onding SQL query which can answer the user request. In the literature this task is usually ignored or done in an ad-ho c manner. In these cases, very complicated inference mechanisms are implemented to obtain final SQL queries.

In our case, we choose to divide this tasks into two steps. In the first one, we try to automatically build up a kind of generic queries having the same structures than those written in SQL. This generic query pro duction constitutes the first step in the "Represen tation Converter" (see figure 3). The principal function of this module is to translate the conceptual represen tation of the sentence into a query representation where the concepts tak e the places of tables and conditions as shown in the figure 8. For example, if we find the concept "Date", we don't know the value of this date but, we can indicate in the generated query that there is a condition on the date. Therefore, the generated query can be written as : select Object from Table Bookmark where Condition Date ;

To achieve this task with an automatic manner, we define a concept hierarc hy which is quite similar to the general structure of an SQL query. The figures 6 and 7 show how we can organize the MIAMM and IVOMOB concepts in a same hierarc hy to fit the SQL queries structure. In these figures, we place the obtained concepts at the tree leaf level and we dra w up the matc hing between each hierarc hy part with an SQL query In this way, we can automatically build generic queries by replacing each concept by its corresp onding SQL phrase.

SQL query generation

As a last phase, we set each concept, in the generic request, by its value deduced by going back to the initial sentence. This is done by a pattern matc hing mechanism which retriev es the prop er object from the sentence and replaces it by the needed database attribute in the final SQL query. This module is a task specific one. In fact, such inference mechanism is very specific one and can not be automated because it has to resp ect the database structure and the initial request of the user. Obviously, this task can't be done in an automatic way. That's why we implement a simple inference engine to obtain at the end well formed SQL queries that we can carry out to extract the required data. This is also shown as a last step in the figure 8.

The speech recognition step

The last step of this work consists in integrating the understanding module in a real platform of automatic speech recognition. For that, we use the recognition output as an input for our understanding module. In our experiments, we use the automatic speech recognition system ESPERE (Engine for SPEech REcognition) developed in our team [START_REF] Fohr | The automatic sp eech recognition engine ESPERE : exp erimen ts on telephone sp eech[END_REF] and based on a Hidden Markov Model (HMM). We choose the following acoustic parameterization: 35 features, namely 11 static mel-cepstral coeff etions (C 0 was removed), 12 delta and 12 delta delta. The chosen HMM is 3 states multigaussien context indep endent. Two bigram language have been trained on the MIAMM and the IVOMOB corp ora.

To adapt the system to our experimental platform, we added some functionalities use it in a real context. We also remove noise at the beginning and at the end of each sentence. By this way, we decrease the insertion rate of the recognition step.

At the end, our oral understanding system is operational. As input, queries can be given as a signal or a text. The output of this palteform is a SQL query which fits perfectly the user's request. In other words, we consider that a system understands what has been uttered if the answer retriev ed from the database via the SQL command corresp onds to what the user asked for. For test we use 400 sentences pronounced by 4 diff eren t speakers for the MIAMM application and 200 sentences pronounced by 2 speakers for the IVOMOB application. It worths to be mentioned that the test sentences are very diff eren t from those used in the training step.

To illustrate the various stages followed in order to generate a good SQL query, an example is given in figure 8 concerning a bookmark consultation request (MIAMM application).

Show me the bookmarks that I used before December 2001

Concepts identification

Order, Object, Bookmark, Date select Object from table_bookmark where condition_date ;

Generic query production SQL query generation

select * from bookmarks where date < #01/12/2001# ;

1.

2.

3.

Fig. 8. Treatmen t sequence : from a natural language request to the corresp onding SQL query.

The obtained results are very encouraging. In fact, with the MIAMM application we achieve a rate of 76.5% of concepts well detected and a rate of 78% of correct SQL requests. Although these results are quite high, the speech recognition system gives only a performance of 62%. When entry is text, the understanding performance reaches 92%. In spite of the recognition errors the understanding speech system we developed yields a good result. So many works have to be done in order to improve the results and to obtain similar results to those with a text entry. Obviously, eff orts have to be done on both speech recognition and understanding pro cess.

Conclusion

In this article, we consider that the automatic speech understanding problem can be seen as an association problem between two diff eren t languages, the natural language and the concept language. Concepts are semantic entities gathering a set of words which share the same semantic prop erties and which express a given idea. We prop osed a Bayesian net work based metho d to automatically extract the concepts, as well as an approac h for automatic sentence lab elling and an engine for generating SQL queries corresp onding to the user requests.

The concept extraction and the sentence lab elling tasks usually carried out manually. They constitute then, the most delicate and the most expensive phase the understanding pro cess. The metho d suggested in this article allows us to avoid the need for the human expertise and gives good results in terms of concepts viabilit y and relevant retriev ed SQL requests. At the end, we obtain 92% and 92.5% of correct SQL queries on the test corp ora of the two treated applications. The prop osed metho d can also be used for several other research fields that use the semantic classification : text categorization, information retriev al and data mining.

We also integrated our understanding module with a speech recognition system in order to carry out a complete interactiv e application. In of a speech recognition rates of 62% and 65.2%, we achieve final understanding performances resp ectively of 78% and 81%. These results show that the understanding pro cess we developed is robust with the speech recognition system errors.

We plan to extend the post-pro cessing module to make it able to react visa-vis new key words not included in the concepts. It is then necessary that our model be able to add new words to the appropriate concepts within the exploitation step. We also plan to integrate our understanding module in a real plate-form of man-mac hine oral dialogue.

Fig. 1 .

 1 Fig. 1. General architecture of a sp eech understanding system.

Fig. 2 .

 2 Fig.2. A simple conceptual modelisation example using HMM.

Fig. 3 .

 3 Fig. 3. Our detailed understanding system architecture.

Fig. 4 .

 4 Fig. 4. The bigram contextual represen tation of words.

  (w a : w b ) = P (w a , w b ) log P (wa | w b ) P (wa )P (w b ) + P (w a , w b ) log P (wa | w b ) P (wa )P (w b ) + P (w a , w b ) log P (wa | w b ) P (wa )P (w b ) + P (w a , w b ) log P (wa | w b ) P (wa )P (w b )

Fig. 5 .

 5 Fig. 5. Represen tation of the word W i by the combined metho d.

  Give me the course level of the Alcatel compan y. Can you provide me the action level of Alcatel compan y. I need the progression of the minim um level of the Alcatel group course. I would like to know the BNP course evolution. I need the course level of the BNP compan y by mail. Can you give me by fax the most high course level of BNP .Just the maxim um of the Alcatel action.

Table 1

 1 Some examples of queries in the MIAMM corpus.Show me the conten ts of my bookmarks.I would like to know if you can tak e the conten ts that I prefer. Do you want to select the titles that I prefer. Is it possible that you select the first of my bookmarks.

	Is it possible to indicate me a similar thing.
	Can you show me only December 2001.
	It is necessary that you prin t the list that I used early this morning.

Table 2

 2 Some examples of queries in the IVOMOB corpus.

  The same remarks can be done with the IVOMOB results where we achieve a rate of 79.1% of concepts well tected and a rate of 81% of correct SQL requests when the speech recognition system gives only performance of 65.2%. With textual entries, the understanding performance reaches 92.5%. results are resumed in the table 4. The speech understanding system developed by Pieraccini[START_REF] Pieraccini | A speech understanding system based on statistical represen tation of seman tics[END_REF] on ATIS corpus (Air-T Information Services) correctly answers 141 queries from a total test set of 195 sentences which is over 72% success rate. With a speech input of the same test set, the system gives more than 50% as understanding rate. Obtained results with the two applications MIAMM and IVOMOB within each understanding step.

			MIAMM	IV OMOB
	The test corpus size	400	200
	Num b er of sp eac kers	4	2
	Recognition rate	62%	65.2%
	Concept detection rate	76.5%	79.1%
	Understanding	rate	78%	81%
	Understanding	rate with text input	92%	92.5%
	Table 4


Giovanni De Magistris

Adrien Pajon
email: adrien.pajon@gmail.com

Sylvain Miossec

Abderrahmane Kheddar

V Ra

V Com

Optimized Humanoid Walking with Soft Soles

Keywords: Compliant soles, Deformation Estimator, Walking Pattern Generator, Humanoid robots, Optimized walking gait

In order to control more efficiently the feet-ground interaction of humanoid robots during walking, we investigate adding outer soft (i.e. compliant) soles to the feet. The deformation subsequent to the contact of the soles with the ground is taken into account using a new walking pattern generator and deformation estimator. This novel humanoid walking approach ensures that the desired zero moment point for stability requirement is fulfilled. We validate our new controller using the HRP-4 humanoid robot performing walking experiments with and without the estimator. Also, to test the robustness of our approach and to obtain low-energy walking, we performed different walking motions.

Introduction

Gaited or non-gaited walking is generated by alternating phases of contact creation and breaking with the environment [START_REF] Bouyarmane | Humanoid robot locomotion and manipulation step planning[END_REF]. With rigid links and without shock absorbing mechanism, impact forces with the ground must be thresholded through contact transitions with nearly zero speed. This considerably limits the walking dynamics. Therefore, compliant mechanisms are used in humanoid robotics to absorb shocks at impacts and prohibit their propagation along the entire structure that results in non-desirable vibrations and eventually unstable behaviors. One common solution is to add flexible mechanisms at the robot ankles [START_REF] Hirai | The development of honda humanoid robot[END_REF][START_REF] Kaneko | IEEE International Conference on Robotics and Automation[END_REF] that also protect the feet embedded force sensors. Unfortunately, such compliant mechanisms also act as passive joints whose deformations are hardly measurable [START_REF] Bruneau | Dynamic walk of a bipedal robot having flexible feet[END_REF]. In this way, the robot attitude is difficult to control, especially in complex maneuvers [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF]. Another solution is to add the compliance between the foot and ground contact, see early work by [START_REF] Yamaguchi | Multisensor foot mechanism with shock absorbing material for dynamic biped walking adapting to unknown uneven surfaces[END_REF]. In fact, humanoid robots have generally a thin rubber sole attached under each robot foot. Due to the thinness of this sole, the impact shocks are mainly absorbed by the ankle flexibility.

Alternatively, we favor removing the ankle flexibilities and investigate the use of thick soft soles under each foot of a humanoid robot (see Fig. 1). These soles not only absorb the impacts due to contact transitions, they also cast ground unevenness resulting in a relative increase of the contact surface. During walking, the compliance of these soles depends on the contact area variations. Also, since the compliance is put outside, it can be decoupled from the rigid dynamics. In order to generate balanced robot movements and directly control the contact with the environment, we developed a new Walking Pattern Generator (WPG) and deformation estimator. We validate our new methods for walking with compliant soles performing different experiments on HRP-4.

Problem formulation

In most simplified model-based planning and control, the ankle flexibility and the sole compliance are not modeled. They are left as 'perturbations' or 'uncertainties' to be tackled by the closed-loop controller. Therefore, controllers have to compensate for the errors in the attitude (i.e. free-floating orientation and position) due to the deformations of the flexible parts. In our control framework, we consider the model of the deformation that results from the contact of the soft sole with the ground using the Finite Element Model (FEM) and the mechanical laws of compliant contacts. To control the humanoid robot attitude during walking, we use the multi-objective Quadratic Programming (QP) control scheme illustrated in Fig. 2. In particular, we developed a new WPG that is coupled with a sole deformation estimator to achieve good balance during walking. We also experimented the HRP-4's humanoid robot performances for different WPG.

In section 3, we detail our new WPG; in section 4 we illustrate how we can estimate the foot position and orientation taking into account the deformations; then in section 5 we explain the QP for the whole-body control of the robot, which tracks the previously determined trajectories; in section 6 we show how our WPG and the deformation estimator improve humanoid robot performances; finally, we conclude and explain the future works in section 7.

Walking pattern generator

The Zero Moment Point (ZMP) and the ankle positions are directly linked (see later Eq. ( 33)). To obtain smooth right ankle and left ankle trajectories in the control framework of Fig. 2, we define the ZMP trajectories using a 5th order polynomial function. As will be explained in section 3.3, the WPG developed in this paper calculates also smooth center of mass (COM) trajectories. This new WPG optimizes both single support phase (SSP) and double support phase (DSP) using a QP optimization.

Several works formulate the walking gait as an optimization problem [START_REF] Herdt | Online Walking Motion Generation with Automatic Foot Step Placement[END_REF][8] [START_REF] Shin | Energy-Efficient Gait Planning and Control for Biped Robots Utilizing the Allowable ZMP Region[END_REF]. They use an inverted pendulum model with a point mass to study feet convex-hull ZMP and not the whole ZMP trajectory of each foot. Therefore, the study of the DSP trajectory and force of each foot are not previously studied in the literature.

For the remainder of the paper, we denote with subscript "1" the foot that leaves the floor at the end of DSP, and with subscript "2" the foot that comes in contact at the beginning of DSP.

Optimization criteria

We want to generate walking gaits that minimize the energy consumption E:

min. (E) (1)
From [START_REF] Shin | Energy-Efficient Gait Planning and Control for Biped Robots Utilizing the Allowable ZMP Region[END_REF], the energy consumption that accounts for the motor and the gear models can be expressed as:

E = j t f ti (a j τ 2 j + b j τ j qj + c j q2 j )dt (2)
where τ j is the force/torque, qj is the joint j velocity, and a j , b j , c j are the coefficients depending on joint j motor parameters and gear ratio.

Using the simplified robot model in Fig. 3a, Eq. ( 2) becomes:

E = t f ti (a p F 2 p +b p F p ṙ+c p ṙ2 +a a Γ 2 a +b a Γ a θ+c a θ2 )dt (3)
where subscript p denotes the prismatic joint and a the ankle joint, F p is the prismatic force, ṙ is the prismatic velocity, Γ a is the ankle torque and θ is the ankle velocity. To obtain a quadratic problem ( ṙ and θ are nonlinear functions of the ZMP and the COM), we keep only the force/torque terms (linear in terms of ZMP and COM parameters, see [START_REF] De Magistris | Humanoid walking with compliant soles using a deformation estimator[END_REF]). Therefore Eq. (3) becomes:

E = λ t f ti F COM (t) 2 dt + (1 -λ) SSP Γ aSSP (t) 2 dt + 1 2 (1 -λ) DSP ( Γ a1 (t) 2 + Γ a2 (t) 2 )dt +µ DSP ( PZMP1 (t) 2 + PZMP2 (t) 2 ) ) dt (4
) where λ and µ are the weights of each criterion; a SSP denotes the ankle of the support foot in SSP; a i refers to the ankle of the foot "1" or "2" in DSP; P ZMP = x ZMP y ZMP T are the ZMP coordinates. The motor characteristics of the simplified model of Fig. 3a are not known and therefore the weights λ and µ are determined from experimental trials (see section 6.3).

Using the cart-table model in Fig. 3b, the COM force can be expressed as:

F COM (t) = -F ZMP (t) = M PCOM (t) + g z ( 5
)
where PCOM is the COM acceleration, M is the robot mass and F ZMP (t) is the floor reaction force (ZMP force). Considering the foot in contact at a fixed position during the whole SSP, we obtain:

Γ a (t) = F ZMP (t) × (P a (t) -P ZMP (t)), (6)
where Γ a is the torque at the ankle, P a is the ankle position and P ZMP is the ZMP position. During the DSP the two feet are in contact and we obtain a similar equation to [START_REF] Yamaguchi | Multisensor foot mechanism with shock absorbing material for dynamic biped walking adapting to unknown uneven surfaces[END_REF] for Γ a1 (t) and Γ a2 (t). The force distribution constraint between feet in DSP is detailed in AppendixB.

For λ = 1 in (4) we obtain a walking pattern close to [START_REF] Herdt | Online Walking Motion Generation with Automatic Foot Step Placement[END_REF][9], while for λ = 0 we obtain a ZMP trajectory similar to [START_REF] Kajita | Handbook of Robotics, chap. Legged Robots[END_REF]. Intermediate values of λ give compromises between the two extreme behaviors. Our experimental trials reveal that the best obtained values for λ are not the extreme ones (i.e. λ = 0 or λ = 1).

In Eq. ( 4), P ZMP and P COM depend on F COM and Γ a whereas P ZMP1 and P ZMP2 depend only on Γ a . For this reason, optimal P ZMP1 and P ZMP2 tend to be close to the ankles, resulting in a ZMP trajectory under each foot going from ankle to heel, then to toe and finally go back to the ankle. For a foot with flexible sole, obtaining an oscillating foot orientation is suboptimal from the point of view of energy consumption due to the ankle velocity in (3). To solve this problem, we added the last term to the optimization criterion (4) dependent on P ZMP1 and P ZMP2 accelerations.

ZMP trajectories

We divided the walking gait in three cycles:

1. Start: the robot begins with a DSP. This cycle starts on the point ZMP init . 2. Walking: the robot does n steps alternating SSP and DSP. 3. Stop: the robot ends with a DSP. This cycle finishes on the point ZMP end .

Each SSP and start/stop DSP are defined by two C 2 polynomials and each remaining DSP is defined by one. Four via-points are defined to parametrize these phases (see Fig. 4). For a given sequence q, the SSP polynomials go from the via-point A q to the via-point B q and from the via-point C q to the via-point D q . The DSP polynomial goes from the via-point B q to the via-point C q . Therefore, we need m = 3n + 4 polynomials to model the whole ZMP trajectory:

x (j) ZMP (t) = 5 i=0 a (j) i (∆t j ) i (7)
where the superscript j denotes the j th phase, a i is the i th polynomial coefficient and ∆t j = t -t j-1 .

To find the polynomial coefficients of [START_REF] Herdt | Online Walking Motion Generation with Automatic Foot Step Placement[END_REF] and to obtain smooth trajectories, we enforced the boundary conditions in position, speed and acceleration at each via-point. In particular, we interpolated the polynomial functions between ZMP init → C 1 during Start, A q → B q , B q → C q and C q → D q during the sequence q and B n+2 → ZMP end during Stop. ZMP init and ZM P end are not necessarily equal to B 1 and C n+2 respectively. In addition, to reduce the number of via-points and using their boundary conditions, A q and D q are combined into one via-point A q .

COM trajectories

Using the cart table model in Fig. 3b, we can write the COM acceleration in x-axis as:

ẍ(j) COM = g z COM (x (j) COM -x (j) ZMP ) (8)
Substituting ( 7) into (8), the COM position can be expressed as following [START_REF] Morisawa | A Biped Pattern Generation Allowing Immediate Modification of Foot Placement in Real-time[END_REF]:

x (j) COM = V (j) c j + W (j) s j + 5 i=0 A (j) i (∆t j ) i (9)
where: j) and W (j) are the unknowns of (9) and a (j)

c j = cosh(ω j ∆t j ) s j = sinh(ω j ∆t j ) ω j = g z (j) COM = g/z COM A (j) i = a (j) i + (5-i)/2 k=1 b (j) i+2k a (j) i+2k for i = 0 • • • 3 a (j) i for i = 4, 5 b (j) i+2k = k l=1 (i + 2l)(i + 2l -1) w 2 j V (
i coefficients are known from [START_REF] Herdt | Online Walking Motion Generation with Automatic Foot Step Placement[END_REF]. This is the difference between (9) and the system of equations in [START_REF] Morisawa | A Biped Pattern Generation Allowing Immediate Modification of Foot Placement in Real-time[END_REF].

Details of how to find V (j) and W (j) are given in the AppendixA.

ZMP of each foot in DSP

As for the ZMP, we define ZMP 1 and ZMP 2 trajectories as C 2 polynomials. We assign to them a new via-point B q and C q respectively. To obtain the coefficients of these polynomials, we interpolated them between B q → B q and C q → C q .

Using the ZMP definition, we obtain:

Γ (j) ZMP (F ZMP1 ) + Γ (j) ZMP (F ZMP2 ) = 0 ( 10
)
where Γ (j) ZMP (F ZMPi ) is the torque of the reaction force F ZMPi under foot i ∈ {1, 2} applied on ZMP during the phase j.

Solving [START_REF] Morisawa | A Biped Pattern Generation Allowing Immediate Modification of Foot Placement in Real-time[END_REF] in the direction y we can write:

2 i=1 (x (j) ZMPi -x (j) ZMP )F z(j) ZMPi = 0 (11)
where x ZMP is the component x of the ZMP position and F z ZMPi is the component z of the reaction force under the foot i ∈ {1, 2}.

From [START_REF] Wieber | On the stability of walking systems[END_REF], we obtain the following relationship between ZMP 1 and ZMP 2 :

x (j) ZMP2 = x (j) ZMP - F z(j) ZMP1 F z(j) ZMP2 (x (j) ZMP1 -x (j) ZMP ) (12)
From ( 12), if ZMP 1 trajectories and the distribution between F z ZMP1 and F z ZMP2 are known, we obtain ZMP 2 trajectories. Details of how to find the force distribution between F z ZMP1 and F z ZMP2 , and the constraints that they must satisfy are given in AppendixB.

During the start and the stop sequences, to obtain a robust optimization algorithm with respect to the different walking parameters, we added new via-points to ZMP 1 and ZMP 2 trajectories, B q and C q respectively. Thus, during these sequences, ZMP 1 and ZMP 2 are defined by two consecutive polynomials interpolated respectively between B q → B q → B q and C q → C q → C q . On these new via-points, ZMP 1 and ZMP 2 trajectories satisfy the boundary conditions in position, speed and acceleration.

Optimization parameters

The WPG is formulated as a QP optimization problem. In this section, we detailed the optimization variables.

F COM , Γ a and PZMPi in ( 4) are linear functions of optimization variables. They can be written as:

F COM = A FCOM X + B FCOM , Γ a = A Γa X + B Γa , PZMPi = A PZMP i X + B PZMP i ( 13
)
where X is the vector of optimization variables:

X = x vp 0 x COM 0 x step 0 x vp 0 T (14)
x vp 0 represents the boundary conditions in position, speed and acceleration at via-points of ZMP (3m+3 parameters), x COM 0 represents the initial and final boundary condition in position and speed of COM (4 parameters), x step 0 represents the foot step positions (n -1 parameters) and x vp 0 represents the boundary conditions in position, speed and acceleration at via-points of ZMP 1 (3m + 6 parameters).

x vp 0 = x (1) 0 ẋ(1) 0 ẍ(1) 0 • • • x (m+1) 0 ẋ(m+1) 0 ẍ(m+1) 0 T x COM 0 = x (1) COM (t 0 ) ẋ(1) COM (t 0 ) x (m+1) COM (t m ) ẋ(m+1) COM (t m ) T x step 0 = x step 1 x step 2 • • • x step n-1 T x vp 0 = x (1) 1 ẋ(1) 1 ẍ(1) 1 • • • x (m+2) 1 ẋ(m+2) 1 ẍ(m+2) 1 T
We imposed:

• Initial and final feet positions In order to generate feasible trajectories, the ZMP trajectories must be inside the support convex hull [START_REF] Wieber | On the stability of walking systems[END_REF]. It is defined by the area of the support foot. To obtain this area, we added inequality constraints to the QP. During the SSP, this condition is linear and can be integrated as inequality constraints into a QP. During the DSP, this condition is not linear. However, if ZMP 1 and ZMP 2 trajectories are in the support convex hull defined by the areas of each foot, the stability condition for the ZMP trajectories is fulfilled. Therefore, these conditions are linear and can also be integrated in the QP optimization.

•
Based on human walking results in [START_REF] Popovic | Ground reference points in legged locomotion : definitions, biological trajectories and control implications[END_REF] and for ensuring safe walking w.r.t the robot lack of precision [START_REF] Kajita | Handbook of Robotics, chap. Legged Robots[END_REF], we defined a feet support areas within a security margin of 5 -10% of the foot length. To avoid self-collisions, we defined a minimal distance between the robot feet (3 cm). To avoid stretched legs' singularity, we chose a maximum step length (30 cm). To respect the boundary condition for the COM position and velocity, we added the equality constraints (A.2) and (A.6) to the optimization.

WPG summary

The problem that we solve is: min

X (E)
subject to equality and inequality constraints [START_REF] Bonnet | The finite element method in solid mechanics[END_REF] where the criteria E comes from (4) and it is based on a simplification of the energy consumption, and the optimization variables X include the ZMP limit conditions, the COM initial and final conditions and the feet positions [START_REF] De Magistris | Design of optimized soft soles for humanoid robots[END_REF].

The inequality constraints are:

1. ZMP ∈ the convex hull during the SSP 2. ZMP 1 and ZMP 2 ∈ the convex hull during the DSP 3. Maximum and minimum foot step stretching distance

The equality constraints are:

1. Initial robot state as condition on the first via-point of ZMP and ZMP 1 and on the initial condition of the COM. 2. Final robot state as condition on the last via-point of ZMP and ZMP 1 (optional ) 3. Final condition on the COM as a goal to reach or condition on the COM over the whole walk (e.g. COM velocity...) (optional ) 4. Foot step placement (optional, e.g. used to define a specific path...)

Deformation estimator

In this section, we develop an algorithm that calculates the ankle position/orientation and the deformation of the flexible sole [START_REF] De Magistris | Humanoid walking with compliant soles using a deformation estimator[END_REF] to match the interaction force F d ZMP (t) and the ZMP trajectory P d ZMP (t) given by the WPG (see the control scheme 2).

The Fig. 6 shows this algorithm. It is divided in two main blocks: frictional contact problem (see section 4.1) and update sole position and orientation (see section 4.2). For a more detailed presentation, see also [START_REF] De Magistris | Design of optimized soft soles for humanoid robots[END_REF].

To simulate the flexible behavior of the sole, a static linear FEM is used [START_REF] Bonnet | The finite element method in solid mechanics[END_REF]. This model is valid for small deformations. Indeed, due to the dense shape of the soles, their deformations are relatively small compared to their size. In future works, we may consider using a nonlinear large-deformation model to improve precision. Static simulation is a good approximation since F d ZMP and P d ZMP variations are negligible w.r.t the sole dynamic response. Ω is the sole shape meshed with 1494 tetrahedron elements (see meshed sole in Fig. 1), Ω I is the interior volume of Ω, ∂Ω gather the surface S D attached to the rigid robot foot (Dirichlet surface) and the surface S S that could be in contact with the ground. S D and S S contain two disjoint set of nodes:

∂Ω = S D ∪ S S , with S D ∩ S S = ∅ (16)
The static linear FEM gives a relationship between the node displacements of Ω and the external forces:

K l U = l F ( 17
)
where l F ∈ R N is the vector of nodal forces and l U ∈ R N is the vector of the node displacements. The superscript l before a letter denotes a vector expressed in the sole frame and no superscript means that the vector is expressed in the world frame.

From the definition, the Dirichlet nodes have a known null displacement. Thus, the system (17) can be rewritten:

  K DD K DI K DS K ID K II K IS K SD K SI K SS     0 l U I l U S   =   l F D l F I l F S   ( 18
)
where I, D and S denotes respectively nodes of Ω I , S D and S S .

As in [START_REF] Bro-Nielsen | Real-time volumetric deformable models for surgery simulation using finite elements and condensation[END_REF], we suppose that the internal interaction forces l F I are zeros and therefore S D is the only surface with nodal forces.

The system (18) can be rewritten:

K II K IS K SI K SS l U I l U S = 0 l F S (19)
Based on [START_REF] Bro-Nielsen | Real-time volumetric deformable models for surgery simulation using finite elements and condensation[END_REF] and [START_REF] Alart | Time-stepping for threedimensal rigid body dynamics[END_REF], a condensed linear elasticity law is defined:

l U S = K SS -K SI K -1 II K IS -1 l F S = K -1 S l F S = C S l F S l U I = -K -1 II K IS l U S (20)
where K S is the stiffness surface matrix and C S is the compliance surface matrix.

In order to obtain the shape deformation from [START_REF] Duriez | Realistic haptic rendering of interacting deformable objects in virtual environments[END_REF], the contact with the ground directly induces l U S , which engender the computation of l U I .

Frictional contact problem

The frictional contact problem finds the contact space displacement vector δ and contact force vector F at each time step t s . This problem must be constrained to verify the Signorini's and Coulomb's laws.

The inputs of this algorithm are:

1. l P free : node positions 2. C S : surface compliance matrix from (20) 3. l P ts-1 : node positions at the previous time step 4. O l : foot position vector 5. Υ = (θ, φ, ψ): Euler rotation angles of the foot

The outputs of this algorithm are:

1. δ: relative position of the sole nodes to its ground projection (see Fig. 5). It is called node contact space displacement vector 2. F : node contact force vector For a contact point α, δ α is defined by:

δ α = P α -Q α ( 21
)
where Q α is the normal projection of α on the ground. The normal component of a vector is denoted by the subscript n such as v n = nv where n = 0 0 1 ; the tangential components of a vector are denoted by the subscript t such as v t = tv, where t = 1 0 0 0 1 0 . The superscript T denotes the transpose operator. The contact model used for calculating the deformation of the sole takes into account two very important mechanical laws: (i) the Signorini's law is used to enforce the non-interpenetration (in the computation) of the sole with the ground, and (ii) the Coulomb's law which defines the dry friction. Based on (21), we have:

1. Signorini's law: 0 ≤ F n,α ⊥ δ n,α ≥ 0 ( 22
)
where F α is the contact force applied to a contact node α, subscript n is its normal component. 2. Coulomb's law. We can distinguish two state conditions: stick and slip motion. (a) Stick condition:

F t,α < µ|F n,α |, δ t,α = 0 (23)
(b) Slip condition:

F t,α = -µF n,α δ t,α δ t,α (24)
where µ is the friction coefficient.

In our simulation, we use a quasi-static contact model. Quasi-static means that at a given instant in time we can assume the problem is static. This assumption works well because the dynamics of the sole deformation are fast compared to the dynamics of the limit conditions applied to it (here Force and ZMP).

The foot velocity is not considered in the static case. In the quasi-static case taking into account the slippage, the foot velocity is considered by computing the state just after the sole stop to slip. Therefore, slip occur until a static state is reached. This gives a non-reversible sole simulation and a sole state that depends on the history of its deformation. In order to obtain P α and Q α and solve this problem, the displacement U S of equation ( 20) (in absolute frame) is used:

U S = P -P free (25)
and then, the sole surface node positions are defined as:

P = HC S H T F + P free def = W F + O l + H l P free (26)
W is the Delassus' operator [START_REF] Moreau | Numerical Treatment of Contact and Friction: The Contact Dynamics Method[END_REF];

H = diag(R, . . . , R),
where R is the foot orientation matrix defined by Υ = (θ, φ, ψ).

During the contact nQ α = 0 and then the normal component of δ α is:

δ n,α = nO l + nR l P free α + n cn β=1 (W αβ F β ) def = δ free n,α + n cn β=1 (W αβ F β ) (27)
where

W αβ def = W t W n def = W tt W tn W nt W nn is the 3 × 3 Delas-
sus' operator coupling the contacts α and β and c n is the number of contact nodes. The tangential component of δ α is:

δ t,α = tO l,α + tR l P free α + t cn β=1 (W αβ F β ) -tQ α ≈ tO l,α + tR l P free α + t cn β=1 (W αβ F β ) -tP (q-1) α def = δ free t,α + t m β=1 (W αβ F β ) (28)
If at t s -1 a node α is already in contact, the reference node at t s is P (ts-1)

α . If α is not in contact at t s -1, the reference contact node at t s is approximated by the normal projection of α on the ground at t s -1. In reality, the contact creation occurs at an intermediate time step between t s -1 and t s . The contact creation position could be found with a variable step size and an event driven simulation. This can be computationally costly for a large number of new contact nodes. However, with a small timestep, the approximation error is negligible.

Based on ( 27) and (28), a contact node α has a contact space displacement defined by:

δ α = O l + R l P free α - P (ts-1) t,α 0 + cn β=1 W αβ F β def = δ free α + cn β=1 W αβ F β (29)
The concatenation of (29) for all α gives the frictional contact problem solved by:

δ = O l + H l P free -Q + W F (30)
where

Q = P (ts-1) t,1 0 T • • • P (ts-1) t,cn 0 T T .
The system (30) has 3c n equations with 6c n unknowns defined by δ and F . Additional 3c n equations are obtained considering that each node in contact α can be in two different conditions:

1. Stick: [START_REF] Wieber | Trajectory free linear model predictive control for stable walking in the presence of strong perturbations[END_REF], δ α = 0 ⇒ 3c stick equations where c stick is the number of contacts in stick case 2. Slip: δ n,α = 0 ⇒ c slip equations where c stip is the number of contacts in slip case.

To define the frictional contact problem, 2c slip equations are still needed. They are obtained using the Coulomb's law [START_REF] Winter | Human balance and posture control during standing and walking[END_REF] in [START_REF] Jean | Numerical methods for three dimensional dynamical problems[END_REF]. From [START_REF] Jean | Numerical methods for three dimensional dynamical problems[END_REF] and (29), a system is built for each contact node α and solved using a Newton's method [START_REF] Alart | Time-stepping for threedimensal rigid body dynamics[END_REF].

Finally, an iterative Gauss-Seidel method in [START_REF] Duriez | Realistic haptic rendering of interacting deformable objects in virtual environments[END_REF][START_REF] Jourdan | A gauss-seidel like algorithm to solve frictional contact problems[END_REF] is used to solve the system (30). This method is very fast and applicable for real-time solution [START_REF] Duriez | Realistic haptic rendering of interacting deformable objects in virtual environments[END_REF].

Update sole position and orientation

In the frictional contact problem of the previous section, the foot position and orientation are the inputs when the contact forces and the contact space displacements are the outputs. Here, a new algorithm is developed to find the good foot position and orientation to match a desired ZMP force F d ZMP (t) and position P d ZMP (t) (see Fig. 6). Seeing the sole as a (complex 6-dof) spring, the principle is to invert the relation between the spring deformation and the force applied to it.

The inputs of this algorithm are the geometric and mechanical properties of the sole and the desired ZMP: i) l P free , ii) C S , iii) l P ts-1 , iv) F d ZMP , P d ZMP . The outputs of this algorithm are the foot position and orientation: i) O l , ii) Υ = (θ, φ, ψ). Recall that the contact node forces F and the displacements δ are computed solving the frictional contact problem explained in the previous section.

Therefore, we can describe our algorithm using a function f :

f (O l , Υ) =   F d ZMP (t) P d ZMP (t) Γ d PZMP,n = 0   (31)
We impose null vertical torque at ZMP Γ d PZMP,n = 0. From the system (30) and the Signorini's and Coulomb's laws, 6c n + 6 unknowns (considering now position and orientation of the sole as unknowns) and 6c n equations are given (see section 4.1). The remaining 6 equations are obtained using the total force at ZMP F ZMP :

F ZMP = cn α=1 F α (32)
and the ZMP definition:

  Γ PZMP,t1 = 0 Γ PZMP,t2 = 0 Γ PZMP,n   = cn β=1   (P t2,β -P ZMP,t2 ) • F n,β -(P t1,β -P ZMP,t1 ) • F n,β (P t1,β -P ZMP,t1 ) • F t2,β -(P t2,β -P ZMP,t2 ) • F t1,β  
where Γ PZMP is the resultant torque at ZMP. The subscripts t 1 and t 2 denote respectively the first and second component of the vector.

To solve the problem (31), we use a Newton's method. A differential relationship between (F ZMP , P ZMP , Γ PZMP,n ) and (O l , Υ) is used to derive a Newton step:

O (ts) l Υ (ts) = O (ts-1) l Υ (ts-1) -J -1   F ZMP -F d ZMP P ZMP -P d ZMP Γ PZMP,n   ( 33
)
where J is the Jacobian matrix. The algorithm in Fig. 6 loops on frictional contact problem followed by a Newton step that gives a new foot position and orientation. To increase the speed of the algorithm, the precision of the frictional contact problem is increased with the convergence of the Newton steps.

The reference ankle position and orientation in the control scheme 2 is computed during the contact phase with:

P a = O l + R l P a (34)
where l P a is the ankle position in the sole frame.

During the SSP, ankle positions and orientations of the foot in the air are computed by interpolating with 5 th order polynomials the absolute ankle orientation, position, velocity and acceleration between take-off and landing phases.

Initial conditions

The previous problem is solved by imposing the initial conditions on position O init l and yaw orientation ψ init of the foot. They are set at t s = 0 to compute the first reference contact position Q. Initial roll and pitch orientation could be discontinuous to fulfill the desired ZMP position. Thus initial roll and pitch orientation of the foot are set as those obtained from the desired ZMP position. The Newton step (33) at the first time is changed accordingly to the tangential reference contact positions:

Q t = tO init l + tR init (θ, φ, ψ init ) l P free α (35)

Multi-objective quadratic program controller

The reference trajectories for the COM, the ankle position and orientation are obtained offline. Those references are online tracked as good as possible by a QP that takes also into account the robot constraints. The obtained motions are then compatible with the COM dynamics and the model of the flexibility. A task-space QP formulation with a weighted hierarchy [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF] is used to generate the whole body motion. The optimization variables are x = qT λ T T where q is the joint acceleration vector and λ is the force intensity along with the linearized friction cone.

The QP is formulated as follows:

min.

x 1 2

x T M x + c T x = min.

x

N i=1 α i E i (x) 2 +α λ λ 2
where N is the number of tasks put in the cost part, E i (x) is the task errors, α i is the task weight and α λ λ 2 is a damping term to ensure positive definite Hessian matrix.

From [START_REF] Vaillant | Multi-contact vertical ladder climbing with an HRP-2 humanoid[END_REF], the tasks put as constraints are:

1. Joint torque limits 2. Joint position/velocity limits 3. No-sliding contacts: zero acceleration for the contact body i 4. Collision avoidance More specifically, the tasks we used are:

1. Ankle: desired ankle trajectories given by the deformation estimator 2. COM: tracking of the desired COM position, velocity and acceleration given by the WPG 3. Torso: fixed torso absolute orientation 4. Posture: fixed posture for the whole upper body (torso, head, and arms)

6. Results

WPG optimization results

The Fig. 7a shows part of the obtained ZMP and COM reference trajectories with fixed ankle position for different values of λ:

• λ = 0: minimization of ankle torque only • λ = 0.5: minimization with equal weights on COM force and on ankle torque

• λ = 1:

minimization of COM force only

For λ = 0 we obtain ZMP and COM trajectories similar to [22][8] with large lateral variations of the COM. For λ = 1 trajectories are similar to [7][23] with small lateral variations of the COM and with a ZMP close to the feet edge. The mid-compromise solution λ = 0.5 gives limited lateral variations of the COM, the ZMP trajectory is smoother like human ZMP trajectories [START_REF] Winter | Human balance and posture control during standing and walking[END_REF]. All solutions result in walk with right and left feet as close as possible to minimize lateral variations of COM (and hence COM force) and variation of ZMP in DSP. The Fig. 7b shows also ZMP 1 and ZMP 2 trajectories during the DSP. ZMP 1 and ZMP 2 are local ZMPs in DSP. When we have ZMP 1 under one foot, we have a ZMP 2 under the opposite one. In Fig. 7b several foot step are drawn and each foot step has a ZMP 1 and ZMP 2 but they don't exist during the same phase.

We chose µ equal to 15 which experimentally gave us the best compromise to have ZMP going from heel to toe under each foot.

HRP-4 walking with compliant sole using the new controller

Here, we show the results of walking experiments with the HRP-4's new feet illustrated in Fig. 8. We removed the original shock absorbing springs located in the ankle (hence the ankle leg link is rigid) and added a flexible sole to absorb the landing impact shock and eventually cast terrain uncertainties. To do this experiments we used the WPG and deformation estimator explained before. The sole material is a flexible foam. Coupling compression test and FEM simulation, we obtained the Young's modulus E = 0.32MPa and Poisson's ratio ν = 0.31 of the sole (which is softer than flexible soles used in [START_REF] Pajon | A humanoid walking pattern generator for sole design optimization[END_REF] where Young's modulus were 5MPa, so that an estimator was not necessary).

Experiments consisted of HRP-4's straightforward walking for 10 footsteps (distance of 0.5 m) with average speeds of 3.5 cm/s and 4.2 cm/s. To validate our control framework in Fig. 2, we used ZMP and COM trajectories obtained using the WPG described in section 2 with λ = 0.8 and µ = 15 in (4). These trajectories are shown in Fig. 9a. Fig. 9b shows the ankle trajectories when the foot is not in contact with the floor. The video 1 and Fig. 10 clearly show the improvements given by the control scheme in Fig. 2. Figs. 10a, 10b and10c show the COM acceleration given by the robot Inertia 1 https://youtu.be/XByYfO3vcuc

Measurement Unit (IMU) for the experiment with the flexible foam. When we did not use the deformation estimator the robot fell down (the problem of the COM acceleration tracking is clearly identifiable around 4 s). When we used the deformation estimator the COM acceleration along the y-axis is close to the reference acceleration (see Fig. 10b). On the contrary, the COM acceleration along x-axis and z-axis are quite different from the reference acceleration (see Figs. 10a and10c). These tracking errors arise from the use of the HRP-4's built-in stabilizer: a black box that adjust the desired robot state from the ones we compute and tuned for the original feet design. Hence, it does not take into account the new sole flexibilities. We also believe that few tracking errors result from some conflicting tasks and constraints of the multi-objective QP controller (see section 5).

We looked also at the ankle force to validate our control framework in Fig. 2. As we can see from Figs. 10e and10f, that represent respectively the norm of horizontal plane ankle forces and the ankle force along z-axes, our controller estimates well the forces. As we can see in Fig. 10d, we also obtained ankle positions close to reference, which shows that the stabilizer does not need to change a lot the relative feet and waist positions to obtain stability, and this is thanks to the estimator.

The most important estimation of our new controller to enhance ZMP stabilization is given on foot orientations in Figs. 10g, 10h and 10i (in particular along x-and y-axes).

We can therefore state that our novel WPG and deformation estimator are a good model of the robot with flexible foam soles and enhances the native stabilizer to keep the desired robot attitude during the walking task.

Optimized HRP-4 walking with compliant soles for different ZMP trajectories

To find the best weight values in (4), we performed experiments on the HRP-4 with different weights using the new control framework in Fig. 2. Experiments consisted of humanoid walking with four different values of λ = {0.2, 0.4, 0.6, 0.8} in (4), robot performed a straightforward walking for 10 footsteps (distance of 0.5 m) with average speed of 3.5 cm/s.

To choose experimentally the best weight λ of the cost function (4), we analyzed the ankle torques (Fig. 10j) and COM accelerations (Fig. 10k). The theoretical values are obtained from the WPG without taking into account the flexibility.

The Fig. 10j shows the right foot ankle torques for different values of the WPG (results are similar for the left foot). Theoretical ankle torques increase with λ since the weight given to the criterion dependent on the ankle torque decreases. Analyzing the Fig. 10j, we can make the following observations:

• For λ = 0.2, the mean torque is bigger than the theoretical one. In this case, the ZMP trajectory of each foot is shorter than for λ > 0.2 and it is close to the ankle position (see Fig. 7a). This position of the ZMP must be more difficult to stabilize because it is the position of minimum sole flexibility when all the foam is equally (fully) compressed.

• From λ = 0.4 to λ = 0.6, mean torques are close to theoretical ones.

• For λ = 0.8, mean torques are lower than theoretical one. In this case, the ZMP trajectory of each foot is closer to the edge of the feet, which makes it difficult to move it further to the edge to stabilize the robot. Hence this walk is not as stable as for λ = 0.4 and λ = 0.6. Maximum ankle torques are then larger, but we have no explanation why the mean torque is lower than the theoretical one.

• Maximum ankle torques have a similar evolution to the theoretical mean ankle torques.

Fig. 10k shows the COM accelerations (which is also the image of COM force). In (4), λ mainly affect the COM acceleration in the lateral direction. Theoretical COM acceleration decreases with λ since the weight given to the minimization of the criterion dependents on the COM force increase. Analyzing the Fig. 10k, we can make the following observations:

• From λ = 0.2 to λ = 0.6, the evolution of COM acceleration is similar to the theoretical one, albeit with some variations due to the HRP-4 built-in stabilizer

• For λ = 0.8, the COM acceleration is not decreasing for the same reason as before; it is a case more difficult to stabilize.

Analyzing these results, we noticed that for λ = 0.4 and λ = 0.6 we obtained results closer to the theoretical expected ones as compared to λ = 0.2 and λ = 0.8, which might be due to problems of stability for extreme λ values.

Conclusion and future work

We investigated adding only flexible soles to a humanoid robot as compliant elements. Using a new control approach, we validate our approach on the HRP-4 robot. Experiments showed how our new WPG and deformation estimator enhances the ZMP stabilization during humanoid walking with flexible/soft soles. In the paper [START_REF] De Magistris | Design of optimized soft soles for humanoid robots[END_REF], we illustrate a new method to optimize the sole shape for improving stability and walking performances. In future work, we will use the controller illustrated in this paper to walk with the optimized sole.

The WPG minimizes the energy consumption and finds smooth heel-toe human-like ZMP force and trajectory under each foot.

We modeled the contact between the sole and the ground using a FEM model of the sole and a contact model accounting for both Signorini's and Coulomb's laws. These models are taken into account in the deformation estimator.

In near future work, we will develop an optimization that takes into account the WPG and the sole deformation at the same time. As a result, this will allow to obtain a more precise and optimized ankle movements. We also plan to design a control of COM with the sole flexibility model to better compensate for perturbations or uncertainty in the model.

Acknowledgments

This research is supported in part from the EU 7th Framework Program (FP7/2007-2013) under grant agreement No 611909 KoroiBot project www.koroibot.eu.

AppendixA. System to find COM trajectories Equation ( 9) has 2m unknowns with m = 3n + 4 phases. These unknowns satisfy the following boundary conditions for the COM position and velocity:

1. Initial x (1) (T 0 ) = V (1) + A (1) 0 (A.1) ẋ(1) (T 0 ) = W (1) + A (1) 1 (A.2)
2. Relationship between two successive sequences

V W (j) + 5 i=0 A (j) i (∆T j ) i = V (j+1) + A (j+1) 0 (A.3) V W ω (j) + 5 i=1 iA (j) i (∆T j ) i-1 = W (j+1) ω j + A (j+1) 1 (A.4) where V W (j) = V (j) C j + W (j) S j V W ω (j) = V (j) ω j S j + W (j) ω j C j C j = cosh(ω j ∆T j ), S j = sinh(ω j ∆T j ) 3. Final x (m) (T m ) = V W (m) + 5 i=0 A (m) i (∆T m ) i (A.5) dx (m) dt (T m ) = V W ω (m) + 5 i=1 iA (m) i (∆T m ) i-1 (A.6)
where ∆T j = t j -t j-1 .

From the boundary conditions (A.1)-(A.5), the total conditions are 2m + 2. Removing COM velocity conditions on initial and final phases (they are solved at the pattern optimization level), 2m conditions remain. The unknowns can be calculated then by the following system:

G • y = N • x + H • l (A.7)
where

y = V (1) W (1) • • • V (j) W (j) • • • V (m) W (m) T x = x (1) COM (t 0 ) ẋ(1) COM (t 0 ) x (m+1) COM (t m ) ẋ(m+1) COM (t m ) T l = A (1) 0 • • • A (1) 5 • • • A (m) 0 • • • A (m) 5 T G h,q =           
1 0 for (h, q) = (1, 1) C m S m for (h, q) = (m + 2, m) G 1,h for q = h -1 G 2,h for q = h 0 otherwise

N h,1 =
   1 0 0 0 for h = 1 0 0 1 0 for h = m + 2 0 0 0 0 otherwise

H h,q =           
-1 0 0 0 0 0 for (h, q) = (1, 1) -(∆T m ) 0 • • • (∆T m ) 5 for (h, q) = (m + 2, m) H 1,h for q = h -1 H 2,h for q = h 0 otherwise with h = 1, . . . , m + 2, q = 1, . . . , m, x

COM (T 1 ) the initial COM position, ẋ(1) COM (T 1 ) the initial COM velocity, x

= C j S h ω h S h ω h C h , G 2,h = -1 0 0 -ω h , H 1,h = - (∆T m ) 0 (∆T m ) 1 • • • (∆T m ) 5 0 (∆T m ) 0 • • • (∆T m ) 4 ,
H 2,h = 1 0 0 0 0 0 0 1 0 0 0 0 .

AppendixB. System to find the distribution of force under the feet in DSP

Based on the simplified robot model used in [START_REF] Morisawa | A Biped Pattern Generation Allowing Immediate Modification of Foot Placement in Real-time[END_REF], the relationship between F z ZMP1 and F z ZMP2 is: To define the function k during start sequence, we took into consideration experimental studies on human walking [START_REF] Winter | Human balance and posture control during standing and walking[END_REF]. During start sequence, humans move their COM laterally to be under the stance limb (we noted this foot '2'). The starting sequence begins with an equal load distribution in each limb under each foot. Then, the lateral movement is induced by a momentary load of the swing limb (we noted this foot '1'). At the final part of the walking phases, the stance limb is fully loaded and the swing limb is unloaded to begin the first SSP. Using these considerations, we chose the function k equal to 0.5 on C 1 , 0.25 on C 1 and 1 on C 1 .

The human behavior during stop sequence is symmetric to start sequence [START_REF] Winter | Human balance and posture control during standing and walking[END_REF]. During the stop sequence, humans move laterally their COM and decelerate their movement. This lateral movement begins with a full load of the stance limb (foot 1) from the end of the SSP. Then, the deceleration is induced by a momentary load of the swing limb (foot 2). The stopping sequence ends with the same load distribution on each limb (under each foot in our case) [START_REF] Winter | Human balance and posture control during standing and walking[END_REF]. For this reason, we chose the function k equal to 0 on B n+2 , 0.75 on B n+2 and 0.5 on B n+2 .

Figure 1 :

 1 Figure 1: (a): Rectangular parallelepiped soles mounted on HRP-4's feet; (b): meshed sole with 1494 tetrahedron elements

Figure 2 :

 2 Figure 2: Control scheme. The ankle reference trajectories change according to sole deformations.

Figure 3 :

 3 Figure 3: Simplified robot model in sagittal plane at left and carttable model at right

Figure 4 :

 4 Figure 4: Walking sequences with via-points (o) and ZMP trajectories during SSP (blue) and DSP (red)

  Initial and final position, speed and acceleration of the ZMP • SSP and DSP duration • COM height • Foot step directions. It can be changed to define the rotation during walking • Foot step positions. It is optional and define a specific walking path

Figure 5 :

 5 Figure 5: Contact between the sole and the ground

Figure 6 :

 6 Figure 6: Framework for contact handling to obtain F d ZMP and P d ZMP . ts -1 is the previous time step and Ψ = F ZMP -F d ZMP , P ZMP -P d ZMP , Γ P ZMP ,n T

Figure 7 :

 7 Figure 7: ZMP and COM trajectories with fixed ankle position

Figure 8 :

 8 Figure 8: (a): new HRP-4's feet; (b): photo of HRP-4's feet with flexible soles

Figure 9 :

 9 Figure 9: (a): ZMP trajectory under the left foot and COM trajectory during the 6 th foot step (SSP+DSP); (b): ankle trajectories when the foot is not in contact with the floor

8 t 8 t

 88 Figure 10: (a,b,c) COM acceleration compared to the reference acceleration along x/y/z axes with and without the deformation estimator; (d) Right ankle position along x/y/z axes; (e) Norm of horizontal plane ankle forces and (f) ankle force along z axes with and without the deformation estimator; (g,h,i) Ankle orientation along x/y/z axes with and without the deformation estimator; (j) Ankle torque for different cost functions; (k) COM acceleration for different cost functions.

  (T m+1 ) the final COM position and ẋ(m+1) COM (T m+1 ) the final COM velocity. G 1,h

F 3 ) 4 ) 5 )

 345 force F z ZMP2 in (B.1) is defined by:F z(j) ZMP2 = k (j) • F z(j) ZMP = k (j) • M g (B.2) k in (B.2) is a 5 th order polynomial representing the distribution of ZMP force under each foot. From (12), (B.1) and (B.2), we obtain:The boundary conditions that must be satisfied at the start of the DSP during a walking cycle are respectively a C 2 contact force and ZMP (due to the soft sole), an arbitrary new ZMP x (j) ZMP2 inside the new foot position and bounded ẋTo satisfy (11) and (B.1) and their first and second time derivatives, (B.4) gives at the start of the DSP:In the same way we obtain at the end of the DSP: .5) and (B.6) are the initial and final conditions to interpolate k in (B.3). k is a C 2 function which varies from 0 to 1.


Keywords: Bacteria, Biomolecules, Bioprocess, Extraction

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

GLYCEROL BIOCONVERSION FOR 3-HYDROXYPROPIONIC ACID PRODUCTION THROUGH AN INTEGRATED PROCESS INCLUDING IN SITU PRODUCT RECOVERY

Submit your abstract below (400 words): Within the framework of the development of the bioeconomy, there is an increasing drive towards the production of chemicals from renewable biomass resources. In this respect, interest in the sustainable production of biobased building blocks, such as the bifunctional carboxylic acid 3-hydroxypropionic acid (3-HP), is growing. The development of a biotechnological process to convert glycerol (by-product of biodiesel industries) into the platform chemical 3-HP is a key issue. Among its various applications, the 3-HP can be used for the synthesis of biobased polymers and acrylic acid 1 . Lactobacillus reuteri is known as a natural producer of 3-HP from glycerol although at low yield and productivity, due to the inhibitory effects of the product and of its metabolic intermediate 3-hydroxypropionaldehyde, as well as the formation of 1,3-propanediol as co-product [START_REF] Dishisha | Flux analysis of the Lactobacillus reuteri propanediol-utilization pathway for production of 3-hydroxypropionaldehyde, 3-hydroxypropionic acid and 1,3 propanediol from glycerol[END_REF] . The first aim was to improve the performance of 3-HP production by regulating operating parameters as pH (free or controlled at 6, 5, 4 or 3), stirring (100 or 250 rpm) and bacterial concentration. The pH control allowed maintaining the cell physiological state compared to a free-pH bioconversion as evaluated by flow cytometry. In fact, the L. reuteri viability (esterase activity assimilated to enzymatic activity) was assessed by cell staining with the fluorescent probe cFDA (carboxyfluorescein diacetate) and the bacterial membrane integrity was checked using PI (propidium iodide). No 3-HP production was observed at pH 3. Between pH 6 and 4, the production remained low probably due to inhibition phenomena, with only slight differences between the three conditions. We proposed to perform a controlled glycerol fedbatch to avoid 3-HPA accumulation. Moreover, original results about the 3-HP inhibition mechanisms were obtained thanks to flow cytometry analyses and Fourier-Transform InfraRed (FTIR) spectroscopy, the latter allowing to study the cell biochemical composition. For the first time, we demonstrated that 3-HP induces damages on the secondary structure of cell proteins (among which enzymes) and alterations of nucleic acids. In situ product recovery (ISPR) was also performed to extract 3-HP along its production. The ISPR approach has been successfully applied to the production of some solvents, organic acids, aromas and fine chemicals [START_REF] John M Woodley | Future directions for in-situ product removal (ISPR)[END_REF] . We used a membrane-assisted reactive extraction with long-chain tertiary and quaternary amines and decanol as diluent, a promising technique that drew a lot of attention for the extraction of hydroxypropionic acids 4 . A challenging bottleneck that has to be addressed deals with this process biocompatibility, studied by flow cytometry. The quaternary amines were shown to induce the highest toxicity compared to decanol with or without tertiary amines. Our work thus underlines the interest of a smart coordination of the upstream reaction and downstream technologies.

  Burgé, Claire Saulou-Berion, Marwen Moussa, Florent Allais, Henry-Eric Spinnler, Violaine Athès-Dutour To cite this version: Grégoire Burgé, Claire Saulou-Berion, Marwen Moussa, Florent Allais, Henry-Eric Spinnler, et al.. GLYCEROL BIOCONVERSION FOR 3-HYDROXYPROPIONIC ACID PRODUCTION THROUGH AN INTEGRATED PROCESS INCLUDING IN SITU PRODUCT RECOVERY: Topic : / Biochemical Engineering / ECAB3 / Bioproducts (or bio-based products). Chemical Engineering and Biochemical Engineering for a new sustainable process industry in Europe -ECAB3, Sep 2015, Nice, France. pp.74-75. hal-01564472 Abstract Submission Submission-381 Topic : / Biochemical Engineering / ECAB3 / Bioproducts (or bio-based products) GLYCEROL BIOCONVERSION FOR 3-HYDROXYPROPIONIC ACID PRODUCTION THROUGH AN INTEGRATED PROCESS INCLUDING IN SITU PRODUCT RECOVERY Grégoire Burgé 1, 2 , Claire Saulou-Bérion * 1 , Marwen Moussa 1 , Florent Allais 2 , Henry-Eric Spinnler 1 , Violaine Athès-Dutour 1 1 UMR 782 Génie et Microbiologie des Procédés Alimentaires, AgroParisTech -Institut National de la Recherche Agronomique (INRA), Thiverval-Grignon, 2 Chaire AgroBiotechnologies Industrielles (ABI), AgroParisTech, Reims, France Authors of the abstract come from :: University or Research Institution

  Type of presentation :: No preference You can add figures: Highlight 1: Bioprocess control to increase the 3-hydroxypropionic acid (3-HP) production. Highlight 2: In situ extraction to avoid product inhibition of bacteria. Highlight 3: Physiological state of Lactobacillus reuteri subjected to 3-HP and to extraction process. Reference 1 :: Choi s. Song cw. Shin jh. Lee sy. (2015). Biorefineries for the production of top building block chemicals and their derivatives. Metabolic engineering, 28:223-239


Grégoire Burgé

Florian Chemarin

Marwen Moussa

Claire Saulou-Berion

Florent Allais

Henry-Eric Spinnler

Violaine Athes-Dutour

BIOBASED 3-HYDROXYPROPIONIC ACID PRODUCTION AND EXTRACTION STARTING FROM GLYCEROL: TOWARDS A SUSTAINABLE INTEGRATED PROCESS

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

ABSTRACT

In the actual environmental and economic context, there is an increased interest in the microbial production at a large industrial scale of the bifunctional weak carboxylic acid, the 3-hydroxypropionic acid (3-HP). Among its various applications, this compound is primarily sought for its direct use as building block mainly for the synthesis of bio-based polymers. Moreover, 3-HP has enormous potential for the production of commercially valuable chemicals. Although 3-HP is currently of limited usage due to its exclusive synthesis via chemical pathways [START_REF] Ferreira | Ten-Year Forecast of Disruptive Technologies in Industrial Biotechnology to[END_REF], its market is expected to significantly grow up to a volume of 20,000 tons per year in 2015 according to a report of the International Energy Agency IEA-Bioenergy (2012) with a cost of 1,100 $ per ton [START_REF] Paster | DOE Industrial Bioproducts: Today and Tomorrow[END_REF]. Moreover, in recent years, the tremendous growth of biodiesel manufacturing industries resulted in a large production of inexpensive and abundant available glycerol as by-product that could be used as substrate. Until now, only lactic acid bacteria from the Lactobacillus genus have been shown to produce 3-HP from glycerol, although at low productivity [START_REF] Talarico | Production and isolation of reuterin, a growth inhibitor produced by Lactobacillus reuteri[END_REF]. In the present work, the bioconversion of glycerol into 3-HP by Lactobacillus reuteri was studied. Product and its metabolic intermediate 3-hydroxypropionaldehyde (3-HPA) are suspected to exhibit inhibitory or toxic effects on the producing microorganisms, thus leading to low productivity and dilute product stream. This entails high costs for isolation and purification of the product. A potential strategy to relieve the inhibitory stress, to increase the performance of the microbial cells and to recover the molecule of interest is to implement an ?In Situ Product Recovery? (ISPR), by which upstream bioconversion is combined to downstream extraction and recovery. A promising technique that drew a lot of attention lately for the extraction of hydroxypropionic acids is the reactive extraction in an organic phase containing an extractant [START_REF] Kyuchoukov | Lactic acid extraction by means of long chain tertiary amines: a comparative theoretical and experimental study[END_REF]. In the present work, the in situ 3-HP recovery was studied using the long chain tertiary amine trioctylamine (TOA) and the quaternary ammonium salt tri-n-octylmethylammonium chloride (Aliquat 336) as extractants, and n-decanol as diluent. Optimisation of the 3-HP reactive extraction assisted by hollow fiber-membrane contactor was displayed. In addition, the 3-HP in situ product recovery was studied and its performances were characterized. These results thus demonstrate the high potential of this ISPR technique applied for the biotechnological production of 3-HP.

  -Polymers and materials BIOBASED 3-HYDROXYPROPIONIC ACID PRODUCTION AND EXTRACTION STARTING FROM GLYCEROL: TOWARDS A SUSTAINABLE INTEGRATED PROCESS Grégoire Burgé / Chaire ABI, 247 rue Paul Vaillant Couturier, Reims Marwen Moussa / INRA, 1 avenue Lucien Brétignières, Thiverval-Grignon Claire Saulou / INRA, 1 avenue Lucien Brétignières, Thiverval-Grignon Florent Allais / Chaire ABI, 247 rue Paul Vaillant Couturier, Reims Henry-Éric Spinnler / INRA, 1 avenue Lucien Brétignières, Thiverval-Grignon Violaine Athes / INRA, 1 avenue Lucien Brétignières, Thiverval-Grignon