Embodied edutainment experience in a museum: discovering glassblowing gestures "Savoir-verre": an interactive installation for informal learning deployed at

Keywords: Interactive installation, Embodied edutainment in museum, Real-time feedback Motion capture, learning by doing, traditional crafts, visual feedback, auditory feedback, real-time interaction

Current technological progress has provided innovative methods and tools for learning and discovering various aspects of manual professions. To valorize their artifacts and mostly the craft and know-how behind them, museums are more and more attracted by these innovative interactive technologies. In this paper, we present an interactive installation for embodied edutainment experience developed in the framework of the exhibition "savoir-verre" implemented at "Musée des Arts et Métiers" in Paris. This installation had a double goal: to augment and improve the entire museum visiting experience (entertainment) and at the same time to go deeper into craft discovery, to learn the basics of the expert postures/gestures (education). In front of the camera, that captured their motions, visitors were invited to imitate 3 gestures of an expert glassblower and received real-time multimodal (visual and auditory) feedback, guiding the execution of the gestures. The quantitative analysis of user's motion data captured during the use of the installation highlights its positive effect on users' engagement, on their spatial and temporal performance, confirming the initial hypothesis that the multimodal real-time feedback can contribute to a better understanding and assimilation of expert gestures in glass blowing.

Introduction

In most cases, museums expose artifacts and objects and for a long time, the "don't touch" spirit dominated the visiting experience. Current technological progress made interactive technologies based on motion capture and artificial intelligence accessible to large public and started the transformation of a visit from a passive to an active process. Museums adopted these technologies and exploited their potential to attract the public and improve the visiting experience. As explained in [START_REF] Hornecker | Getting a grip on tangible interaction[END_REF] interactive installations are used with various objectives: first of all to facilitate immersion, stimulate curiosity, attract attention, and add an entertaining dimension by generating excitement. A systematic literature review [START_REF] Komianos | Immersive applications in museums: An analysis of the use of XR Technologies and the provided functionality based on Systematic Literature Review[END_REF] revealed that AR, VR, XR technologies are very promising and are preferred for on-site use. In [START_REF] Trajkova | Move your body: Engaging museum visitors with human-data interaction[END_REF] the "Move Your Body" installation improved the engagement of Science Museum visitors since it enabled people to explore large sets of data using interactive visualizations that users control with their own gestures and body movements. Embodied exploration has been also used to facilitate understanding and learning, beyond pure entertainment. In the Spanish Civil War museum [START_REF] Schaper | Learning about the past through situatedness, embodied exploration and digital augmentation of Cultural Heritage Sites[END_REF] researchers used the physical world to interact with digital content by using AR projections and embodied enactment. Also, embodied interactive technologies are appropriate to inform and sensitize the public not only about the material dimension of the object exposed but also the intangible heritage beyond it, the know-how necessary for its creation [START_REF] Alivizatou-Barakou | Intangible Cultural Heritage and new technologies: Challenges and opportunities for cultural preservation and development[END_REF]. For example, a mixed reality application has been developed to illustrate the making process of Cantonese porcelain while the user was scanning the object with a tablet/smartphone [START_REF] Tan | Case study: Creating embodied interaction with learning intangible cultural heritage through webar[END_REF]. The qualitative analysis of this case study revealed that the embodied interaction here enables a better perception of pedagogical contents from multiple sources (video, audio, etc.) Another recent work, done in the framework of the EU-funded project MINGEI, underlines the benefits of immersive technologies in discovering a traditional craft by combining 3D projections and animation of virtual humans [START_REF] Laure | Mixed-reality demonstration and training of glassblowing[END_REF].

Developing the interactive installation for

informal learning of glass-blowing gestures

Technological and museographical set-up

The exhibition "Savoir-verre" organized from 10/2021 to 04/2022 at "Musée des Arts et Métiers" in Paris aimed to valorise the tangible and intangible aspects of a glass masterpiece: a carafe created by a brilliant glassmaker G. Bontemps [START_REF] Laure | Mixed-reality demonstration and training of glassblowing[END_REF]. To permit the visitor to physically play the role of a learner glass-blower a space was dedicated where a real glass maker workbench and blowing pipe were installed, various parts of the carafe were presented in dedicated showcases, and 3D projections were made on a video wall to augment the immersion effect [START_REF] Laure | Mixed-reality demonstration and training of glassblowing[END_REF]. An RGB-D camera was installed on the upper part of the screen. The visitor was invited by a visual message to sit on the bench, take the pipe and activate the installation by raising the right hand. Computers and speakers were hidden behind the showcases. Videos of the expert Jean-Pierre Mateus from CERFAV, performing the 3 gestures (G1,G2,G3) for the creation of the carafe were then projected on a wall (fig. 1, fig. 2) and right after the user was invited to start imitating the gestures. In G1 the master turns the pipe, in G2 he uses pliers to form the neck and finally in G3 he stands up to blow the glass sphere. Real-time visual and auditory feedback artificially created and activated by the output of the user's kinematic performance evaluation module was provided to the user. The goal was to activate the learning-by-doing and imitation principles where the learner can continuously follow the gesture's example, try to reproduce it, and update his/her sensorimotor loop. A video illustrating the installation can be found on youtube 1 .

User's spatial and temporal evaluation based on motion capture and Gesture Operational Modelling (GOM)

1 https://www.youtube.com/watch?v=_7GwbvxjHrc

To evaluate user's spatial and temporal performance, kinematic deviation of user's joints (X and Y) from expert ones was calculated. For this, expert glassblower motions were first captured with various technologies such as wearable IMUs and cameras leading to the creation of a dataset [START_REF] Olivas-Padilla | Motion capture benchmark of real industrial tasks and traditional crafts for Human Movement Analysis[END_REF]. Then expert skeleton was extracted from the videos by using the OpenPose framework [START_REF] Cao | Realtime multiperson 2D pose estimation using part affinity fields[END_REF] which detects body key points (or joints) on the RGB image and extracts a skeletal model together with the 2D positions of each body joint which is then normalized by using the neck as reference point. However, in the framework of an informal learning and edutainment experience, a comparison between absolute values of expert and user wrists positions can't be used, because this would mean that a strict proximity of gestures should be achieved and this could be detrimental for the motivation of the user. The Gesture Operational Model (GOM) developed in [START_REF] Manitsaris | Human movement representation on multivariate time series for recognition of professional gestures and forecasting their trajectories[END_REF]. was used to mathematically represent interdependencies between joints and to estimate the upper and lower bounds, which correspond to a static zone of tolerance to spatial variance, within which user's gesture can be considered acceptable integrating thus a certain degree of flexibility. More details on the calculation of these bounds can be found in [START_REF] Glushkova | Interactive sensorimotor guidance for learning motor skills of a glass blower[END_REF] and [START_REF] Manitsaris | Human movement representation on multivariate time series for recognition of professional gestures and forecasting their trajectories[END_REF]. The 2-dimensional Euclidean distance between the user's and expert's wrists was computed and a real-time control done to verify if this distance was inside or outside the acceptable margin. In parallel a score was calculated where for each timeframe within the acceptable bounds a +1 point is added to reach 100 points. The experience passes automatically to the next gesture a) whenever the user reaches 100 points (in total approximately 20 sec of correct performance) or b) in case he/she doesn't reach the points within a time limit of 90 sec., defined empirically since each gesture is executed by the expert circularly until he/she judges that the goal has been achieved. It has been thus considered important to give the possibility to the user to perform multiple trials of the same gesture, continuously. A too short or long-time limit could be harmful to the engagement of the user as well as to the interpretation of the feedback.

Multimodal real-time feedback

To design the feedback the natural "in-person transmission" has been observed at CERFAV, France. First, "observationbased learning" is initiated where learner observed how does the expert perform the gestures. Then he/she is invited to reproduce and imitate them, activating thus the "learning-bydoing" principles, trying to embody the master's kinematic patterns, while receiving in parallel his/her comments and guidance. The edutainment experience was developed Inspired by this sensorimotor interaction.

Embodied edutainment experience in museum: discovering glass blowing gestures ICMI '23, October, 2023, Paris, France First of all a video of the expert performing the 3 gestures was projected to give the possibility to the user to attentively observe the gestures to imitate. Then the output of the evaluation module described in 2.2 provided input for the activation of 3 different feedbacks by using 2 modalities, 2 visual feedbacks and 1 auditory: 1) visual abstract concurrent (continuous) sensorimotor feedback where the user could see himself in a video stream, as a mirror effect, juxtaposed with the video of the expert. The skeleton extraction was visualized to help the user better understand, "feel" and embody his/her wrists positions in space. Circles were continuously projected corresponding to the expert position and their radius represented the tolerance defined. A Color coding was used to indicate whenever the user deviated from the acceptable margin: the circles became red when the user was out and green where he/she was inside. This type of visual annotation contributes to the knowledge of performance according the typology of feedbacks [START_REF] Dhara | Effectiveness of knowledge of result and knowledge of performance in the learning of a skilled motor activity by healthy young adults[END_REF] contrarily to another complementary technique where 2) visual feedback provides knowledge of result. Here it was given to the user through a horizontal green bar that visualized his/her progress score (the points collected up to 100). And 3) auditory feedback in the form of a positive alarm was implemented and activated whenever the deviation was within acceptable bounds and deactivated when it was not. Three metaphoric sounds (one for each gesture) inspired by the natural sounds of the glassblowing environment were mapped to each gesture.

Evaluation and results

User engagement

One of the goals of this installation was to positively affect the interest of visitors when discovering the glass-blowing craft. In total, the installation has been activated 1697 times. This number has been counted based on the triggering system deployed (mentioned in 2.1).

As presented in the table below, among these 1697 activations, the G1 has been performed 1099 times. The difference with the total number of activations could be due to various parameters such as the visitor not being willing to experiment either because he/she didn't have time to dedicate, or didn't want to touch the pipe for Covid-related reasons or other reasons. Among these 1099 trials, 821 users also performed the G2. The difference between the number of persons who didn't try at all (-598) and the ones who tried the G1 and then didn't continue to G2 has been considerably reduced (-278). The same tendency can be observed when the user passes from G2 to G3 since the number of G3 performances was reduced by only 87 trials. This means that the more the user advances in imitating the gestures the less he/she tends to give up the experience and the more he/she is motivated to reach the end. This can be also explained by the fact that the more the visitor interacts with the feedback the better he/she understands how it works and the more complete the feedback appropriation.

# of trials

Spatial performance

When looking at the spatial performance of users for G1, 564 trials out of 1099 (51%) have been successfully completed meaning that the error/deviation was acceptable (within the thresholds defined) within the time limit defined (90 sec). While continuing to G2 the users seem to better understand the general functioning of the interaction, they have a better ability to interpret the feedback since the percentage of successful trials in comparison to the G1 has augmented (589 out of 821 = 72%). The same phenomenon can be observed with the G3 where 73% of trials were successful. Progressively the users improve their spatial performance and reduce the deviation while they are advancing from G1 to G2 and G3.

It is also possible to observe that 423 trials out of 821 have been successfully completed for both G1 and G2, which corresponds to 52% of trials. This number represents the number of users who directly "caught" the idea of being guided by the feedback and correctly performed the first 2 gestures. For these users the interpretation of the feedback was intuitive and the guidance provided was beneficial for the performance. The percentage is slightly reduced when it comes to the total successful trials of the 3 gestures where 47% of trials were completed successfully. However, it means that almost half of the users managed to perform correctly the 3 gestures within the time limit defined. In the score histogram (Fig. 3) the distribution of scores from 0 to 100 is presented by groups of 20 points, per gesture. In general, more than 60 % of trials for all gestures were successful illustrating that the feedback is meaningful and helpful. The best score was achieved by more than 70% of trials for G2 and G3. The G1 still has a lower percentage confirming that the G1, since it comes first, corresponds to a phase where the user learns how to interact and interpret the feedback.

Temporal performance

As explained in Section 2, a maximum time duration has been defined when the experience passes automatically to the next gesture. In the histogram below we can see the time duration of trials where all 3 gestures were completed (345 trials). it presents the percentage of users performing the 3 gestures with different durations from the shortest (20 sec.) to the longest (90 sec.). The best temporal performances, are located at the beginning of the horizontal axis. Almost 70% of users finished the G2 and G3 in less than 40s, unlike the G1 that only 30% of the users finished in less than 40s. These results show that the users probably needed some time to get used to the interaction mechanism and once this was achieved the majority managed to reach an almost optimal time of performance. Even for the G3 where the users needed to stand up and change their posture completely, their performance was slightly deteriorated in comparison to G2.

Discussion

In this work an embodied interactive installation was developed and deployed in a museum, confirming the initial hypothesis that the multimodal feedback arouses curiosity and permits the visitor to put himself for some minutes in a place of a craft master, while trying to physically imitate his gestures realizing thus the complexity of the tasks. The more the users have been engaged in the process of trying to imitate the gestures, the more motivated they were to complete the experience and to go until the end while progressively improving their capacity to interpret the feedback and reaching an optimal kinematic performance.

Embodied interactive systems have the potential to support informal human learning, which merits additional research. Finding the best sensorimotor feedback format for engagement and learning is challenging because it also takes into account factors like cognitive load, motor skill memorization, user interpretation abilities, etc. However, even outside of cultural settings like museums, the field of crafts suffers from a severe lack of interactive pedagogical tools that allow users to actively demonstrate the gestures they are trying to learn while also having them evaluated simultaneously by technological tools.

Figure 1 :

 1 Figure 1: The glass bench, the showcases to present the carafe, and the camera used to track the user.

Figure 2 :

 2 Figure 2: Example of visual feedback tested in the lab and the final interface in the image below.

Figure 3 :

 3 Figure 3: Percentage of users that achieved from 0 to 100 points (per group of 20 points) per gesture (blue G1, orange G2, green G3)

Figure 4 :

 4 Figure 4: Proportion of users performing the 3 gestures with different durations from the shortest to the longest

Table 1 : activations and trials performed in the museum

 1

	% of activations and
	trials

ACKNOWLEDGEMENTS

This work was supported by the EU Horizon 2020 Innovation Program and Grant No. XXX, XXX project. The authors would like to express their gratitude to the glass-blower Jean-Pierre Mateus, from CERFAV for the motion capturing sessions. Embodied edutainment experience in museum: discovering glass blowing gestures ICMI '23, October, 2023, Paris, France


Pavel Král

Christophe Cerisara
email: cerisara@loria.fr

Automatic Dialogue Act Recognition with Syntactic Features

Keywords: Dialogue Act, Language Model, Sentence Structure, Speech Act, Speech Recognition, Syntax

This work studies the usefulness of syntactic information in the context of automatic dialogue act recognition in Czech. Several pieces of evidence are presented in this work that support our claim that syntax might bring valuable information for dialogue act recognition. In particular, a parallel is drawn with the related domain of automatic punctuation generation and a set of syntactic features derived from a deep parse tree is further proposed and successfully used in a Czech dialogue act recognition system based on Conditional Random Fields. We finally discuss the possible reasons why so few works have exploited this type of information before and propose future research directions to further progress in this area.

1 Introduction

Definition

Modelling and automatically identifying the structure of spontaneous dialogues is very important to better interpret and understand them. The precise modelling of dialogues is still an open issue, but several specific characteristics of dialogues have already been clearly identified. Dialogue Acts (DAs) are one of these characteristics.

Although the term "dialogue acts" that is commonly used nowadays has been defined by Austin in [START_REF] Austin | How to do Things with Words[END_REF], a number of other seminal works have proposed very similar notions, including speech acts proposed by John R. Searle in [START_REF] Searle | Speech Acts: An essay in the philosophy of language[END_REF], conversational game moves introduced by R. J. D. Power in [START_REF] Power | The organization of purposeful dialogues[END_REF], adjacency pairs proposed by Schegloff in [START_REF] Schegloff | Sequencing in conversational openings[END_REF][START_REF] Sacks | A simplest semantics for the organization of turn-taking in conversation[END_REF] or acts of communication in the plan-based approaches to understanding introduced by Litman et al. in [START_REF] Litman | Plan recognition and discourse analysis: an integrated approach for understanding dialogues[END_REF][START_REF] Kautz | A formal theory of plan recognition[END_REF][START_REF] Carberry | Plan Recognition in Natural Language Dialogue[END_REF]. The theory of the dialogue acts has been further developed by Hary Bunt in [START_REF] Bunt | Context and Dialogue Control[END_REF]. The dialogue acts represent the meaning of an utterance in the context of a dialogue, where the context is divided into several types, with both global and local views: linguistic, semantic, physical, social and cognitive. Bunt also developed a multidimensional taxonomy of the dialogue acts, while David R. Traum developed the notion of speech acts in [START_REF] Traum | Speech acts for dialogue agents[END_REF] with dialogue agents. A better overview of the notion of dialogue acts can be found in [START_REF] Stolcke | Dialog Act Modeling for Automatic Tagging and Recognition of Conversational Speech[END_REF].

In this work, the dialogue act is seen as a function of an utterance, or its part, in the dialogue. For example, the function of a question is to request some information, while an answer shall provide this information.

Table 1 illustrates the dialogue acts that may occur in a dialogue between the passenger (P) and the agent (A) in a ticket reservation task. The corresponding dialogue act labels are also shown. Each utterance is labelled with a unique dialogue act. This example is taken from our Czech corpus (see Section 5.1). Dialogue acts represent useful and relevant information for many applications, such as dialogue systems, machine translation, automatic speech recognition, topic tracking [START_REF] Garner | A Theory of Word Frequencies and its Application to Dialogue Move Recognition[END_REF] or talking head animation. For instance, in dialogue systems, dialogue acts might be used to recognize the intention of the user and thus differentiate situations where the user is requesting some information from situations where the user is simply giving some information or backchannels. In the former case, the system has to react, while in the latter case, a system reaction may be perceived as intrusive. In the machine translation domain, recognizing dialogue acts may bring relevant cues to choose between alternative translations, as the adequate syntactic structure may depend on the user intention. Automatic recognition of dialogue acts may also be used to improve the word recognition accuracy of automatic speech recognition systems, as proposed for instance in [START_REF] Wright | Automatic Utterance Type Detection Using Suprasegmental Features[END_REF], where a different language model is applied during recognition depending on the dialogue act. Finally, dialogue act recognition is a fundamental building block of any understanding system and typically completes semantic role labelling and semantic frame inference.

The usefulness of dialogue act recognition has thus been demonstrated in a number of large applicative systems, such as the VERBMOBIL [START_REF] Alexandersson | Insights into the Dialogue Processing of VERBMOBIL[END_REF], NE-SPOLE [START_REF] Lavie | The NESPOLE! System for multilingual speech communication over the Internet, Audio, Speech, and Language Processing[END_REF] and C-STAR [START_REF] Blanchon | Speech translation for French within the C-STAR II consortium and future perspectives[END_REF] machine translation and dialogue systems that rely on dialogue act classification.

Objectives

The main objective of this work is to propose and investigate the usefulness of syntactic features to improve dialogue act recognition in Czech. In previous works, we have first designed a baseline dialogue act recognition system for the Czech language that was based on generative models [START_REF] Král | Combination of Classifiers for Automatic Recognition of Dialog Acts[END_REF]. Although reasonably good results have been obtained, this approach was limited because it only exploits the local context around any given word of the utterance. We then proposed in [START_REF] Král | Automatic Dialog Acts Recognition based on Sentence Structure[END_REF] and [START_REF] Král | Lexical Structure for Dialogue Act Recognition[END_REF] several approaches to address this limitation and include global features in the model that represent the sentence structure. One of these approaches consists in modelling the word position in the sentence as a random variable and integrating this variable in the generative model. Intuitively, this information is important for dialogue act recognition, as for instance, the word "who" is often located at the beginning of sentences for questions and at other positions for declarative sentences. In the following, we propose a different approach to model such global information implicitly, via a conditional stochastic model. The second and most important contribution of this work concerns the design and exploitation of syntactic features for dialogue act recognition in Czech. As summarized in Section 2, only a few types of features are generally used in the literature to automatically recognize dialogue acts: lexical, Part-of-Speech (POS) tags, dialogue history and prosody. Furthermore, word sequences are most of the time modelled by statistical ngram models, which encode the relationship between words and dialogue acts only locally. While we have already shown the importance of global informa-tion such as word position in the utterance for dialogue act recognition, the current work goes beyond this type of information by investigating whether the conditional distribution of the target dialogue act depends on the syntactic structure of the utterance.

In the following section, we briefly review the state of the art about dialogue act recognition, with a focus on how syntactic information has already been considered for this task and for related tasks. In Section 3, we propose and describe new syntactic features. The proposed model is described in Section 4. The relative importance of each of these features is evaluated on a Czech dialogue corpus in Section 5. In the last section, we discuss these results and propose some future research directions.

Related Work

We will now briefly review the standard definitions of dialogue acts, the different types of models classically used for dialogue act recognition and the standard types of information used in such models. Then, we review and discuss the previous design of syntactic features for dialogue act recognition as well as in closely related domains.

Some generic sets of domain-independent dialogue acts have been proposed in the state-of-the-art and are now commonly used to create the baseline tag set for most types of applications. Hence, in [START_REF] Stolcke | Dialog Act Modeling for Automatic Tagging and Recognition of Conversational Speech[END_REF], 42 dialogue acts classes are defined for English, based on the Discourse Annotation and Markup System of Labelling (DAMSL) tag-set [START_REF] Allen | Draft of Damsl: Dialog Act Markup in Several Layers[END_REF]. The Switchboard-DAMSL tag-set [START_REF] Jurafsky | Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation (Coders Manual[END_REF] (SWBD-DAMSL) is an adaptation of DAMSL in the field of telephone conversations. The Meeting Recorder Dialogue Act (MRDA) tag-set [START_REF] Dhillon | Meeting Recorder Project: Dialog Act Labeling Guide[END_REF] is another very popular tag-set, which is based on the SWBD-DAMSL taxonomy. MRDA contains 11 general dialogue act labels and 39 specific labels. Finally, Jekat [START_REF] Jekat | Dialogue Acts in VERBMOBIL[END_REF] defines for German and Japanese 42 dialogue acts, with 18 dialogue acts at the illocutionary level, in the context of the VERBMOBIL corpus. The ISO standard 24617-2 for dialogue annotation has been published in 2012. DIT++1 is a recent implementation of this standard. Because of the limited size of the available corpus, as well as several other technical reasons, these tag sets are frequently reduced by merging several tags together, so that the number of final actual generic tags is often about 10. Part of such typical generic dialogue acts, also referred to as speech acts, include for instance [START_REF] Shriberg | Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?[END_REF] statements, questions, backchannels, commands, agreements, appreciations as well as a broad "miscellaneous" class. In addition to such generic tags, application-specific tags may be defined, such as "request booking" for a hotel booking application.

Manually annotating dialogue acts on every new corpus may be very costly and efforts have been put into developing semi-automatic methods for dialogue act tagging and discovery. Hence, the authors of [START_REF] Orkin | Semi-automated dialogue act classification for situated social agents in games[END_REF] propose a predictive paradigm where dialogue act models are first trained on a small-size corpus and used afterwards to predict future sentences or dialogue acts. In a related vein, unsupervised dialogue act tagging of unlabelled text has recently raised a lot of attention [START_REF] Joty | Unsupervised approaches for dialog act modeling of asynchronous conversations[END_REF][START_REF] Crook | Unsupervised classification of dialogue acts using a dirichlet process mixture model[END_REF], but we will limit ourselves in the following on supervised approaches.

The dialogue act recognition task is often considered jointly with the segmentation task. We assume in our work that sentence segmentation is known, because we rather prefer to concentrate on the challenge of designing relevant syntactic features for dialogue act recognition. Yet, many related works propose powerful solutions for the segmentation task as well. In particular, the work described in [START_REF] Petukhova | Incremental dialogue act understanding[END_REF] considers the input text as a stream of words and segments and tags it incrementally with a BayesNet model with lexical, prosodic, timing and dialogue act-history features. Zimmermann et al. successfully use in [START_REF] Zimmermann | Joint segmentation and Classification of Dialog Acts in Multiparty Meetings[END_REF] for joint DA segmentation and classification hidden-event language models and a maximum entropy classifier. They use word sequence and pause duration as features. The authors of [START_REF] Dielmann | Recognition of dialogue acts in multiparty meetings using a switching dbn[END_REF] exploit a Switching Dynamic Bayesian Network for segmentation, cascaded with a Conditional Random Field for dialogue act classification, while [START_REF] Quarteroni | Simultaneous dialog act segmentation and classification from human-human spoken conversations[END_REF] jointly segments and tags with a single model.

The dialogue act modelling schemes that are commonly used for dialogue act recognition are traditionally chosen from the same set of general machine learning methods used in most natural language processing tasks. These include Hidden Markov Models [START_REF] Stolcke | Dialog Act Modeling for Automatic Tagging and Recognition of Conversational Speech[END_REF], Bayesian Networks [START_REF] Keizer | Dialogue Act Recognition with Bayesian Networks for Dutch Dialogues[END_REF], Discriminative Dynamic Bayesian Networks [START_REF] Ji | Dialog Act Tagging Using Graphical Models[END_REF], BayesNet [START_REF] Petukhova | Incremental dialogue act understanding[END_REF], Memory-based [START_REF] Lendvai | Machine Learning for Shallow Interpretation of User Utterances in Spoken Dialogue Systems[END_REF] and Transformation-based Learning [START_REF] Samuel | Dialogue Act Tagging with Transformation-Based Learning[END_REF], Decision Trees [START_REF] Mast | Automatic Classification of Dialog Acts with Semantic Classification Trees and Polygrams[END_REF], Neural Networks [START_REF] Levin | Domain Specific Speech Acts for Spoken Language Translation[END_REF], but also more advanced approaches such as Boosting [START_REF] Tur | Model adaptation for dialogue act tagging[END_REF], Latent Semantic Analysis [START_REF] Serafin | LSA: Extending latent semantic analysis with features for dialogue act classification[END_REF], Hidden Backoff Models [START_REF] Bilmes | Backoff model training using partially observed data: Application to dialog act tagging[END_REF], Maximum Entropy Models [START_REF] Ang | Automatic dialog act segmentation and classification in multiparty meetings[END_REF], Conditional Random Fields [START_REF] Quarteroni | Simultaneous dialog act segmentation and classification from human-human spoken conversations[END_REF][START_REF] Dielmann | Recognition of dialogue acts in multiparty meetings using a switching dbn[END_REF] and Triangular-chain CRF [START_REF] Jeong | Triangular-chain conditional random fields[END_REF].

Regarding features, most dialogue act recognition systems exploit both prosodic and lexical features. The dialogue history is also often used as relevant information. Some cue words and phrases can also serve as explicit indicators of dialogue structure [START_REF] Webb | Cue-based dialog act classification[END_REF]. For example, 88.4% of the trigrams "<start> do you" occur in English in yes/no questions [START_REF] Jurafsky | Automatic Detection of Discourse Structure for Speech Recognition and Understanding[END_REF].

Prosody is an important source of information for dialogue act recognition [START_REF] Shriberg | Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?[END_REF]. For instance, prosodic models may help to capture the following typical features of some dialogue acts [START_REF] Kompe | Prosody in Speech Understanding Systems[END_REF]:

a falling intonation for most statements a rising F0 contour for some questions (particularly for declaratives and yes/no questions) -a continuation-rising F0 contour characterizes (prosodic) clause boundaries, which is different from the end of utterance

In [START_REF] Shriberg | Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?[END_REF], the duration, pause, fundamental frequency (F0), energy and speaking rate prosodic attributes are modelled by a CART-style decision trees classifier. In [START_REF] Mast | Dialog Act Classification with the Help of Prosody[END_REF], prosody is used to segment utterances. The duration, pause, F0-contour and energy features are used in [START_REF] Wright | Automatic Utterance Type Detection Using Suprasegmental Features[END_REF] and [START_REF] Wright | Using High Level Dialogue Information for Dialogue Act Recognition using Prosodic Features[END_REF]. In both [START_REF] Wright | Automatic Utterance Type Detection Using Suprasegmental Features[END_REF] and [START_REF] Wright | Using High Level Dialogue Information for Dialogue Act Recognition using Prosodic Features[END_REF], several features are computed based on these basic prosodic attributes, for example the max, min, mean and standard deviation of F0, the mean and standard deviation of the energy, the number of frames in utterance and the number of voiced frames. The features are computed on the whole sentence and also on the last 200 ms of each sentence. The authors conclude that the end of sentences carry the most important prosodic information for dialogue act recognition. Shriberg et al. show in [START_REF] Shriberg | Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?[END_REF] that it is better to use prosody for dialogue act recognition in three separate tasks, namely question detection, incomplete utterance detection and agreements detection, rather than for detecting all dialogue acts in one task.

Apart from prosodic and contextual lexical features, only a few works actually exploit syntactic relationships between words for dialogue act recognition. Some syntactic relations are captured by HMM word models, such as the widely-used n-grams [START_REF] Stolcke | Dialog Act Modeling for Automatic Tagging and Recognition of Conversational Speech[END_REF], but these approaches only capture local syntactic relations, while we consider next global syntactic trees. Most other works thus focus on morphosyntactic tags, as demonstrated for instance in [START_REF] Verbree | Dialog-act tagging using smart feature selection; results on multiple corpora[END_REF], where a smart compression technique for feature selection is introduced. The authors use a rich feature set with POS-tags included and obtain with a decision tree classifier an accuracy of 89.27%, 65.68% and 59.76% respectively on the ICSI, Switchboard and on a selection of the AMI corpus. But while POS-tags are indeed related to syntax, they do not encode actual syntactic relations.

A very few number of works have nevertheless proposed some specific structured syntactic features, such as for instance the subject of verb type [START_REF] Andernach | A machine learning approach to the classification of dialogue utterances[END_REF]. The authors of [START_REF] Serafin | LSA: Extending latent semantic analysis with features for dialogue act classification[END_REF][START_REF] Di Eugenio | Dialogue act classification, higher order dialogue structure, and instance-based learning[END_REF] exploit a few global syntactic features, in particular POStags and the MapTask SRule annotation that indicates the main structure of the utterance, i.e., Declarative, Imperative, Inverted or Wh-question, but without obtaining a clear gain from syntax in their context, hence suggesting that further investigation is needed. Indeed, syntax is a very rich source of information and the potential impact of syntactic information highly depends on the chosen integration approach and experimental setup. We thus propose in the next section other types of syntactic features and a different model and show that syntax might indeed prove useful for dialogue act recognition in the proposed context. But let us first support our hypothesis by briefly reviewing a few other papers that also support the use of syntax for both dialogue act recognition and closely related domains.

First, as already shown, word n-grams features, with n greater than 1, do implicitly encode local syntactic relations and are used successfully in most dialogue act recognition systems. But more importantly, a recent work [START_REF] Klüwer | Using syntactic and semantic based relations for dialogue act recognition[END_REF] concludes that both dialogue context and syntactic features dramatically improve dialogue act recognition, compared to words only, more precisely from an accuracy of 48.1% up to 61.9% when including context and 67.4% when further including syntactic features. They use in their experiments a Bayesian Network model and their syntactic features are the syntactic class of the predicate, the list of arguments and the presence of a negation. Although this work actually focuses on predicate-argument structures, while our main objective is rather to exploit the full syntactic tree without taking into account any semantic-level information for now, this work supports our claim that syntactic information may prove important for dialogue act recognition. In addition, Zhou et al. employ in [START_REF] Zhou | Dialog-act recognition using discourse and sentence structure information[END_REF] three levels of features: 1) word level (unigram, bigram and trigram), 2) syntax level (POS-tags and chunks recognized as Base Noun Phrase (BNP)) and 3) restraint information (word position, utterance length, etc.). Syntactic and semantic relations are acquired by information extraction methods. They obtain 88% of accuracy with a SVM classifier on a Chinese corpus and 65% on the SWBD corpus.

We further investigated closely related domains that have already explored this research track in more depth. This is for instance the case of automatic classification of rhetorical relations, as reviewed in [START_REF] Sporleder | Using automatically labelled examples to classify rhetorical relations: A critical assessment[END_REF]. Another very close task is punctuation recovery, which aims at generating punctuation marks in raw words sequences, as typically obtained from speech recognition systems. In particular, this implies to discriminate between questions (ending with a question mark), orders (ending with an exclamation points) and statements (ending with a period), which is a task that is obviously strongly correlated to dialogue act recognition. Interestingly enough, a richer set of syntactic features have been exploited in the punctuation recovery domain than in the dialogue act recognition area. Hence, the authors of [START_REF] Favre | Syntactically-informed models for comma prediction[END_REF] design several syntactic features derived from the phrase structure trees and show that these features significantly reduce the detection errors. This is in line with our own previous conclusions published in [START_REF] Cerisara | Commas recovery with syntactic features in French and in Czech[END_REF] regarding the use of syntactic features for punctuation recovery, where a large improvement in performances is obtained thanks to syntactic information derived from dependency trees. Similar gains are obtained on a Chinese punctuation task [START_REF] Guo | A linguistically inspired statistical model for Chinese punctuation generation[END_REF], where including rich syntactic features, such as the word grammatical function, its ancestors and children, its head, the yield of the constituent or subtree border indicators, improve the F-measure from 52.61% up to 74.04%.

Finally, we have shown that there is an increasing amount of work that successfully exploits structural syntactic dependencies both for dialogue act recognition and in related domains such as punctuation recovery. We further believe that parsing of natural language utterances will constitute a fundamental pre-processing step of most if not all subsequent NLP modules, although it has probably not been as widely used as POS tagging for instance because of its complexity and lack of robustness to ill-formed input. However, thanks to the current progress in Bayesian approaches and feature-rich log-linear models, we expect parsing to be more and more robust to automatic speech recognition errors in the near future. Other recent reviews of the literature about dialogue act recognition are realized in [START_REF] Geertzen | Dialog act recognition and prediction[END_REF] and [START_REF] Webb | Cue-based dialog act classification[END_REF].

Syntax for Automatic Dialogue Act Recognition

In our previous work [START_REF] Král | Automatic Dialog Acts Recognition based on Sentence Structure[END_REF], [START_REF] Král | Sentence Structure for Dialog Act recognition in Czech[END_REF] and [START_REF] Král | Lexical Structure for Dialogue Act Recognition[END_REF], we proposed to include in our DA recognition approach information related to the position of the words within the sentence. In this work, we propose a different approach that derives features from the sentence parse tree and includes these features as input to a conditional random field. The parse trees are defined within the dependency framework [START_REF] Hajičová | Dependency-Based Underlying-Structure Tagging of a Very Large Czech Corpus[END_REF] and are automatically computed on the input sentences. We evaluate this approach in two conditions, respectively when the input sentences are manually and automatically transcribed.

Features

We distinguish next two types of features, respectively the baseline and syntactic features. The baseline features are:

words inflected form lemmas part-of-speech tags pronoun or adverb at the beginning of the sentence verb at the beginning of the sentence The syntactic features rely on a dependency parse tree:

dependency label root position in the utterance unexpressed subjects basic composite pair (subject-verb inversion) All these features are described in details next.

Baseline Features

Words Inflected Form The word form is used as a baseline lexical feature in most modern lexicalized natural language processing approaches [START_REF] Stolcke | Dialog Act Modeling for Automatic Tagging and Recognition of Conversational Speech[END_REF][START_REF] Jurafsky | Automatic Detection of Discourse Structure for Speech Recognition and Understanding[END_REF][START_REF] Keizer | Dialogue Act Recognition with Bayesian Networks for Dutch Dialogues[END_REF][START_REF] Ji | Dialog Act Tagging Using Graphical Models[END_REF]. In our case, sentence segmentation is known but capitalization of the first word of the sentence is removed, which decreases the total number of features in our model without impacting accuracy, thanks to the insertion of a special "start-of-utterance" word. Although word bigrams or trigrams are commonly used in other systems, we only use word unigrams because of the limited size of the training corpus. We rather compensate for this lack of local structural information by investigating global syntactic dependencies. The word forms are obtained in our experiments using both manual and automatic transcriptions of speech audio files.

Lemmas We used the lemma structure from the Prague Dependency Treebank (PDT) 2.0 1 [START_REF] Hajič | The Prague Dependency Treebank: A Three-Level Annotation Scenario[END_REF] project, which is composed of two parts. The first part is a unique identifier of the lexical item. Usually it is the base form (e.g. infinitive for a verb) of the word, possibly followed by a digit to disambiguate different lemmas with the same base forms. The second optional part contains additional information about the lemma, such as semantic or derivational information. Lemmas may in some circumstances bring additional information, notably by removing irrelevant variability introduced by inflected forms. This may have some importance in particular for rare words that may occur with different inflected forms but still may have some impact on the dialogue act decision process. The lemmas are obtained automatically in our experiment with a lemmatizer.

Part-of-Speech (POS) tags The part-of-speech is a word linguistic category (or more precisely lexical item), which can be defined by the syntactic or morphological behaviour of the lexical item in question. There are ten POS categories defined in the PDT [START_REF] Hajič | The Prague Dependency Treebank: A Three-Level Annotation Scenario[END_REF] for the Czech language: nouns, adjectives, pronouns, numerals, verbs, adverbs, prepositions, conjunctions, particles and interjections. The part-of-speech tags are inferred automatically in our experiment with a POS tagger.

Pronoun or Adverb at the Beginning of the Sentence This boolean feature indicates whether the utterance starts with a pronoun or an adverb. It can be particularly useful for detecting Wh-questions, which usually start with a Pronoun (POS-tag "P") or an Adverb (POS-tag "D"), such as in: "Kdy přijdeš domů?" (When do you come home?).

Note that similar features that emphasize the importance of initial words in the sentence have already been proposed, for instance in [START_REF] Jurafsky | Speech and Language Processing: An Introduction to Natural Language Processing, Speech Recognition, and Computational Linguistics[END_REF][START_REF] Webb | Cue-based dialog act classification[END_REF][START_REF] Ang | Automatic dialog act segmentation and classification in multiparty meetings[END_REF].

Kdy přijdeš domů ?

When come home ?

Adv Adv

Fig. 1 Example of adverb as first word.

Verb at the Beginning of the Sentence This feature is also a boolean indicator of the presence of a verb as the first word of an utterance. It can be particularly useful for the detection of Commands and Yes-no questions, which usually start with a verb, such as in: "Jdi domů!" (Go home!) and "Půjdeš domů?" (Do you go home?).

Jdi domů ! ... Půjdeš domů ? Go home ! ...

Go home ?

Adv Adv

Fig. 2 Example of verb as first word.

1 http://ufal.mff.cuni.cz/pdt2.0/

Syntactic Features

All syntactic features are computed from the syntactic tree obtained after automatic parsing of the target sentence: a detailed description of our automatic parser is given in Section 5.2. We have chosen to represent the syntactic relations with dependencies, as it is commonly done nowadays for many natural language processing tasks. Furthermore, we have chosen the Prague Dependency Treebank to train our stochastic parser and our annotation formalism thus follows the one used in the PDT.

An example of such a dependency tree is shown in Figure 3, where the words represent the nodes of the tree and the arcs the dependencies between words. Dependencies are oriented, with each arrow pointing to the dependent word of the relation. Each dependency is further labelled with the name of a syntactic relation, such as Sb for subject, Pred for predicate, Atr for attribute, Obj for object, etc. Fig. 3 Example of an instance of a Czech statement dialogue act (Two points out of six are worthy of attention.) with its parse tree.

Dependency Label The first generic feature derived from the parse tree is the label of the dependency from the target word to its head. For example, the values taken by this feature in Figure 3 are, for each word: Atr, Sb, AuxP, Atr, Root, Pnom, Obj.

Root Position in the Utterance In theory, every utterance is parsed into a single dependency tree. The position of the root of this tree is likely to depend on the type of sentence and dialogue act. Hence, intuitively, the root tends to be positioned in the middle of declarative sentences, as in Figure 3, while it is more often located at the start of utterances for commands/orders, such as in: "Zavři dveře!"(Close the door!). This feature is the absolute position of the root, after normalization of the sentence length to 10 words. The normalization is realized with a standard binning technique, eventually filling empty internal bins with virtual non-root words for short sentences, so that the word at the middle of the sentence is in bin 5 and recursively in the left and right halves of the sentence.

Unexpressed Subject This feature is a boolean feature that is true if and only if a subject dependency exists for the first verb in the sentence. In- deed, verbs without subjects may intuitively occur more frequently in commands/orders than in declarative sentences, as illustrated in the previous example. This is however not always true, especially in the Czech language, where unexpressed subjects are quite common and thus often occur in most dialogue acts, such as in: " Šel do kina." (He went to the cinema.).

Šel do kina

Went to cinema AuxP Adv Fig. 5 Example of unexpressed subject.

Basic Composite Pair This feature is a boolean value that encodes the relative position of each pair Subject and verb. When the verb precedes the subject, this is often viewed as strong evidence in favour of detecting a question in many European languages such as English and French. However, in the Czech language, this is not always true because of two main factors:

1. Subjects may be omitted, as explained in the previous section. 2. A statement can start with a Direct Object, followed by a Verb and its Subject, such as in "Květiny dostala matka." (The mother got flowers).

Kvȇtiny dostala matka

Flowers got mother

Dialogue Act Model

General Principle

The general principle of our dialogue act recognition approach is to decompose the problem of tagging a complete sentence into the (easier) problems of tagging individual words. Our basic assumption is that every single word contributes to the global dialogue act depending on its form, nature and global context. The proposed approach thus assigns a single dialogue act tag to every word and then combines all the dialogue act tags that contribute to the same sentence to infer a single dialogue act tag for this sentence. The word-tagging process is implemented with a Conditional Random Field (CRF) while the sentence-tagging process is realized with two simple combination models that are described next.

Training and Pre-processing

Only the word-level CRF model is trained. The second combination stage is realized by a non-parametric decision process and thus does not need any training.

The manually annotated dialogue act tag associated to each training utterance is first duplicated and assigned to every word of the utterance. Then, these utterances are automatically tagged with POS-tags and parsed with the Malt parser [START_REF] Nivre | MaltParser: A language-independent system for data-driven dependency parsing[END_REF] to produce a dependency tree. A vector of lexical and syntactic features is then derived from this parse tree for each word of the utterance. A special word is inserted before every utterance, with a single feature that indicates the start of an utterance. This special word is given the same dialogue act tag as the other words of the sentence. Finally, all these feature vectors, along with their associated dialogue act tags are pooled together in sequence and the CRF is trained on this corpus with the classical L-BFGS algorithm.

The data pre-processing procedure described above also applies to the test corpus.

Testing and Dialogue Act Inference

During testing, both word-level and sentence-level models are involved to infer dialogue acts. In the first step, the previously trained CRF is applied on the current words sequence and outputs one dialogue act tag for every word of the sentence. Then, the sentence-level decision process converts this resulting sequence of dialogue act tags into a single dialogue act tag per sentence.

Note that an alternative, single-stage strategy may have been to use a nonstochastic global approach, for instance with a maximum entropy model and global features. However, such an approach usually exploits a bag-of-word hypothesis or otherwise implies to explicitly define sentence-global features. Although we have already used with some success a similar approach in a previous work with words position [START_REF] Král | Lexical Structure for Dialogue Act Recognition[END_REF], we rather investigate in the current work the proposed two-stage strategy, which focuses on modelling the succession of word-level dialogue act tags.

Hidden Markov Models, Maximum-Entropy Markov models (MEMMs) and Conditional Random Fields (CRF) are amongst the most common stochastic classifiers. We have chosen CRF because Laferty et al. have shown in [START_REF] Lafferty | Conditional random fields: Probabilistic models for segmenting and labeling sequence data[END_REF] that CRFs avoid the label bias problem, as compared to MEMMs. Furthermore, CRFs are conditional models, and as such, make a better use of their parameters than generative models such as HMMs to model the target distribution of probability. They have also proven in recent years to be superior to most variants of HMMs in many natural language processing tasks and in particular in a punctuation generation application, which is closely related to dialogue act recognition. Hence, Favre et al. compared in [START_REF] Favre | Syntactically-informed models for comma prediction[END_REF] three sequence models: Hidden-Event Language Model (HELM), factored-HELM and Conditional Random Fields (CRF) for comma prediction. They have shown that the best results are obtained with CRFs, although CRFs may not scale easily to large databases.

We thus use CRFs to compute the conditional probability:

P (DA|F ) = P (c 0 , c 1 , • • • , c n |f 0 , f 1 , • • • , f n ) (1) = n i=0 P (c i |f i , c 1 , • • • , c i-1 )
where

F =< f 0 , f 1 , • • • , f n >
represents the sequence of features vectors, n is the number of words in the utterance, f 0 the initial start word and DA =< c 0 , c 1 , • • • , c n > is the output sequence of dialogue acts.

Sentence-level combination and decision process

We investigate two approaches for the final decision process, which shall output a single dialogue act tag for the whole utterance: Majority voting and Naive Bayes classification.

Majority Voting

The final dialogue act tag is simply the tag with the highest frequency counts amongst the n tags c 1 , • • • , c n . Ambiguous cases are resolved by choosing the tag with the largest posterior probability.

Naive Bayes Classification

In the "Naive Bayes" classifier [START_REF] Grau | Dialogue Act Classification using a Bayesian Approach[END_REF], every tag c i is assumed independent from all others given the Markov assumption. Hence, the probability over the whole utterance is given by Equation 2:

P (c|F ) = n i=1 P (c i = c|f i , c i-1 ) (2)
where

P (c i = c|f i , c i-1 )
is the word-level posterior probability returned by the first-order CRF, when the CRF is constrained to follow the sub-optimal path (c 0 = c, c 1 = c, • • • , c n = c). Note that this tag sequence is different from the one used in the "majority voting case", where the global optimal path returned by the CRF is used.

The resulting dialogue act is the one that maximizes the a posteriori probability: ĉ = arg max c P (c|F )

Evaluation

The proposed two-step model is evaluated on a Czech train reservation corpus and compared with a unigram model and with a baseline CRF model that only exploits lexical and morpho-syntactic features. The evaluation metric is the dialogue act recognition accuracy. In the following, we first describe the Czech corpus, then the two pieces of software that have been used to compute the morphosyntactic tags and the parse tree and we finally discuss the experimental results.

Corpus

The corpus used to validate the proposed approaches is the Czech Railways corpus that contains human-human dialogues. It was created at the University of West Bohemia mainly by members of the Department of Computer Science and Engineering in the context of a train ticket reservation dialogue expert system. The whole corpus has been recorded in laboratory conditions and contains about 12 hours of audio recordings. The audio files have been both manually and automatically transcribed. We thus evaluate our dialogue act recognition approach on both types of transcriptions, in order to further assess its robustness to speech recognition errors.

Automatic transcription has been realized with the jLASER [START_REF] Pavelka | JLASER: An automatic speech recognizer written in Java[END_REF] recogniser, which has been developed in our LICS 1 laboratory. It is based on a so called hybrid framework that combines the advantages of the hidden Markov model approach with those of artificial neural networks. We use HMMs with state emission probabilities computed from the output neuron activations of a neural network (such as the multi-layer perceptron). jLASER has been trained on 6234 sentences (about nine hours), while 2173 sentences (about three hours) pronounced by different speakers are used for testing. Because of the size of the corpus, a class-based 3-gram language model has been used.

All sentences of this "test" corpus have been manually labelled by three different annotators with the following dialogue acts: statements (S), orders (O), yes/no questions (Q[y/n]) and other questions (Q). The DA corpus structure is reported in Table 2, where the number of dialogue acts is shown in column 2. This choice of dialogue acts has been done because our DA recognition module is designed to be used with a rule-based dialogue system that only exploits these four types of information as an input. The following dialogue act recognition experiments are realized on this labelled corpus using a cross-validation procedure, where 10% of the corpus is reserved for the test, another 10% for the development set and 80% for training of the CRF.

Tools

For lemmatization and POS-tagging, we use the mate-tools http://code. google.com/p/mate-tools/. The lemmatizer and POS tagger models are trained on 5853 sentences (94,141 words) randomly taken from the Prague Dependency Tree Bank (PDT 2.0) [START_REF] Hajič | The Prague Dependency Treebank: A Three-Level Annotation Scenario[END_REF] corpus. The PDT 2.0 is a collection of Czech newspaper texts that are annotated on the three following layers: morphological (2 million words), syntactic (1.5 million words) and complex syntactic and semantic layer (0.8 million words). In this work, only the syntactic dependencies of the second layer are considered. The performance of the lemmatizer and POS tagger are evaluated on a different set of 5181 sentences (94,845 words) extracted from the same corpus. The accuracy of the lemmatizer is 81.09%, while the accuracy of our POS tagger is 99.99%. Our tag set contains 11 POS-tags as described in Table 3. Our dependency parser is the Malt Parser v 1.3 trained on 32,616 sentences (567,384 words) from PDT 2.0. The dependency set is thus: Adv, AdvAtr, Apos, Atr, AtrAdv, AtrAtr, AtrObj, Atv, AtvV, AuxC, AuxG, AuxK, AuxO, AuxP, AuxR, AuxT, AuxV, AuxX, AuxY, AuxZ, Coord, ExD, Obj, ObjAtr, Pnom, Pred, Sb. The Labelled Attachment Score (LAS) of our parser is about 66%.

Our CRF toolkit is based on the Stanford OpenNLP library2 , which has been modified in order to include syntactic features. The resulting model has about 3200 parameters.

Baseline rule-based system

Our claim in this work is that structured syntactic features, which cannot be simply derived from word forms, bring relevant information that help a classifier to discriminate between some dialogue acts, even in Czech, which is known to be a free-word order language. We actually show next that despite the theoretical linguistic constructions in Czech, which do not a priori strongly constrain the grammatical structures with regard to word orders, common usage in Czech exhibits statistical properties that are discriminative for the few dialogue acts considered here. Furthermore, we show that such statistical properties cannot be captured with simple deterministic rules, but that they must be considered instead in context within a stochastic model like the proposed CRF that is trained on real data.

To illustrate this idea, let's consider the particularly difficult case of yes-no questions vs. statement. Table 4 shows a typical example of such a case, which cannot be captured by syntactic information in Czech.

Table 4 Example of discrimination between yes/no question and statement that cannot be realized from syntactic information in Czech. Punctuation is not shown because it is not available at the output of speech recognizers and is thus not used by our system.

English Czech Statement

He loves her Miluješ ho Yes/no question Does he love her Miluješ ho

However, despite such difficult theoretical constructions, we have automatically parsed our Czech speech corpus and analyzed the relative frequency of subject relations with the verb on the left of the subject (feature "Basic composite pair"): 48% of such inverted relations occur in statements, which corresponds to a pure random ratio and complies with the free-word order property of Czech, while this ratio goes up to 88% in yes/no questions, which demonstrates that such a feature is indeed informative in common usages of Czech. Nevertheless, we also show next that this observation, in itself, is not enough to accurately discriminate between yes/no questions and statements, and that it must be considered in context to be really useful.

In order to validate this claim, we build next a deterministic baseline model that classifies the four proposed dialogue acts using hand-crafted rules that:

-Include common lexical knowledge, such as interrogative words -Use syntactic rules that match the proposed features described in Section 3.1.2, such as the subject-verb inversion rule just described.

The set of rules is described in Table 5. When several rules apply on the same sentence, the chosen dialogue act is decided with a majority vote. In case of equality, the winner amongst competing dialogue acts is the one with the higher prior probability on the corpus, i.e., in decreasing order: Q, S, Q[y/n], O.

The recognition accuracy of the rule-based system is shown in Table 6. We evaluate two cases: manual word transcription and automatic transcription by jLASER recognizer. Table 6 shows that errors from the speech recognizer don't play an important role for DA recognition, resulting in a decrease of accuracy of about 4%.

The highest score for class O might result from the precision of the set of rules defined for this class. Conversely, the lower score of class S may be due to the difficulty to define a specific rule for this class. We thus only used a list of key-words for S and sentences are mainly classified into this class when no rule from another class is triggered.

Experiments

Two experiments are realized next. The first one performs dialogue act recognition on manual word transcriptions and evaluates and compares the impact of the proposed lexical and syntactic features and the relative performances of both sentence-level combination models. The unigram model corresponds to a very basic baseline approach that only exploits lexical unigram probabilities. We further compare the proposed approach with more advanced baselines that are also based on a CRF but with lexical and morphosyntactic features only (word forms, lemmas and POS-tags). In the second experiment, the same set of models is applied on automatic word transcriptions. This allows assessing the robustness of both our parsers and feature sets to speech recognition errors.

Manual transcription evaluation

Table 7 shows the dialogue act recognition accuracies obtained with the different proposed models. We have computed statistical significance of the difference between two models with the McNemar test, as suggested in [START_REF] Gillick | Some statistical issues in the comparison of speech recognition algorithms[END_REF] for a similar classification task. The p-value is in general below the traditional threshold of 0.05. The p-values of some important comparisons are for instance:

Table 5 Hand-crafted rules used in the deterministic baseline system

Rule

Trigger DA Description 1. Lexical Rules R1 S Occurrence of a word in the list: "bych": conditional form, first person singular "bychom": conditional form, first person plural "jsem": "to be", first person singular "jsme": "to be", first person plural "potřebuji", "potřebujeme" "I need", "we need" "chci", "chceme", "chtěl", "chtěla", "chtěli" "I want", "we want", "he wanted", "she wanted", "they wanted" "znát", "vědět" and "doptat" "to know" and "to ask" R2 Q[y/n] The first word in the sentence is one of: "můžu", "můžete", "můžeme" "can you" in singular and in plural, "can we" "má", "máte", "máme" "do you have" in singular and in plural, "do we have" R3 Q "Wh*" word or a word from the list below is at the beginning of the sentence: "jak", "jakpak" and "kolik" "how" and "how many" We can first observe that the Naive Bayes combination gives in general better results than majority voting, which was expected, as Naive Bayes ex- ploits the posteriors, which are a richer source of information than just the knowledge of the winning class. This table also shows relatively low recognition scores for the class O. This is probably due to the relatively smaller amount of training data for this class. This analysis is supported by the good recognition accuracy obtained by the baseline rule-based system for this class, which does not depend on any training corpus. The best recognition rate is for the class Q, which is both the most frequent class and which is characterized by strong cues, especially concerning the influence of the first word in the sentence (B4NB) as well as distinctive interrogative word forms (B1NB, B2NB).

The most important remark is that the combination of all proposed syntactic and baseline features significantly outperforms all baseline features, which confirms that the proposed syntactic features bring complementary information. This result supports our claim that structured syntactic information might prove useful for dialogue act recognition.

Automatic transcription evaluation

Table 8 shows a similar evaluation to the one in Table 7, except that the textual transcriptions are now obtained automatically with the jLASER speech recogniser. Sentence recognition accuracy is 39.8% and word recognition accuracy is 83.4%. The complete annotation process starts from these imperfect transcriptions, including: lemmatization, POS-tagging, parsing and dialogue act recognition. This experiment thus assess the robustness of the complete processing chain to speech recognition errors, in order to match as closely as possible the actual use of the proposed approach in realistic conditions. We can first observe that the impact of speech recognition errors is moderately large, but not dramatic and thus does not jeopardize the applicability of the proposed approach in real conditions. Hence, while the dialogue act classification errors increase by 30% with the unigram model, they increase by 113% with the baseline CRF B3NB, which was expected because the CRF exploits the correlation between successive words and tags, which may propagate errors amongst words. However, despite its lower robustness, the CRF model still performs better in absolute value than the unigram model. The increase in classification error of the syntactic-aware model is about 183%, which is due to the greater sensibility of the processing chain for this model. Indeed, speech recognition errors are known to have a large impact on POS-tagging and parsing performances. The derived syntactic features are thus also largely impacted by such errors. This also explains why the simple proposed baseline features, such as B4NB, are also the most robust ones.

Conclusions

This work extends our previous works that tended to demonstrate the importance of global structural information for dialogue act recognition by implicitly modelling local constraints with Conditional Random Fields and explicitly proposing global syntactic features derived from automatic parsing of the sentence. Regarding the efficiency of syntactic features for dialogue act recognition, we have provided a number of evidence to support our claim that syntactic information might be important for dialogue act recognition and that the main reason why they have not been widely used so far in this domain is due to (i) the difficulty to reliably parse speech and dialogues; (ii) the intrinsic complexity of the syntactic material as compared to the classical lexical and morphosyntactic tags; and (iii) the lack of robustness of parsers to speech recognition errors. This claim is based on a review of several companion works that show the importance of syntax for both dialogue act recognition and closely related domains such as punctuation generation. Second, we have proposed several simple as well as more complex syntactic features that are derived from a full deep parsing of the sentence and have shown that the use of such features indeed significantly improves the dialogue act classification performance on our Czech corpus. Finally, we have studied the robustness of the proposed system and have shown that, as expected, the most complex syntactic features are also the most sensitive to speech recognition errors.

Hence, given the evidence collected in this work, we conclude that syntax information might prove important for dialogue act recognition, as it has already been shown relevant for many other Natural Language Processing tasks. The main challenge that remains is to increase its robustness to speech recognition errors, but we expect this challenge to be soon overcome, thanks to the great progresses realized in the automatic parsing community in recent years.

Fig. 4

 4 Fig. 4 Example of leftmost root position.

Fig. 6

 6 Fig. 6 Example of inverted subject.

Table 1

 1 Example of a dialogue between the passenger (P) and the agent (A) in a ticket reservation task with the English translation

	Speaker DA	Dialogue in Czech	English translation
	P	Question	Kdy pojede první vlak do	When will the first train go
			Prahy ?	to Prague ?
	A	Question	Chcete rychlík ?	Do you want the express
		yes/no		train ?
	P	Statement	To je jedno.	I don't care.
	A	Statement	V osm hodin.	At eight o'clock.
	P	Order	Dejte mi tedy jeden lístek,	Give me one ticket, please.
			prosím.
	A	Statement	Tady je.	Here it is.

  Dva body ze šesti jsou hodné pozoru

			Sb
	Atr	AuxP	Atr		Pnom	Obj
	Two points of	six	are	worthy attention

Table 2

 2 Description of the Czech Railways DA corpus

			DA corpus
	DA	No.	Example	English translation
	S	566	Chtěl bych jet do Písku.	I would like to go to Písek.
	O	125	Najdi další vlak do Plzně!	Give me the next train to Plzeň !
	Q[y/n]	282	Řekl byste nám další spojení?	Do you say next connection ?
	Q	1200	Jak se dostanu do Šumperka?	How can I go to Šumperk ?
	Sent.	2173

Table 3

 3 Tag-set description

	Abbreviation	Description	Abbreviation	Description
	A	Adjective	P	Pronoun
	C	Numeral	V	Verb
	D	Adverb	R	Preposition
	I	Interjection	T	Particle
	J	Conjunction	Z	Punctuation
	N	Noun

Table 6

 6 Dialogue act recognition accuracy for the baseline rule-based system with manual and automatic word transcription by jLASER recognizer SyNB vs. B3NB: p < 0.001 -SyNB vs. B4NB: p = 0.016 -SyNB vs. BANB: p = 0.002

			2. Morpho-syntactic Rules
	R4	O	Verb with imperative form at the beginning of the sentence
	R5	O	Verb at the beginning of the sentence has a suffix amongst:
			"ej", "me", "te"
			Suffix of imperative form for 2nd person singular, 1st person plural, 2nd
			person plural
				3. Syntactic Rules
	R6	O	The first word of the sentence is the syntactic root
	R7	O	A verb doesn't have any subject
	R8	Q[y/n]	Subject-verb inversion
				4. Default Rule
	R9	S	When no previous rules apply, by default, the statement is chosen
		Transcription type	S	O	Q[y/n]	Q	Global
			Manual	67.3	95.2	92.2	87.9	83.5
			jLASER	67.5	88.8	85.1	82.0	79.0
	-

Table 7

 7 Dialogue act recognition accuracy for different features/approaches with manual word transcription

				Accuracy in [%]
		Features/Approach	S	O	Q[y/n]	Q	Global
		1. Unigram
	B0	Words	93.5	77.6	96.5	89.9	91.0
		2. Majority Voting
	B1MV	Words	87.63	76.61	81.21	99.42	92.50
	B2MV	Words + Lemmas	87.63	76.61	82.27	99.42	92.82
	B3MV	Words + POS-tags	87.63	72.58	81.21	99.50	92.50
	B4MV	Words + Pronoun at the beginning	90.28	76.61	95.39	99.33	95.17
	B5MV	Words + Verb at the beginning	87.63	79.03	95.04	99.42	94.61
	BAMV	Words + All baseline features	88.87	84.68	94.68	99.42	95.21
	S1MV	Words + Dependency labels	87.81	74.19	88.30	99.42	93.51
	S2MV	Words + Root position	89.05	87.10	92.20	99.33	95.03
	S3MV	Words + Unexpressed subject	89.22	78.23	84.75	99.25	93.55
	S4MV	Words + Basic composite pair	88.34	78.23	84.75	99.33	93.37
	SyMV	All features	89.93	92.74	94.68	99.42	95.95
		3. Naive Bayes Classifier
	B1NB	Words	94.52	71.77	83.33	99.25	94.38
	B2NB	Words + Lemmas	94.88	87.90	91.13	99.42	96.50
	B3NB	Words + POS-tags	95.05	80.65	88.30	99.00	95.53
	B4NB	Words + Pronoun at the beginning	96.47	83.06	94.68	99.33	97.05
	B5NB	Words + Verb at the beginning	88.87	48.39	94.33	99.00	92.86
	BANB	Words + All baseline features	92.05	86.29	94.68	99.5	96.18
	S1NB	Words + Dependency labels	94.52	81.45	89.01	99.00	95.53
	S2NB	Words + Root position	90.11	85.48	93.62	99.00	95.21
	S3NB	Words + Unexpressed subject	93.11	85.48	85.82	99.08	95.03
	S4NB	Words + Basic composite pair	93.11	85.48	86.52	99.08	95.12
	SyNB	All features	95.23	95.16	96.81	99.33	97.70

Table 8

 8 Dialogue act recognition accuracy for different features/approaches with automatic word transcription using jLASER speech recogniser

				Accuracy in [%]
		Features/Approach	S	O	Q[y/n]	Q	Global
		1. Unigram
	B0	Words	93.1	68.8	94.7	86.3	88.2
		2. Majority Voting
	B1MV	Words	82.69	29.84	62.06	99.25	86.14
	B2MV	Words + Lemmas	83.92	41.13	65.25	99.17	87.48
	B3MV	Words + POS-tags	81.80	28.23	63.48	99.17	85.96
	B4MV	Words + Pronoun at the beginning	86.04	54.03	80.50	98.75	90.52
	B5MV	Words + Verb at the beginning	81.10	25.00	79.43	98.17	87.11
	BAMV	Words + All baseline features	88.52	58.47	81.25	97.97	90.84
	S1MV	Words + Dependency labels	80.04	33.06	70.57	99.25	86.74
	S2MV	Words + Root position	83.39	50.00	79.08	98.33	89.18
	S3MV	Words + Unexpressed subject	81.63	33.87	61.35	99.33	86.05
	S4MV	Words + Basic composite pair	81.63	33.87	60.28	99.33	85.91
	SyMV	All features	84.28	71.77	82.98	98.17	91.07
		3. Naive Bayes Classifier
	B1NB	Words	88.16	29.84	79.79	99.17	89.83
	B2NB	Words + Lemmas	90.81	33.87	82.27	99.33	91.16
	B3NB	Words + POS-tags	89.93	30.65	80.85	99.08	90.42
	B4NB	Words + Pronoun at the beginning	91.34	66.13	89.36	98.67	93.21
	B5NB	Words + Verb at the beginning	82.51	29.03	82.62	97.83	87.94
	BANB	Words + All baseline features	88.34	68.55	87.59	97.67	92.27
	S1NB	Words + Dependency labels	87.99	33.06	81.56	99.25	90.24
	S2NB	Words + Root position	84.81	69.35	83.33	88.42	91.25
	S3NB	Words + Unexpressed subject	89.93	34.68	80.50	99.08	90.61
	S4NB	Words + Basic composite pair	89.75	33.06	80.14	99.17	90.47
	SyNB	All features	91.17	70.97	88.65	98.00	93.46

http://dit.uvt.nl

http://liks.fav.zcu.cz

http://incubator.apache.org/opennlp

Acknowledgment

This work has been partly supported by the European Regional Development Fund (ERDF), project "NTIS -New Technologies for Information Society", European Centre of Excellence, CZ.1.05/1.1.00/02.0090. We would like also to thank Ms. Michala Beranová for some implementation work.


Amandine Decker
email: amandine.decker@loria.fr

Ellen Breitholtz

Christine Howes
email: christine.howes@gu.se

Staffan Larsson

Topic and genre in dialogue

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

In this paper we argue that topic plays a fundamental role in conversations, and that the concept is needed in addition to that of genre to define interactions. In particular, the concepts of genre and topic need to be separated and orthogonally defined. This would enable modular, reliable and controllable flexible-domain dialogue systems.

In communicative activities, genre and topic tend to be interleaved in the sense that the manner in which a particular topic is addressed can differ significantly across various genres. For instance, a conversation about politics may unfold differently in a formal debate compared to a casual conversation among friends, and a recipe for a dish can be the topic of an instructional dialogue, or a discussion among participants as how to best prepare the dish.

Analysing the influence of genre on the treatment of topics would allow us to understand how general features of interaction are adapted to specific conversations. In this paper we discuss the treatment of topics and genres in different linguistics theories and how studying the way they influence each other may help designing reliable and controllable open-domain dialogue systems that could be adapted to task-oriented conversations in many different domains.

Topic and genre in linguistic theories

There are several areas of research which aim to categorise interactions in ways that are predictive of their communicative (including linguistic) features. These theories are based on a variety of concepts such as (social) (communicative) activity [START_REF] Allwood | An activity based approach to pragmatics[END_REF], (communicative) project, frame [START_REF] Levin | Dialoguegames: Metacommunication structures for natural language interaction[END_REF][START_REF] Henrik | Dialogue games: An approach to discourse analysis[END_REF], (language) (dialogue) game [START_REF] Lewis | Scorekeeping in a language game[END_REF][START_REF] Ginzburg | The interactive stance: Meaning for conversation[END_REF], genre [START_REF] Wong | Conversational types: a topological perspective[END_REF], etc.

When defining genres, a frequently used concept is that of activity in the context of which language occurs. On Allwood's account an activity type is characterised by the goals, roles, artefacts and environment that are associated with it. The carrying out of an activity consists of a number of sub-goals being completed. These may be more or less communicative in nature. For example, instances of the activity type "Buying/selling coffee in a café" are made up of sub-goals such as "conveying which product one wants to order", "conveying how much the costumer should pay", and, finally, "paying/receiving money". These sub-goals could be topics in a discussion carrying out the activity type "Buying/selling coffee in a café" but they could be organised in many ways with for example all the topics following each other linearly or on the contrary being all embedded in each other.

Similarly, genres can be seen as a set of actions that must be realised, or a set of questions under discussion that must be resolved [START_REF] Ginzburg | Semantics of dialogue, Cambridge Handbooks in Language and Linguistics[END_REF], to make a certain interaction successful. In that sense, the genre sets the minimal requirements in terms of outcome for a conversation but it does not say anything on the content of it beyond these requirements. Besides, while genre constrains the surface structure, content plays an important role in the detailed one. Formalising topics and understanding the way they can be articulated could help modelling a hierarchical structure of content.

Topics have been discussed in different ways in the literature, the definitions mostly vary by their granularity. A sentence topic [START_REF] Bolinger | Linear modification[END_REF][START_REF] Firbas | On defining the theme in functional sentence analysis[END_REF][START_REF] Michael | Notes on transitivity and theme in english part i[END_REF]Givón, 1983) is an element of the sentence, usually a noun phrase, that the sentence comments on [START_REF] Hockett | A Course in Modern Linguistics[END_REF]. Discourse topics, on the other hand, are not necessarily explicit. They refer to what a piece of discourse is about, though the formalisation of this "aboutness" is debated. Discourse topics have been defined as based on the "question of immediate concern" [START_REF] Ochs Keenan | Acquiring conversational competence, chapter Questions of immediate concern[END_REF], explicitly stated or not, or as "the proposition or set of propositions that the question of immediate concern presupposes" [START_REF] Schieffelin | Topic as a discourse notion: A study of topic in the conversations of children and adults[END_REF]. Discourse topics are also considered in the frame of certain discourse modelling theories such as Segmented Discourse Representation Theory (SDRT) [START_REF] Asher | Discourse topic[END_REF]. Coming up with a theory organising these different levels of granularity would enable us to come up with a hierarchical modelling of topics [START_REF] Teh | Hierarchical dirichlet processes[END_REF].

Variability in dialogue

The fulfilment of a conversation goal can be achieved in many different ways, encompassing linguistic and extra-linguistic elements, and their various combinations. Consider a scenario at a café where a customer wishes to order a drink. This goal can be accomplished by pointing at the desired drink, providing a verbal description, employing both actions simultaneously, or in some cases, no action may be necessary if the customer is a regular one with a well-known The diverse range of methods exemplifies the flexibility inherent in achieving conversation goals.

The straightforwardness of attaining conversation goals also varies. Sometimes, intermediate questions need to be resolved before reaching a final decision. For instance, a customer may inquire about the type of milk used in the café and only place their order once they know which drinks are lactose-free. In such cases, the fulfilment of the conversation goal is contingent upon gathering additional information and resolving relevant queries. Such examples show how a straightforward request for action can sometimes turn into something more complex, where information is requested and different alternatives can be discussed and compared.

Conversations may also deviate from a strictly goal-oriented path, allowing for detours and tangential discussions. For instance, while inquiring about a specific product, an individual might share an anecdote related to the product itself. Questions about lavender cookies could trigger memories of holidays in Provence and lead to a spirited debate about the finest variety of lavender or even spark a discussion about the seller's vacation plans. Such diversions from the primary topic rely on the participants' freedom and inclination to explore different avenues within the conversation.

While many conversational goals are associated with a default genre (and related dialogue structures), it sometimes happens that dialogue partic-ipants deviate from these defaults. The extent to which default structure diverge from the original goals of a conversation is likely influenced by the genre of the conversation and its level of formality or standardisation. Additionally, the social aspect of the interaction also plays a central role. It appears that conversations with a greater social orientation tend to afford participants more freedom to deviate from the central goal.

Application to dialogue systems

Variability in dialogue is a challenge for generalpurpose dialogue models. There may well be an open-ended universe of dialogue genres (language games, dialogue types), which we cannot hope to map out [START_REF] Wittgenstein | Philosophical Investigations[END_REF]. In any case only a limited number of dialogue genres has so far received attention from the dialogue systems / conversational AI community (including industry). Having a better understanding of the way topics and genre interact could help creating a more modular and general framework that could be fine-tuned for more specific tasks.

Different dialogue genres will be associated with different kinds of dialogue patterns. In a sense, a notion of dialogue genre is not strictly necessary for dialogue systems. What is needed in each domain is dealing with the dialogue patterns that appear there. However, we believe that the notion of genre can serve as a powerful abstraction, allowing dialogue designers to understand which dialogue patterns are relevant in a domain.

Discussion

Distinguishing genre and topic and treating the two as orthogonal contributing factors could provide insights regarding the structural analysis of conversation. In terms of dialogue systems it would improve the adaptation of the model's interventions based on the current topic and its links to the previous ones as well as the type of conversation. However, topic modelling is a complex task even for human annotators [START_REF] Purver | Topic Segmentation[END_REF] and creating guidelines to annotate dialogues based on the topics they discuss and the hierarchy between them is not a trivial problem. Such an annotation guide would make it possible to analyse the differences in terms of topical structure between different types of conversations and make dialogue systems adaptable to specific genres of conversations following this analysis.