Philippe Teutsch
email: philippe.teutsch@univ-lemans.fr

Christophe Kolski
email: christophe.kolski@univ-valenciennes.fr

De Keyser

J M Hoc

M Lind

P Millot

N Moray

J Rasmussen

T B Sheridan

D D Woods

Methodological Approach for Designing Training Courses Tutoring Tools, Analogies with HMI Design for Industrial Supervisory Control

Keywords: Distance training, monitoring, supervisory, human-machine interface, design methodology

This paper presents an on-going work about the design of monitoring human-machine interfaces (HMI) in the domain of distance training course.

The objective is to define interactive tools that facilitate the task of the online teacher. The teacher needs to see the path and the contributions of the learners in the Learning Management System (LMS). The hypothesis is that the approach has strong analogies with design of HMI for industrial supervisory control, where human operators perform supervisory and monitoring tasks concerning variables related to a dynamic process. In both cases, HMI plays a key role in the quality of the human-machine system. The paper presents a comparison between both areas (training monitoring and industrial supervisory control): problematics, task modelling, tools design. Perspectives concern a methodological framework for HMI design and evaluation in the context of supervisory control in distance course.

Introduction

Distance training platforms create complex and highly dynamic system where the online teacher faces difficulties to monitor learning activities of the participants. These difficulties are related to the position of distance working and to the lake of global perception of the individual situations. The tutors have to reconstitute for themselves the puzzle of the learners' activities through the information that the environment provides them. Thus, they need structured information about the learners' activities in order to bring them relevant and effective helps. The aim of the research is to aid teachers in distance training course to gain understanding of their learners and become aware of that what happening in the group they manage (Mazza and Dimitrova 2003).

Training Courses Monitoring Activity

The job of tutor applies in different contexts and to different levels. For example: to guide a group of learners during a synchronous activity during a couple of hours (Gueraud et al. 2004), to manage an on-line forum during several weeks (Georges 2004), or to help a learner along his or her personal path in a set of multimedia resources for several months [START_REF] Gueye | Instrumentation des activités du tuteur en ligne : le cas de CROISIERES, dispositif de formation ouvertes et à distance en langues[END_REF]. In all cases, the quality criteria of the monitoring tools concern the capacity to take into account the different user profiles: pedagogical tutors [START_REF] Barros | Analyzing student interaction processes in order to improve collaboration. The DEGREE approach[END_REF], and also learners themselves for a reflexive point of view on learning path (Heraud et al. 2004), or designers, analysts and teaching trainees [START_REF] May | Keeping Tracks of Users' Activities on Discussion Forum: Reflection and Proposal[END_REF]. This section presents different aspects of the tutoring domain: what is the object of supervisory, what are the tasks of the tutors and what are the Human-Machine Interfaces (HMI) needed to instrument these tasks?

Learning Process Modelling

Training devices are often described at different levels of detail. For example, a training curriculum is composed of several modules, themselves described as sequ ences of courses and exercises. Thus, the learning situation observed by the tutor consists of several levels of data. Each level has its own character of evolution during the training session. The training model proposed by [START_REF] Laperrousaz | Tutor follow-up of a Distance Collective Activity[END_REF]) organizes the learning situation of each learner in three stages of data: (1) the identity and the training context of the person (no significant evolution during the session); (2) the profile of the person, i.e. the disciplinary skills (level of knowledge rationally improving during the session), the control of the training device (experience to quickly acquire at the beginning of the session) and the "how to learn" skills (that can be partially improved during the session); (3) the personal path into the device. This journey includes the planned way, the real path, declined in intermediate points and overall balance, and the local situation, expressed in contributions to the proposed activities. The profile corresponds to the representation built by the tutor concerning the level of the learner's abilities. The learning process can therefore be seen as a set of data of various kinds, relatively independent but inter-related. This data set is observed at different levels of focus: global training pathway, recent period of learning activity or current phase of exercise for example. Each view is characterized by its own dimensions of duration, content and actors, and is guided by its own objectives: observation and monitoring. For example, in the case of foreign language learning, a grammatical error is not fatal if the purpose of the activity is to develop communication skills.

Pedagogical Monitoring Task Modelling

The pedagogical monitoring is divided into three parts according to [START_REF] Després | Synchronous tutoring in distance learning: a model for the synchronous pedagogical monitoring of distance learning activities[END_REF]: the perception of the learners' activity, the learners' support and the management of the activity of monitoring itself. The perception part requires to situate the individual activity inside the training device and inside the group. For that, the tutor has to take an interest in the learners' productions, in the exchanges between learners, and in the group dynamics [START_REF] Laperrousaz | Tutor follow-up of a Distance Collective Activity[END_REF]. Nevertheless, the central task of the tutor is to observe each learner's learning path. The group is an element of the context, like the scenario or time constraints. Learning Management Systems (LMS) usually keep a record of the students' activities in a distance course. LMS accumulate large logs of data of student activities and usually have built-in monitoring features that enable the instructors to view some statistical data such as the history of pages visited, the messages read or posted in discussions, marks achieves in quizzed, and so on (Mazza and Dimitrova 2003). This complex information is rarely used by teachers since it is predominantly in numerical or alphanumerical format, with a poor logical organization, and difficult to manage. For instance, Figure 1 shows a view on the logs captured by a LMS (Moodle): logs are simply listed in the structure of the course. Generally, tutors are aware students' actions throw up three kinds of information: productions to assess, messages posted in forum or by email, and system notifications.

In M.sayed-mouchaweh (Ed.), 28th European Annual Conference on Human Decision-Making and Manual Control, Reims, pp. 58-63, september, ISBN 978-2-915271-34-8. The tutor needs to be able to simultaneously guide substantial numbers of learners, each with a personal path, and with a period of reactivity not exceeding a few hours. These objectives require rapid identification of the learner's situation; not only a record of the learner's activity on the device, but also the articulation of this current situation with the history of the last few sessions, with the complete learning journey, or with paths and situations of the other learners.

Pedagogical Monitoring HMI Needs

LMS platforms allow access to the interactions between user and system, but don't propose synthetic views to the coaching teacher [START_REF] France | Monitoring Virtual Classroom: Visualization Techniques to Observe Student Activities in an e-Learning System[END_REF]). This lack is a problem of designing and developing human-machine interfaces dedicated to the on-line tutor needs (Mazza and Dimitrova 2007). We can include the project FORMID (Gueraud et al. 2004) that allows to monitor and to compare the progress of a group of learners (Figure 2). The system is based on a scenario and tracks both finely modeled. In practice, the tutor seeks to be informed of the progress of each learner in his/her learning path. This information may come directly from the system tracking: participation in activities, quality of quiz answers, exchanges on email or forum, and so on. This information can also be built by the system in the form of alerts: delay in a post or blocking on an exercise for example. In any case, it is necessary to have links between each of these events and the learning context of the learner. The tutor may also need global views on the group of learners i , for example to highlight trends in the difficulty of some exercises at the global level in a topic. It is also important to note that the response times (individual or collective) can be relatively varied, while evolving over time, depending on the objectives, learning strategies or conditions of work of each. The experience of the tutor will be essential to better integrate them into its job. Conceptually, a tutoring interface depends on three principles linked to the tutor's perception of the learner's situation: (1) Context of emergence. The tutor can access the emergency context of the solicitation, i.e. the activity page where the learner was at the moment of the solicitation (production, message or warning). ( 2) Articulation: for each learner, the tutor has to decide message priority; which messages, warnings or production to treat next. The answer can be in the story (recent or not) of the learner with the device. The tutor has to visualize the learner's work using a "zoom" on the learning situation. The levels of focus correspond to the levels of granularity of this situation.

(3) Individual assessment: this is built according to the real journey and the solicitations coming from the learner and his activities.

For instance, CROISIÈRES is an online language learning system; it offers a set of structured resources in order to reach a threshold level in French [START_REF] Gueye | Instrumentation des activités du tuteur en ligne : le cas de CROISIERES, dispositif de formation ouvertes et à distance en langues[END_REF]. It is based on a modular structure: the learners can choose their learning activities in relation to their goals and learning strategies. Several monitoring instruments are available [START_REF] Laperrousaz | Tutor follow-up of a Distance Collective Activity[END_REF]): dashboards, individual assessments and paths, detailed views on contributions.

The chronological view (Figure 3) presents a synthesis of the learner's journey (Principle 3, individual assessment). The time is divided into three periods: the past which represents (synthetically) the journey of the learner, the present which details recent learning activity, and the future which indicates the time available to the end of the course. The page shows three types of information: productions to assess, messages from learners, warning notifications generated by the system (the observation of the learner's work identifies long periods of inactivity, repeated failures in the same exercise, or excessive navigation between several modules). These different items can be related to each other (Principle 2, articulation). The tutor can access productions and the context of activity of the learner from each segment of the learning unit. (Principle 1, context of emergence).

Connection with Industrial Supervisory Control Domain

The goal is to make easier the tutor's task of perception of the learning activity. The means are to structure the organization of the training activity, and to define a model of monitoring of distance learning activity. The challenge that designers of web-based learning environments face is to process complex multi-dimensional data provided by LMS and to transform these data into appropriate representations that can be used by teachers in their practice. The problematics seems to be the same for HMI design for Industrial Supervisory Control. This section proposes a connection between both domains: training monitoring and industrial supervisory control.

Brief recall concerning Industrial process monitoring

Control rooms exist in many industrial domains (chemical processes, transport, power plants…). Human operators have to perform cognitive tasks which cannot be performed by automatisms [START_REF] Hoc | Supervision et contrôle de processus: la cognition en situation dynamique[END_REF]Moray 1997), relatively to dynamic processes composed by hundreds or even thousands of variables. Indeed the integrated automation of industrial processes allows system autonomy during certain normal situations; in these situations the human operators have to perform monitoring tasks, and to verify the production performance, quality and/or safety. In case of accident or incident, they have to intervene to correct, compensate or anticipate problems by using their individual skills (Rasmussen 1986;Stanton 1994). Certain situations are unusual, and the human work can be realized under temporal constraints which can generate stress and risks of human error (Reason 1990). Since many years, a particular attention is given to HMI design and evaluation; it is indeed very important to adapt them to the human tasks to perform and the operator profiles. One considers that the HMI play a central role in the global human-machine system efficiency (Millot 1988;[START_REF] Kolski | Interfaces homme-machine, application aux systèmes industriels complexes (2ème édition)[END_REF][START_REF] Ezzedine | Démarche d'évaluation d'IHM dans les systèmes complexes, application à un poste de supervision du trafic ferroviaire[END_REF]Sheridan 1992Sheridan , 2002)).

In M.sayed-mouchaweh (Ed.), 28th European Annual Conference on Human Decision-Making and Manual Control, Reims, pp. 58-63, september, ISBN 978-2-915271-34-8.

Global comparison and discussion

Apparently, training monitoring and industrial supervisory control do not share the same goals, and do not apply the same methods of intervention (Table 1). Control, command, regulation of processes

Operator modeling

No available formal models of tutoring and tutors' behavior

Several models available about human decisional behaviors Table 1. Differences between the domains However, these domains seem to share the same problematics concerning the instrumentation of the human operators (Table 2). The need of global design approach in the domain of training monitoring seems to find food for thought in the industrial supervisory control domain.

Training Monitoring

Industrial Supervisory Control -Process modeling using different abstraction levels and different points of view -Complex and heterogeneous nature of the variables implied in the various situations -Assistance needs concerning the identification of events and phenomena -Dynamic nature of the processes, monitoring of instantaneous values, patterns, histories -Central role of monitoring and control HMI in (1) the instrumentation and (2) the quality of supervisory control task -Assistance provided by the system concerning computing of indicators and alarm management -Effectiveness and efficiency criterion: performance of operator-instrumentation combination Table 2. Similarities between the domains, focus on HMI design This comparison emphasizes that monitored processes are not the same, but the real approximation concerns the situation of supervision. Whatever the process to oversee, the monitoring task is directly dependent on the tools in terms of HMI. In both cases, the HMI reifies the supervision approach: modeling supervised process, reporting (including risk statements), direct access to data, interaction with components (individual or at a higher level) of the process, assistance in decision-making.

Initial design recommendations can be highlighted. First, the experience of HMI design in industrial processes can help HMI design in training monitoring. Then, the condition of effectiveness is a distinction between, in one hand, the aspects of Domain Modeling, carrying significant differences (industrial supervising versus pedagogical monitoring) and, in other hand, the instrumentation approach, common to both supervision situations. Finally, this instrumentation approach focuses on modeling the process of supervision in terms of situation assessment and decision making [START_REF] Hoc | Supervision et contrôle de processus: la cognition en situation dynamique[END_REF] and on software architecture for integrating tools to support supervision. The process of HMI design can be based on the principle of triangulation [START_REF] Mackay | HCI, Natural Science and Design: A Framework for Triangulation Across Disciplines[END_REF] between theoretical models, field practices and development of artifact. The evaluation phase of the artifacts is essential in the design process.

Conclusion

In both domains, training monitoring and industrial supervisory control, the process to monitor is essentially dynamic, and can be seen according to different abstraction levels. The modeling of supervisory tasks is essential to understand how to meet the needs of the human actors involved. This paper was concerned by monitoring HMI in the domain of distance training course; a first overall comparison has been made with the industrial supervisory control domain.

It seems that some concepts and needs are common, and that the first domain may benefit some progress and proposals from the second one. Many research perspectives can be identified. First it would be possible to work on a methodological framework for the design and evaluation of HMI for distance training monitoring. Stimulation could come from design methods thought for industrial supervisory control, or complex systems (cf. works around the U model, [START_REF] Lepreux | A human-centred methodology applied to decision support system design and evaluation in a railway network context[END_REF], or more generally on HCI-enriched model [START_REF] Kolski | An HCI-enriched model for supporting human-machine systems design and evaluation[END_REF]. It will be important to consider the contribution of analysis and modeling methods from the reliability domain (Villemeur 1992); indeed different situations (normal or abnormal) have to be taken into account. It would be also pertinent to examine the work about ecological HMI (Vicente and Rasmussen 1992; [START_REF] Burns | Ecological Interface Design[END_REF] to consider their benefits in the situation of distance training monitoring.

Acknowledgement

The present research work has been supported by International Campus on Safety and Intermodality in Transportation, the Nord-Pas-de-Calais Region, the European Community, the Regional Delegation for Research and Technology, the Ministry of Higher Education and Research, the ANR and the National Center for Scientific Research. The authors gratefully acknowledge the support of these institutions.

Figure 1 .

 1 Figure 1. View on traced activities in a LMS (Moodle)

Figure 2 .

 2 Figure 2. Project FORMID: view on the learners' progress in the learning scenario (anonymized screenshot, Gueraud et al. 2004)

Figure 3 .

 3 Figure 3. Project CROISIERES: overview of the learning path

  In M.sayed-mouchaweh (Ed.), 28th European Annual Conference on HumanDecision-Making and Manual Control, Reims, pp. 58-63, september, ISBN 978-2-915271-34-8.

i In the case where members of the learning group interact with each other (communicate, exchange documents, motivate each other, develop autonomy), an analogy can be made with a system composed of agents (in terms of multi-agent systems, see for instance(Wooldridge 2009)). This allows us to consider other perspectives that are not described in this paper.

Conference on Learning Through Online Exchanges: Tools, Tasks, Interactions, Multimodality, Corpora (EPAL 2007), Grenoble, France. Mazza, R., Dimitrova, V. (2003)


Yoann Lebrun

Emmanuel Adam

René Mandiau

Christophe Kolski

Interaction between tangible and virtual agents on interactive tables : Principles and case study

Keywords: Multiagent System, Situated Agents, RFID, Interactive table, Smart object

Interactive tables (tactile, tangible and mixed tables) are mainly used as support of interaction to design applications and value-added services. Indeed, new collaborative applications emerged from theses new supports using classical interaction (through keyboard, mouse and screen devices). Nowadays, new sensory entries enable users using their hands to manipulate different objects. This article is indented to propose a global interaction model that associates multiagent system concepts to interactive table entities. The model proposes two types of entities that can be used simultaneously : virtual and tangible entities. The virtual entities can result from video projection or from screen use on the interactive table. The tangible entities are physical objects that can be manipulated by one or several users. These are detected through dedicated sensors. At the end of this article, a case study is presented. It illustrates the management of a road traffic simulator using the TangiSense interactive table. The illustrations show mainly the communication between the different agents. In addition, the simulator presents the advantages of the proposed model in order to facilitate the implied users' decision during the simulation process.

Introduction

For several years, simulators have been proposed in several application fields. These simulators are used, for instance, in the domains of transportation [START_REF] Tumer | Distributed agent-based air traffic flow management[END_REF], ecology [START_REF] Mostaccio | Simulation of ecologic systems using mpi[END_REF] and medical care [START_REF] Ramis | A simulator to improve waiting times at a medical imaging center[END_REF]. Most of them are based on a standard architecture. This architecture uses mainly the usual sensory entries : <screen, keyboard and mouse > [START_REF] Hartmann | A practical guide to computer simulation[END_REF][START_REF] Zeigler | Theory of Modeling and Simulation[END_REF]. However, many simulators require experts to be exploited correctly. The simulators can require software complex functioning. To improve the global functioning of the system, indeed such simulation tools have to enable the collaboration between various users (e.g. to exchange on a common objective). In addition, we intend to work further on more interactive approaches (that can allow an intuitive manipulation by persons stemming from different domains and who are not inevitably experts in the complex simulator functioning). These approaches are different from those related to computer use.

As the spectacular evalution of technologies, computers size became smaller and smaller [START_REF] Weiser | The computer for the twenty-first century[END_REF]. This leads to new interaction platforms such as PDAs, tablets, smartphones and interactive tables. Then, interaction devices can be substituted by the human body movements (mainly fingers). Platform takes too into consideration the manipulation of daily objects. These objects are communicating and/or detected by an interactive table.

Although the users' interaction with the computer is done individually, interactive table interaction can be exploited by several users. The used technology can be intuitive when manipulated by an uninitiated public (e.g. using more natural interactions [START_REF] Ishii | Tangible bits : towards seamless interfaces between people, bits and atoms[END_REF]). The interactive tables use is revolutionized by the new forms of human-machine interaction. This implies to design differently applications.

The reminder of this paper is organized as follows : the first section includes a state of the art about interactive tables and the associated interaction principles. This section covers also technical and software characteristics of these supports. Indeed, we intend to make more intuitive and realistic the applications using more adapted behavior of the used entities (evolving in these interactive support).

Then, we propose a global interaction model through a detailed description of the interactive support entities. The proposed model describes the interaction and communication principles between the various entities and users. This description identifies the sensory entries of the support. Then, it adapts the behavior of the entities/agents of a multiagent system according to the involved environment ((e.g tangible 1 and/or virtual for the interactives tables).

Next, a case study is presented. It illustrates how the objects react to the environment modifications and how they can interact together. The application case deals with an application of road traffic simulation. This application is instantiated on the interactive table TangiSense. This table exploits RFID technology. This technology enables to detect road signs. These signs are provided with "RFID" tags. Agents' association with the entities enables the object to interact and dialogue together and with simulated vehicles videoprojected on the table surface.

The paper ends with a conclusion as well as the evolution of multiagent system applied to theses supports.

Related work

Ambient computing is, nowadays, present in our daily life [START_REF] Abras | Advantages of MAS for the resolution of a power management problem in smart homes[END_REF][START_REF] Buchmayr | A simulator for generating and visualizing sensor data for ambient intelligence environments[END_REF]. The user can interact with several platforms to be connected to the rest of the world. This connection is mainly done thanks to Internet and wireless technologies progress [START_REF] Mhemed | An energy efficient fuzzy logic cluster formation protocol in wireless sensor networks[END_REF][START_REF] Weber | Internet of Things : Legal Perspectives[END_REF]. As these technologies are evolving, new interaction platforms are more used. In this article, we are interested by interactive table as interaction platform. Such tables are considered as innovative interaction support. Indeed, they do not require neither the mouse nor the keyboard to ensure Human-system interaction.

Among the tactile interaction devices, we can cite the "multi-touch" devices [START_REF] Dietz | Diamondtouch : a multi-user touch technology[END_REF]. The interaction through "Multi-touch" devices enables users to interact with the material entities using several contact points. Interaction can be based also on hearing devices [START_REF] Patten | Audiopad : a tag-based interface for musical performance[END_REF]. The user interface can be projected on the interactive table and then can be exploited by several users simultaneously by using digital content. Comparing to traditional interaction platforms, the main advantage of interactive table is mainly to provide the ability of social interaction.

The combination of an interactive table and a tangible user interface (tangible user interfaces make it possible to overlay a virtual model to our perception of the reality) permits new surface, tangible interactive tables [START_REF] Manches | Physical manipulation : evaluating the potential for tangible designs[END_REF]. These tables like the ReacTable [START_REF] Jordà | The reactable : tangible and tabletop music performance[END_REF], the SMART Table 2 , The Table-robots [START_REF] Beaufort | Interactive table to study interactions between swarm of robots and active environments[END_REF], etc. These tables enable manipulating physical objects on their surface. These objects can be as various forms. Their positions are detected according to the used technology : camera [START_REF] Jordà | The reactable : exploring the synergy between live music performance and tabletop tangible interfaces[END_REF][START_REF] Kaltenbrunner | reactivision : a computer-vision framework for table-based tangible interaction[END_REF], magnetic follow-up [START_REF] Weiss | Madgets : actuating widgets on interactive tabletops[END_REF], weighty detection [START_REF] Schmidt | Context acquisition based on load sensing[END_REF], radio frequency [START_REF] Martinussen | Designing with rfid[END_REF], etc. The objects can be identified using different technologies : bar code, image, specific shape, RFID chip, etc.

Then, recognition systems (e.g. TUIO [START_REF] Kaltenbrunner | Tuio -a protocol for table based tangible user interfaces[END_REF] used by IntuiFace and ReacTIVision [START_REF] Kaltenbrunner | reactivision : a computer-vision framework for table-based tangible interaction[END_REF] used by ReacTable) identify and analyze the movements on the tactile surface and/or tangible table. This analysis lists the users' movements possible to occur and that can be recognized by the support. In addition, a processing system can use the received/successful data from the recognition system. They are processed for instance to interpret the users body movements.

Generally, the resulting actions from a recognition system are produced following a state change. For instance, when a pressure sensor detects a presence, it switches on the light. This change does not integrate external factors that are able to alter the object perception (the pressure sensor detects a presence but not if there is enough luminosity in the room, consequently it does not switch on the light). The used systems by interactive tables do not integrate such a concept. The associated actions are related to an actuator (I put the object -I realize an action).

To handle problems related to the majority of interactive surfaces, several systems integrated "artificial intelligence" concept in order to endow the tables of an intelligent behavior. This behavior is ensured through the manipulated entities. The artificial intelligence is exploited in this case by using software agents. These agents are endowed with behaviors, a processing and communication ability like Intelligent Table 3 , the Table-Robots [START_REF] Beaufort | Interactive table to study interactions between swarm of robots and active environments[END_REF], or still the table Blip-Tronic 3000 4 . The Blip-Tronic is a musical instrument to play sounds according to the type of object detected. The tone is varied in dependence on the distance between the objects.

Although many solutions exist, they are generally related to an instantiated application (i.e. for particular interactive tables). The evolving entities do not integrate (or do not enough integrate) the behavior that can be deported to enrich the system intelligence. Furthermore, the entities interact only with the support. We find rarely cases where entities together to exchange the environment data. In order to overcome such limit, in the next section, we propose a global interaction model that can be exploited simultaneously by various virtual and tangible entities evolving on the interactive table surface.

Proposition : Global model interactions for interactive supports using agents typology

This section presents our proposition for managing virtual and/or tangible entities. These entities are evolving on the interactive table surface using the agent concepts. These concepts are used to design and implement heterogeneous applications for interactive tables (e.g. applications for colors and numbers learning in education domain [START_REF] Kubicki | RFID-driven situation awareness on TangiSense, a table interacting with tangible objects[END_REF], games, musical creation, etc.). They take into consideration the environment dynamics characteristics. In addition, it considers the heterogeneousness of the entities, distribution of the information, the system complexity, etc. Then, we list the various implied agents as well as their interactions with the interactive support environment.

The proposed interaction model is based on a multiagent architecture. This architecture is completely described in [START_REF] Kubicki | Simulation in contexts involving an interactive table and tangible objects[END_REF][START_REF] Lebrun | A multi-agent system approach for interactive table using rfid[END_REF]. This architecture illustrates the concepts of instantiated agents through an UML diagram. This diagram defines the agents that are involved in the reception of the sensory entrances from the interactive tables. It defines also the behavior associated to the manipulated entities. This behavior enables the agent autonomy and is represented by a set of roles. These roles apply rules, aims and constraints. To implement the agent behavior, we need to define their typology and the possible interactions between them and the users.

We distinguish three types of agents : tangible agents, situated virtual agents and non situated agents (detailed below). The types form specific groups. For example, the group of the situated agents includes tangible agents and situated virtual agents. The non situated agents group correspond to all the agents types that ensure the good multiagent system continuity (for example, a virtual agents observers).

The tangible agents. A tangible agent is connected to a tangible object (object which can be manipulated by one or several users).The agent association to tangible object is elaborated in order to attribute roles to objects through agents. The tangible agents are agents situated in a plan corresponding to the interactive support detection surface. They react to the environment modifications issued from the objects movements.

The situated virtual agents. A situated virtual agent is associated to virtual objects that are endowed with a graphical representation on the interactive support. Unlike tangible objects, virtual objects are able to change forms, colors, size and to disappear throughout the application. Every situated virtual agent can reason and have a set of roles that enable it to evolve in an environment. It also has a limited view limited to a perimeter. This perimeter can vary according to its roles according to the instantiated application. In the case of tactile supports, the user has the possibility to directly move virtual objects.

The non situated agents. The non situated agents handle the administration of the interactive support and the application progress. Virtual agent's observers are charged to ensure the best operation progress between the various agents (tangible or virtual ones). Their role corresponds to receive and transfer information from other interactive support layers (hardware, middleware or software). They manage also the dynamics of the agent roles and check the coherence of these roles. In addition to that, they contribute to integrate new agents thanks to the environment global view.

The interactions between agents are based on the language FIPA-ACL [START_REF] Bellifemine | Developing Multi-Agent Systems with JADE[END_REF] to exchange information and data. Figure 1 illustrates the principles of interactions between the various groups of agents as well as the interactions between users. The interactive support can be exploited by a user (named U n ). The interaction between users and the support results from the manipulation of tangible objects and/or from the manipulation of virtual objects if the support offers this possibility (e.g. to manipulate virtual objects, the virtual support must be tactile).

We distinguish several types of interactions in the system : the observation interactions between non situated agents and situated agents. They enable, for example, situated agent to obtain a new role ; the exchange interactions that correspond to the messages sent between the groups of tangible agents and the situated virtual group ; the material interactions that are used to receive the signals emitted by physical environment and to emit notifications in the materials (for example, the LED activation, the sound activation if the hardware permits it) ; the users' interactions that result from the objects manipulation. The user can move tangible and virtual objects (the virtual is movable : either indirectly through a tangible object or directly on the table if it has sensors) ; the strong interactions that result from the subordination of the position of an element according to the position of other one. In this case, we identify the interactions between the agents (tangible or virtual) and between users and agents (tangible or virtual) as illustrated in Figure 2. Then, we deduct for example that a tangible agent can act on the position of virtual agents and not the opposite. In this situation, when the user moves a tangible object, he/she visualizes and modifies the position of a virtual object.

S V A =⇒ S V A T A =⇒ S V A Legend T A =⇒ T A =⇒ : act on the agent location U =⇒ T A U =⇒ S V A Figure 2.

Interaction between agents and users

From this modeling, we suggest applying the various agents' typologies as well as their interactions in the following section through an application for road traffic simulation. This application is applied on the interactive table TangiSense. It enables the manipulation of tangible objects using RFID technology.

Case study : agent-based simulation on interactive table

In this section, we propose to implement interactions between the agents presented in the previous section (Section 3) on the interactive table TangiSense. The TangiSense table integrates RFID technology allowing detecting the position of objects through the provided tags.

According to existing simulators and their related problematic (described in section 1 such as use difficulty, lack of collaboration, etc.), we show the interest of a simulator dedicated to the management of transportation network.

The simulator of road traffic proposed is a network composed by links (roads, highways, etc.) and nodes (crossroads). Thus, it is a set of roads. The simulator is intended to be used by experts in security, architecture, transportation and also by non expert like local elected member to obtain agreement on road or infrastructure modifications. The Figure 3 illustrates example of road traffic simulation. The network is visualized through a video projection on the TangiSense table. Tangible objects are used for road sign (traffic lights, no-entry sign, one-way street, etc.). They are manipulated by the users and they interact with other virtual objects. These objects are equipped with RFID tags to be able to modify the network structure by modifying the crossraod type or by reducing the limit speed, etc.

Other crossroad objects are used by being moved, by enlarging the map, by showing the street name, etc. These objects are associated to a tangible agent and used to facilitate the interactions between the users and the table. According to computer use, the actions are more intuitive and can be multiple (according to the number of persons and objects used simultaneously).

The simulator aims to test hypotheses of waiting time reduction in the crossroad, crisis situations management (accidents for example) and infrastructure modifications. Other objectives concern the consideration of the user actions on the application while validating the principle of association of virtual agents and tangible agents.

Figure 4(a) illustrated that roads and vehicles are initialized and shown when launching the application. Roads are generated from OpenStreetMap 5 cartography (a free map that enables to simulate the road traffic of any city). Vehicles are associated to virtual agents and generated from an XML file. This file contains their starting point. The XML file concerns also if vehicles move randomly or toward a set of objectives to achieve. Every crossroad is associated to a virtual agent which manages all the entries with roles. To modify a crossroad, the user has to put a tangible object on the road. When the object is detected by the support, we choose to light LEDs beneath it to indicate to the users the taking into account of the object by the system. The agent in charge of the crossroad validates the user action when entrances are affected (Figure 4(b)) and executes the entrance behavior. In the case of a crossroad with traffic lights, vehicles move then according to the lights state. During the simulation, the virtual vehicles/agents dialogue with the tangible objects/agents put on the table by the users. to the road limited to 50 km/h. During the simulation, the vehicle respects the road sign imposed by the users and accelerates its speed according to the light state (red, green or orange). When the light communicates its state to the vehicle agent, the waiting time of the vehicle is more or less long and varies according to the light transition time. For example, when the first light is red, the waiting time is approximately 20 seconds while the second and the third time are respectively of 28 and 8 seconds. The speed limit objet or roadwork object put by users shows the impact on the speed of the vehicle.

Accordingly, we validated such representative example, the principles of agent to tangible and virtual entities operating at the surface of interactive tables. We showed the combination of agents behavior in order to achieve a set of roles during the simulation but also for their interaction/communication during the simulation.

Conclusion

In this article, we presented new concepts for managing interaction platforms (such as interactive tables exploiting agents). We propose a solution that handles some existing problems using interactive tables. The complexity of these platforms results from multiple technologies related to object detection and display. It integrates also new forms of interactions such as : multi-user, tactile, etc.

We suggested modeling the interactions between various agent from the proposed typology that represent the tangible and/or virtual entities. These entities are evolving on the interactive table surface. The entities have to be able to interact between them and also with users. The personalization of these entities, virtual or physical, will allow them through their intelligence to realize a set of actions and to adapt themselves to different environments.

Then, we validate the various agent typology on the TangiSense interactive table. They interact with both virtual and tangible objects. We highlighted all the elements and the concepts proposed throughout this article to enrich the objects behavior by communication/interaction between them or with human beings using interactive supports.

We intend to expand our work by using several interactive supports (interactive tables, tablets, smartphones, etc.) simultaneously. These supports can be connected through local area network using Internet. The objective is to distributed solutions for solving problems concerning crisis. These problems require strong coordination of actors that can be performed locally (on the same device) or remotely through a set of devices.

Figure 1 .

 1 Figure 1. Different interactions on interactive support

Figure 3 .

 3 Figure 3. A road traffic simulation on the TangiSense interactive table

  (a) Establishment of road sign objects at a crossroads (b) Management of road sign by an agent

Figure 4 .

 4 Figure 4. Changing the virtual agent environment using tangible objects

Figure 5 Figure 5 .

 55 Figure 5. Instantaneous speed of a vehicle based on the road signs.

Panasonic. Intelligent Table. Website, http ://en.akihabaranews.com/7007/household/hdtv-the-interactive-and-intelligent-tableby-panasonic. 2006. (accessible in 2012)

4. Peter Benett, Sean Toru, and Lisa Tutte-Scali. Blip-Tronic 3000. Website, http ://www.petecube.com/bliptronic3000/. (accessible in 2012).

Acknowledgements

This research was partially financed by the French Ministry of Education, Research & Technology, the Nord/Pas-de-Calais Region, the National Center for Scientific Research (CNRS), the FEDER program, CISIT (Plaiimob project), and the French National Research Agency (ANR TTT and IMAGIT projects, financial IMAGIT support : ANR-10-CORD-017). The authors would like to thank the partners in the TTT and IMAGIT projects : LIG, RFIdees, the CEA and Supertec.


Sylvain Grellet -Brgm

Hylke Van Der Schaaf

Integrating new OGC Standards

•

  Suggestion from the Water Quality IE experience o We plan to contribute back to the standard baseline o Water ML2.0 Part 1 'Timeseries' ○ Needs to be updated to take into account the revision of O&M (ISO 19156:2011) => Observations, Measurements and Samples a.k.a 'OMS' (ISO 19156:2023) ○ And also TimeSeries ML (issued from Part 1) : also currently integrating OMS ○ And define a best practice to expose it using SensorThings API o Water ML 2.0 Part 5 'Water Quality' ○ Building on the output of the Water Quality IE ○ High chance the IE will recommend in its engineering report to o Generate a Best Practice document o And also revise Part 1 (as we touched quantity during the IE) 3 Integrating new OGC Standards: what does that imply ? • OGC Innovation Continuum 1°/ Water Quality IE output 2°/ Present the findings and suggestions to the Hydro DWG • OGC Innovation Continuum ○ Water ML2.0 Part 5 : Water Quality Best Practice review 3°/ No need to create a Standard Working Group (will be faster) Needs to prepare the document and submit it to the Hydro DWG for review & discussion Integrating new OGC Standards: what does that imply ? • OGC Innovation Continuum ○ Water ML2.0 Part 1 : review 3°/ A review of the standard should trigger a discussion/work at the Standard Working Group • No matter the timeline, there is a move towards more and more use of OGC SensorThings API to exchange Observation data and, soon, Sample data => let's start learning / integrating it ! • We are already working on having a better implementation coverage o Server side : FROST already implementing the proposed Water Quality Model o Client side : SensorThings API Part 1 being implemented in QGIS core. Plugin will only be for Graphical User Interface • But 1st let's get acquainted with SensorThings API Part 1 OGC SensorThings API? • A standard for exchanging sensor data and metadata ○ Historic data & current data ○ JSON Encoded ○ RESTful ○ Adapting OASIS OData URL patterns and query options ○ Supporting OASIS MQTT messaging • Easy to use & understandable

Do It Yourself

• Docker Quick-Start: https://fraunhoferiosb.github.io/FROST-Server/deployment/docker.html • Full SensorThings API Tutorial https://fraunhoferiosb.github.io/FROST-Server/sensorthingsapi/1_Home.html • Demo service: https://ogc-demo.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1 (As long as nobody deletes everything) (Or the server catches fire) SensorThings -Live Demo https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1 ~760 000 000 Observations ~21 000 Datastreams ~5 000 Stations Tutorial: https://datacoveeu.github.io/API4INSPIRE/sensorthingsapi/1_Home.html Data • 1 Map https://api4inspire.k8s.ilt-dmz.iosb.fraunhofer.de/servlet/is/107/ https://lubw.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1 https://sensorthings-wq.brgm-rec.fr/FROST-Server/v1.0 https://sta4hydrometry.brgm-rec.fr/FROST-Server/v1.1


Michèle Duribreux
email: duribreux@univ-valenciennes.fr

Christophe Kolski

Bernard Houriez

INTEGRATION OF COGNITIVE ERGONOMICS CONCEPTS IN KNOWLEDGE BASED SYSTEM DEVELOPMENT METHODOLOGIES

Keywords: Human-centered design, Knowledge engineering, Interactive approaches, Cooperation methodology, Human factors

A Knowledge Based System (KBS) has to permit to one or more users, generally not experts, to solve problems concerning of human expertise. Present KBS development methods help designers to define systems which correctly function from expert competence point of view, but which have important deficiencies from interaction point of view with users. This paper presents basis of a methodology which views at once a co-operative approach between expert, user, computer engineer, human factors specialist and knowledge engineer for development of acceptable, useful and usable systems.

INTRODUCTION

At present, the problem of users and Knowledge Based Systems (KBS) interaction comes up in a crucial way since, among the available systems, few of them are really used. Indeed, in most of applications, KBS are only developed from experts knowledge, and end users are only considering during system integration within its environment [START_REF] Graw | User interfaces for knowledge-based systems[END_REF].

In first part, the knowledge acquisition will be presented as a model-building approach including both elicitation and analysis. Considering this approach, it may be pointed out that expert reasoning acquisition remains one of the prominent aims of knowledge engineers. However, our attention will be devoted most particularly to the participation of KBS users throughout the developing process. Actually, it has been demonstrated by many authors that by taking human factors into account and by setting a participation approach, KBS integration in the Man-Machine system is thus made easier, thereby improving most particularly its operative ergonomics. Therefore, in second part, a system development cooperative methodology is proposed. Third part of this paper presents an industrial example about the use of this methodology.

KBS METHODOLOGIES

Knowledge acquisition

The outcome of a Knowledge Based System (KBS) development project depends most of all upon the initial knowledge acquisition phase [START_REF] Gaines | Eliciting knowledge and transferring it effectively to a knowledge-based system[END_REF]. As a matter in fact many studies have been aimed at improving knowledge acquisition process. Thus, knowledge acquisition no longer tries to be passed on directly from expert knowledge right to an implemented representation, but rather to formulate an intermediate expertise model, followed later on by its implementation [START_REF] Johnson | Knowledge elicitation for second generation expert systems[END_REF][START_REF] Wielinga | Towards a unification of knowledge modelling approaches[END_REF]. While knowledge elicitation techniques are now reported to be under control, the expertise modeling phase is still deeply complex [START_REF] Krivine | L'acquisition des connaissances vue comme un processus de modélisation: méthodes et outils[END_REF].

Human sciences contribution is important to outcome a project which aims to model and broadcast experts knowledge. Cognitive ergonomics, which deals with study about interactions between human and its environment, constitute a preference inspiration source for designers. This induces to present the cognitive ergonomics intervention in KBS development.

Cognitive ergonomics intervention

The cognitive ergonomics intervention could be examined on different levels as displayed by the examples hereafter, according with acceptability, utility and usability criteria [START_REF] Nielsen | Usability engineering[END_REF]:

• knowledge elicitation: the human factors specialist makes use of theoretical knowledge relative to mental processes as well as tested methods to report on the knowledge elicitation phase, mainly resulting from work analysis methods (i.e. task and activity-based analysis).

Concerning this topic, presently active researches in cognitive sciences, but also in software and human engineering about human tasks analysis have been brought to the fore. Some graphical methods among the presently most famous and most used which aims to model them could be quoted: GOMS [START_REF] Card | The psychology of human-computer interaction[END_REF] and its recent variations [START_REF] John | Towards real-time GOMS: a model of expert behaviour in a highly interactive task[END_REF], SADT/Petri Network [START_REF] Abed | Task analysis and modelisation by using SADT and PETRI network[END_REF]. Such methods are very useful in order to specify KBS which really are centered on users tasks.

• ergonomic specification and evaluation of Man-Machine interfaces: during the determination of the interactions between KBS and its users, the human factors specialist is an essential contribution to their specification as well as their evaluation. To illustrate this point, in the field of ergonomics literature, hundreds of recommendations are made aiming to the improvement of Man-Machine interfaces [START_REF] Smith | Guidelines for designing user interface software[END_REF][START_REF] Schneiderman | Designing the user interface: strategies for effective humancomputer interaction[END_REF][START_REF] Brown | Human-computer interface design guidelines[END_REF][START_REF] Scapin | Organizing human factors knowledge for the evaluation and design of interfaces[END_REF]. It may also be noticed that it is essential for ergonomic interface development to use a KBS potential user model. Indeed, even though expert knowledge constitutes the KBS base, it is nonetheless necessary to consider the subsequent use which will be made. This model will be found again in the methodology which is proposed below. Concerning KBS evaluation, richness of available methods have had been insist on. Classifications of interactive systems evaluation methods could be seen about this topic (Wilson and Corlett, 1990;[START_REF] Senach | Evaluation de l'ergonomie des interfaces homme-machine: du prototypage aux systèmes experts[END_REF][START_REF] Sweeney | Evaluating user-computer interaction: a framework[END_REF][START_REF] Grislin | Towards an Organization of Man-Machine Interface Evaluation Techniques using a Usability Criteria Grid[END_REF].

• following on-site installation: the human factors specialist should analyze the possible constraints resulting from KBS (that is: work organization and tasks divisions) to quote but a few of them and as a consequence should propose adequate improvements. The previously quoted tasks modeling methods will have a great importance, since they will permit to compare theoretical tasks model (which is define during prior analysis) with real model of KBS user activities. This comparison allows to put in light problems which concerns its usefulness and usability [START_REF] Abed | Task analysis and modelisation by using SADT and PETRI network[END_REF][START_REF] Millot | Man-Machine Cooperation in Telerobotics: Problematics and Methodologies[END_REF].

Then, this set of observations induces us to bring to light the dependency between knowledge engineer and human factors specialist. Indeed, the knowledge engineer, together with the human factors specialist is to analyze the user's needs while taking the user's abilities and knowledge as well as the final system facilities and limits into account.

INTERACTIVE METHODOLOGY FOR KBS DEVELOPMENT

The proposed methodology aims to give some answer elements to the sensitive problem concerning KBS integration within its operational environment with a great attention to end users expectations.

This methodology, covering the whole KBS development life-cycle describes the necessary activities to define:

• the expertise model to acquire and characterize expert knowledge.

• the use model to take the user's needs into account in order to specify Man-Machine interfaces.

• the organization model to take operational environment of the final system into account.

Tridimensional life cycle

These three activities are integrated in a life cycle which organize the different KBS development phases, as shown in figure 1. This development is composed of the five following phases:

• Problem analysis which leads to a book of requirements. This document , which is established from client's needs, points to the problem to solve and expectations in terms of results too.

• Specification which elaborates a detailed conceptual solution independently of any development means, based on the book of requirements. Specification is split up in three analysis facets: expert knowledge analysis, physical environment analysis, and use forms specification. The expertise model itself is completed by two models characteristic of system cooperation form with the user. They are organization model and use model.

• Design phase associates these three models with a view of describing future system functionalities, that is identify the main knowledge modules and their inter-relations but also define the user's interfaces.

• Implementation define possible solution from organization and real characteristics of software and hardware. Modules are gathered to build the final system. • Exploitation confronts system and maintaining operations which integrate debugging and concern KBS evolution too.

Validation is made all along life cycle and constitute by itself a current research topic (participate process, As shown in figure 1, KBS development requires three types of competencies marked in cycle by columns with different gray levels:

• Competencies in knowledge engineering to elicit and analyze expert knowledge in order to build knowledge base.

• Competencies in cognitive ergonomics to take future use system into account.

• Competencies in computer science to allow KBS kernel elaboration and system integration within its physical environment.

Actors involvement in life cycle

KBS development requires now the participation of actors with varied competencies and interests instead of two main interveners like before (expert and knowledge engineer). Particularly, experts and knowledge engineers, users and human factors specialists, and computer engineers are concerned in turn during different life cycle phases (figure 2). • knowledge engineer who competencies are close to human sciences (psychology more particularly) elicits and analyses expert knowledge, elaborates models.

• Computer engineer, who masters computer and artificial intelligence techniques, defines trends for physical development and concretizes system by implementing different modules.

• Human factors specialist who analyses and/or anticipates user-system interactions forms, analyses user needs, follows their evolution and manage adaptations until system exploitation.

In order to evaluate this tridimensional life cycle, this methodological framework was tested on a real application.

INDUSTRIAL APPLICATION

During its activities, the L.A.M.I.H., in collaboration with SOLLAC steel works of Dunkirk and IRSID (Institut de la Recherche et de la Sidérurgie Française), developed a knowledge based system dedicated to help in the diagnosis of defect causes on steel rails (blooms) and plate made in the steelworks [START_REF] Benkirane | Methodological knowledge elicitation approach: application to the design and realization of a knowledge based system dedicated to the technical diagnosis[END_REF].

The setting up of continuous casting of steels produces blooms, about ten meters long, about one large and half a meter thick, from melted steel (figure 3). These blooms are later re-used through a rolling mill with the aim of its transformation in plate.

This type of installation can be the cause of quality problems. Indeed, according to the blooms format, melting steel quality, process supervision parameters, the users are sometimes faced with an epidemic occurrence of surface aspect defects or/and inside the blooms. The latter, observable on blooms or revealed on plates after lamination, impair the productivity (repairs on blooms and/or plates, scraps, manufacture stimulation ...) and come from causes sometimes difficult to identify.

fig. 3: The process of blooms and plates fabrication

The main difficulties met to make a diagnosis result from the gigantism of the installations and from the wide range of semi-finished products. These various factors of the expert in front of a very large quantity of information, widely spread in time and space. It is in this context that the LAMIH, with the help of experts in continuous casting of SOLLAC steel works of Dunkirk and IRSID, designed and realized the "COCCINELLE" system.

The KBS development follows the sequence of phases shown on figure 1. This article focuses on the specification phase with the elaboration of the expertise model, the use model and the organization model.

Expertise Model

Expertise model framework is based on KADS model [START_REF] Wielinga | Towards a unification of knowledge modelling approaches[END_REF] which considers differentiation between knowledge types according to three levels:

• Domain level contains domain elements which are concerned by the problem.

• Inference level describes functional point of view about this domain knowledge.

• Task level specify goals and the way to organize and control reasoning steps to reach these goals.

The obtained model simultaneously contains a specification of problem solving method as well as a domain knowledge organization, which depends on their function in solving process.

In COCCINELLE application, expert reasoning is done through three steps which are put in light in inference structure presented in figure 4: • Identify population at the origin of crisis (component) from noted defect (problem)

• Specify crisis characteristic clues (symptoms)

• Generate a list of probable causes of these defects (diagnostic).

Organization Model

Organization model specifies system function, how it is split up into tasks and sub-tasks by expert. The SADT method (Structured Analysis and Design Technique) (Ross 1977) is an important contribution to elaborate this model. On the one hand, it represents system functional hierarchies and data flows. On the other hand, it simplifies communications between expert, knowledge engineer and computer engineer.

The actigram in figure 5 describes system global function and its decomposition into three sub-tasks. It permits to:

• Distinguish different modules which spilt up the system.

• Visualize clearly tasks management .

• Point out the user type for main system tasks. In the example, to realize detection and crisis description tasks, the consultant has to be an expert. Effectively, expertise lies in the choice of significant rates to characterize crisis.

• Present technical constraints of system integration in its environment such as the necessary link between system and site production data and the requirement to dispose of reference thresholds which are characteristic of firm know-how. • Inference level describes modifications to bring expertise model inference level in order to take users competencies and goals into account.

• Task level take users competencies into account navigate within previous level. Each task is described with its goal, its control and presentation constraints. Explanations about taken decisions and obtained results after solving are conceivable at this level.

CONCLUSION

In synthesis, a KBS development methodology is proposed here, not involving expert reasoning but also:

• Taking the final system operational environment into account.

• Setting up a coherent dialogue with the user by using particularly suitable vocabulary.

• Generating explanations suited to different types of users.

Thus, KBS development should take knowledge engineering into account as regards knowledge acquisition as well as cognitive ergonomics as for the final system adaptation to the user. In both disciplines, the former conditions KBS quality while the latter is essential for the acceptance of the system by the user.

COCCINELLE is now using in industrial environment. This shows the system responds to users expectations and that our co-operative methodology which is the base of the development of COCCINELLE is validated.

Fig. 1 .

 1 Fig. 1. Tridimensional life cycle

Fig. 2 .

 2 Fig. 2. Actors and life cycle phasesIn KBS development, three types of specialists are distinguished:• knowledge engineer who competencies are close to human sciences (psychology more particularly) elicits and analyses expert knowledge, elaborates models.• Computer engineer, who masters computer and artificial intelligence techniques, defines trends for physical development and concretizes system by implementing different modules.• Human factors specialist who analyses and/or anticipates user-system interactions forms,

  Fig. 4. Inference " diagnose causes"

  To be consistent with expertise model, use model have to keep same organization, i.e. KADS formalism: • domain level puts in relation users domain knowledge with expert's one ( own user terminology, synonymous, data presentation symbolic such as textual comments, graphical objects, ...).

	Scenarios simulation give the advantage to elicit
	information both user and expert. From user point of
	view, simulation permits to study needs in terms of
	information type and nature and to specify his
	competence and understanding levels in domain.
	From expert point of view, simulation puts in light
	aid type which expert introduces in problem solving,
	tasks sequence according to simulated problems,
	explanation levels expert bring either to guide solving
	method, or to give precision about solving, or to
	clarify misunderstanding points. This simulation
	gives occasion for knowledge engineer to evaluate
	knowledge level difference between expert and user.
	Elaborate use model consists in rationalizing and
	organizing the obtained results during simulation to
	describe the way by which a end-user has to represent					Sys tem
	himself system functioning.				strategy	model
		Data production	DIA GNO ZE DEFECTS	Failure
		SOLLAC	CAU SES	Causes
		referencies			A0
				Analyze	Technical
				module	module
					Cons ultant
		strategy				Sys tem model
	Data
	referencies production SOLLAC	Detect a cris is	A1	Identified defect	cris is Describe the	symptoms Crisis
					A2
							Generate the	Failure
							caus es	A3	Causes
		Analyze
		Expert consultant module			Novice consultant Technical module
	Fig. 5. Actigram of the two first application levels
	4.3. Use model
	Expert knowledge make up KBS base but they do not
	generally take use into account. A cognitive
	approach, similar to expert knowledge elicitation,
	allow to consider users' needs.
	The cognitive ergonomics role is to disclose
	knowledge on users' needs and describe knowledge
	domain and expertise level from user point of view.
	User activity analysis takes human factors into
	account in order to characterize working habits,
	users' needs and technical constraints. The simulated
	putting in situation of expert and user problem
	solving, for instance with "wizard of Oz" technique
	(Salembier 1991), generates dialogues which are
	recorded and then analyzed. Knowledge engineer, in
	collaboration with expert, is able to propose
	situations bringing into play problem types. Then,
	human factors specialist studies interactions and
	assistance types.


Aynaz Adl Zarrabi

Melissa Jeulin

Pauline Bardet

Lionel Naccache

Jean-Julien J Aucouturier

Emmanuel Ponsot

Marie Villain

Aynaz Adl

Mélissa Jeulin #"

Pauline Commère

Separating representational and noise components of speech prosody perception after brain stroke

Keywords: stroke, prosody, reverse-correlation, internal noise, perception

come   L'archive ouverte pluridisciplinaire

Introduction

After a right hemisphere stroke, more than half of the patients present a communication disorder such as aprosodia [START_REF] Blake | Prevalence and patterns of right hemisphere cognitive/communicative deficits: Retrospective data from an inpatient rehabilitation unit[END_REF][START_REF] Côté | Towards a description of clinical communication impairment profiles following right-hemisphere damage[END_REF][START_REF] Villain | Affective Prosody and Depression After Stroke: A Pilot Study[END_REF] , the impossibility to produce or comprehend speech prosody -or the "melody" of speech. Despite the social-cognitive implications for patients of not being able to process e.g. emotional or attitudinal expressions [START_REF] Etchepare | Toward a Two-Dimensional Model of Social Cognition in Clinical Neuropsychology: A Systematic Review of Factor Structure Studies[END_REF] , aprosodia following stroke has received scant attention: first, existing assessment tools based on simple perceptive tasks are insufficiently sensitive; second, we lack mechanistic understanding of why patients perform poorly on such tasks and therefore lack practical therapeutic targets for their subsequent rehabilitation [START_REF] Benedetti | Assessment tools and rehabilitation treatments for aprosodia following acquired brain injury: A scoping review[END_REF] .

When studying the neural mechanisms that relate physical stimuli to perception, the modern field of psychophysics has largely moved from simply measuring sensory thresholds and psychometric functions, and now provides a toolbox of techniques to measure and fit multi-staged models able to simulate participant behaviour 6 . Notably for the example of speech prosody, the psychophysical technique of reverse-correlation (or "classification images") [START_REF] Murray | Classification images: A review[END_REF] allows estimating, at the individual level, not only what sensory representations subtend the normal or abnormal perception of e.g. interrogative prosody [START_REF] Ponsot | Cracking the social code of speech prosody using reverse correlation[END_REF] , but also "internal noise" parameters that capture aspects of behavioral variability that are of potential neurological relevance [START_REF] Neri | How inherently noisy is human sensory processing?[END_REF][START_REF] Faisal | Noise in the nervous system[END_REF] . While the representation + noise model has a rich history in healthy participants, with or without peripheral hearing impairment [START_REF] De Boer | On cochlear encoding: Potentialities and limitations of the reversecorrelation technique[END_REF][START_REF] Hoyland | Reverse Correlation Uncovers More Complete Tinnitus Spectra[END_REF] , its use in participants with neurological or developmental disorders has received relatively little attention [START_REF] Vilidaite | Internal noise estimates correlate with autistic traits[END_REF][START_REF] Park | High internal noise and poor external noise filtering characterize perception in autism spectrum disorder[END_REF]15 .

Here we show on a sample of N=22 right-hemisphere brain stroke survivors that such simple procedures promise to enrich the current clinical toolbox with more sensitive and informative markers of receptive aprosodia. impairments, attentional impairments, neglect, and dysphasia etc. Patients included in the study (Table 1) had a history of supratentorial right-hemisphere ischaemic stroke, corroborated by clinical assessments NIH stroke scale (NIHSS; M=10.8) and brain MRI, and dating less than 1y (M=4 months) at the time of inclusion; were first-language French speakers; and had no disorders of wakefulness/consciousness, dementia, severe dysarthria, psychiatric antecedents (>2 months inpatient) or major visual or auditory impairment (> 40dB HL). Patients with language comprehension deficits (score < 10/15 on the BDAE instruction-following task) were excluded from the study. In addition, a group of N=12 controls matched in age, sex and degree of hearing loss was recruited via the INSEAD-Sorbonne Université Center for Behavioural Science, also in Paris, France.

Materials and methods

Participants

Clinical assessment

Two subtests of the French version of the "Montréal Evaluation de la Communication" (MEC) protocol [START_REF] Joanette | Protocole Montréal d'Evaluation de la Communication[END_REF] were administered to the patients to assess their linguistic prosody capacities (comprehension and repetition). The linguistic prosody comprehension subtest evaluated the ability to identify linguistic intonation patterns. This subtest consists of four semantically neutral simple sentences and each one is presented to the patient with three different intonations, for a total of 12 items. After listening to a sentence the patient is asked to select the correct intonation among the three The patient is asked to repeat each sentence with the same intonation. The maximum score is 12 for both subtests.

In order to exclude patients with a significant hearing impairment from the study, patients were assessed using Lafon's cochlear lists of monosyllabic words (List 2 and List 3) 17 . These were calibrated at an intensity of 40 decibels (dB) and played through headphones. Only patients who scored 80% or more on both lists were included. In addition, the Boston Diagnostic Aphasia Examination (BDAE) command execution subtest 18 was used to exclude patients with comprehension disorders. Only patients with a score of 12/15 or higher were included.

To assess possible mood disorders, the Hospital Anxiety and Depression Scale (HADS) selfquestionnaire [START_REF] Zigmond | The Hospital Anxiety and Depression Scale[END_REF] was administered to patients to assess their current level of anxiety and depression. It contains 7 questions for the anxiety part and 7 questions for the depression part, with a separate score for each. A score of 11 or more for each part indicates a possible anxiety and/or depression state.

To assess auditory attention, we used the sustained auditory attention subtest of the "Logiciel d'Attention en Modalité Auditive" (LAMA) [START_REF] Ambert-Dahan | Capacités attentionnelles auditives et presbyacousie[END_REF] . The assessment and rehabilitation software "Aide Informatisée pour la Rééducation des Troubles Auditifs Centraux" (Airtac2) [START_REF] Tessier | Aide informatisée pour la rééducation des troubles auditifs centraux 2 (Airtac2)[END_REF] was used to assess central auditory processing. Intensity discrimination and duration discrimination of non-verbal sounds were proposed to compare central auditory processing abilities with the results of the Reverse Correlation task. Finally, the Montreal Battery of Evaluation of Amusia (MBEA) [START_REF]Varieties of Musical Disorders : The Montreal Battery of Evaluation of Amusia[END_REF] was selected to assess the music perception abilities of patients. Since the disorder of music perception (Amusia) is primarily a disorder of pitch perception [START_REF] Patel | Speech Intonation Perception Deficites in Musical Tone Deafness (Congenital Amusia)[END_REF] , the three tasks in the melodic organization part were selected.

Procedure

We recorded a 426-ms utterance of the French word "vraiment" ("really"), and generated prosodic variations by dividing it into six segments of 71 ms and randomly manipulating the pitch of each breakpoint independently using a normal distribution (SD = 70 cents; clipped at ± 2.2 SD), hereafter referred to as "stimulus noise". These values were linearly interpolated between time points and fed to an open-source pitch-shifting toolbox developed for this purpose [START_REF] Burred | CLEESE: An open-source audio-transformation toolbox for data-driven experiments in speech and music cognition[END_REF] . We then presented patients with 150 successive pairs (controls: 600 pairs) of such manipulated utterances (really/really?) asking them to judge which, within each pair, sounded most interrogative. The sequence was divided into 3 blocks of 50 pairs for patients and 6 blocks of 100 pairs for controls. Without the participant's knowing, the first and last block of each sequence contained identical pairs of sounds (a procedure called double-pass [START_REF] Neri | How inherently noisy is human sensory processing?[END_REF][START_REF] Burgess | Visual signal detection IV Observer inconsistency[END_REF] , allowing us to examine response variability), but all other sounds in the sequence were otherwise distinct, N=9 patients were additionally tested 4 times, and these data points are not included in the statistics except for figure 3. Sounds were delivered using closed headphones (Beyerdynamics DT770) at a comfortable level (70 dB SPL). The inter-stimulus interval in each pair was 500 ms, and the interval between successive pairs was 1s. The procedure took about 15min to complete for patients and 1hr for controls.

Reverse-correlation analysis

For each participant's response data, we fitted a 2-stage psychophysical model consisting, first, of a prosodic template (or "internal representation") to which sound stimuli are compared and, second, of a level of "internal noise" which controls how consistently this representation is applied to incoming stimuli (Figure 1). Participants' internal representations (a time x pitch representation of an ideally interrogative pitch contour) were computed using the classification image technique 7 : the mean pitch contour of the voices classified as non-interrogative was subtracted from the mean pitch contour of the voices classified as interrogative, and the resulting representation was normalized by dividing it by the sum of their absolute values. For each patient, we then quantified how similar their internal representation is to the average representation in the control group, by computing the mean squared error between the two representations, and used this "representation typicality" as a parameter to correlate with clinical measures. Representations for controls were computed using the same procedure, using only the first 150 trials of each session in order to match the number of trials seen by patients.

Participants' internal noise (expressed in units of the standard deviation of stimulus noise) was inferred from response consistency and response bias across the repeated double-pass trials, using the simulation procedure of Neri (2010). [START_REF] Neri | How inherently noisy is human sensory processing?[END_REF] In short, we computed an idealized participant model responding to repeated stimuli pairs of various sensory evidence, perturbed its response with Both of these analyses (internal representations and internal noise) were conducted using an opensource Python toolbox built for this purpose (https://github.com/neuro-team-femto/palin). See supplementary material (analysis code) for a complete description of the procedure.

Statistical analysis

Group comparisons: because distributions of representation typicality and internal noise scores between patients and controls were non-normal, we compared population means using nonparametric (Mann-Whitney) independent sample t-tests.

Correlation with clinical measures: linear associations between representation typicality and internal noise, and clinical assessments (MEC, Prosody Comprehension, Prosody Repetition, Airtac2) met the homoskedasticity assumption and were therefore estimated using ordinary least-square regressions without robust (HC) norms, as these are considered to increase false positive rates when testing small samples [START_REF] Burgess | Visual signal detection IV Observer inconsistency[END_REF] . In addition, because regression residuals were non-normal, we estimated statistical significance using bootstrapped confidence intervals. The analysis was implemented with the pymer.lm package.

Results

Both measures extracted from the reverse-correlation procedure allowed separating patients from controls: internal representations of interrogative prosody computed from control group responses exhibited a typical final-rise contour 8 , with a marked increase of pitch at the end of the second syllable (Figure 2-left), and control participants were able to apply these representations remarkably consistently across trials, with internal noise values M=0.51 (SD=0. 19) in the range of those typically observed for lower-level auditory and visual tasks 9 (Figure 2-right). In contrast, patients' internal representations had both lower amplitude (indicating less discriminative power) and more variable shape across individuals (Figure 2-left; see also Figure 3), and were applied with higher levels of internal noise (M=2.57, SD=1.93; Figure 2 An oft-quoted limitation of the MEC instrument is its poor sensitivity, with patients above the pathological cut-off on the MEC prosody comprehension scale (9/12) still complaining of communication difficulties [START_REF] Benedetti | Assessment tools and rehabilitation treatments for aprosodia following acquired brain injury: A scoping review[END_REF] . Interestingly, our measures allowed clear separation of this group of Finally, to examine the convergent validity and specificity of internal representation and internal noise measures, we investigated whether they were statistically associated with other constructs linked to central deficits common in stroke rehabilitation. Expectedly, both measures were associated with difficulties discriminating tone intensity and tone duration, as measured by AIRTAC2

(representation: R

Discussion

In this report, we introduced a novel, simple psychophysical procedure which, by combining systematic digital manipulations of speech stimuli and reverse-correlation analysis, allows estimating the internal sensory representations that subtend how individual patients perceive speech prosody, as well as the level of internal noise that govern behavioral variability in how patients apply these representations in prosodic perceptual tasks.

Tested on a sample of N=22 right-hemisphere stroke survivors, our two proposed parameters of representation typicality and internal noise provide a promising alternative to the clinical gold standard for evaluating impairments of prosody processing (MEC). First, internal noise (and, to a lesser extent, internal representations) strongly associate with receptive, and not expressive, aprosodia measured by MEC within the patient group. Second, internal representations (and, to a lesser extent, internal noise) have better sensitivity than MEC for separating high-functioning patients from controls. Finally, both measures appear to have relatively good specificity with respect to nonprosody-related impairments of auditory attention and auditory processing, although internal noise was also found associated with mood disorders which, in our sample, were also predictors of MEC scores.

12 Adl Zarrabi et al. | 2023 By separating these different profiles of pathology, it is our hope that the representation+noise model will provide more effective and individualized therapeutic targets for rehabilitation of individuals with impaired speech prosody perception than existing measures. For example, patients with the highest levels of internal noise may benefit from therapies that focus on attentional and executive skills. In larger samples of patients, it will be also interesting to characterize the separate physiopathological determinants of abnormalities in either representations or noise, for instance in terms of different lesion locations or sizes. Finally, as the procedure is easy to dispense remotely and can be optimized to even shorter durations than the relatively light (~15min) procedure used here [START_REF] Compton | Stimulus whitening improves the efficiency of reverse correlation[END_REF] , we are also interested in the potential prognosis value of measuring changing levels of representation typicality and noise longitudinally, along the course of rehabilitation.

3 Adl

 3 N=22 brain stroke survivors (male:17; M=57 yo, SD=13.0), and N=12 age-matched controls (male: 6;M=59 yo, SD=17.6) took part in the study. All participants were in-or out-patients of the Physical Medicine & Rehabilitation Department, APHP Pitié-Salpêtrière Hospital in Paris, France, undergoing speech therapy for different deficits post-stroke like swallowing difficulties, neurovisual . Zarrabi et al. | 2023

7 Adl

 7 Zarrabi et al. | 2023

Figure 1 .

 1 Figure 1. The representation + noise model. Patients were presented with 150 successive trials consisting of pairs of manipulated prosodies (A) and asked to judge, within each pair, which sounded most interrogative (B). Patient responses in each trial were fitted with a 2-stage psychophysical model (C), consisting, first, of a prosodic template (or "internal representation") to which sound stimuli are compared and, second, of a level of "internal noise" which controls how consistently this representation is applied to incoming stimuli. See main text for details about the model-fitting procedure. In this work, we estimate the two model parameters (representation and noise) for each patient individually and compare them with patient records to test their value as markers of receptive aprosodia.

10 Adl

 10 MEC-negative patients (N=12/22) and controls (N=12), both in terms of typicality of representation (M=-0.18 [-0.32 --0.06], U(-0.68)=23.0, p=.005) and internal noise (M = 1.71 [0.71, 2.83], U(-0.7)=71,p=0.014). Zarrabi et al. | 2023

Figure 2 .

 2 Figure 2. Patient parameters (internal representations and internal noise) estimated by reverse-correlation separate controls from patients above and below the pathological cut-off on the MEC prosody comprehension scale (9/12). Left: Internal representations of interrogative prosody computed from control group responses exhibited atypical final-rise contour, with a marked increase of pitch at the end of the second syllable. In contrast, patients' internal representations had both lower amplitude and more variable shape across individuals. The bottom waveform illustrates the shape of the base sound used to generate stimuli (a male-recording of the word vraiment/really). Right: control participants were able to apply these representations remarkably consistently across trials, with internal noise values < 1 standard deviations of stimulus noise. In contrast, patients' internal noise level were larger and more variable, and scaled with prosodic difficulties measured by MEC.

Figure 3 . 13 Adl

 313 Figure 3. The representation+noise model captures a rich diversity of sensory/cognitive mechanisms underlying impairments of prosody processing after stroke. Center: Distribution of representation typicality and internal noise for controls and patients (considering all 4 sessions), overlaid with by kernel density estimate. Histograms on the marginal axes show univariate distributions for each variable in the patient group. Corners: Corner boxes show internal representations (top) and behavioral series of responses (bottom) for 4 illustrative patients. Patients in top corners have internal representations (blue) that are similar to controls (orange), but vary in amounts of internal noise (e.g. showingexcessive response perseveration; top-right). Patients in bottom corners have atypical representations (blue), but some nevertheless retain healthy levels of internal noise (e.g., being normally consistent in wrongly expecting question phrases to decrease rather than increase in pitch; bottom-left). The estimation of internal noise was limited to the range [0; + 5std]; data points in the upper side of that range may either correspond to true internal noise values, or to larger values for which we could not provide an exact estimate, as illustrated here with a dotted line in the central panel (see Appendix for details).

Table 1 : Patients and control demographics and clinical characteristics

 1 . N=22 right-hemisphere stroke survivors and N=12 age-matched controls took part in the study. Abbreviations used: MEC: Montréal Evaluation de la Communication; BDAE: Boston Diagnostic Aphasia Examination

NIH stroke scale (NIHSS), median [Q1,Q3]

			controls	patients
	n		7	22
	Sex, n (%)	f	3 (42.9%)	5 (22.7%)
		m	4 (57.1%)	17 (77.3%)
	Age, median [Q1,Q3]		59.0 yo [52.5,71.5]	60.5 yo [52.2,63.0]
	Month after stroke, median [Q1,Q3]			4.0 mo [1.0,5.0]
	Stroke type, n (%)	HEM		3 (33.3%)
		ISCH		6 (66.7%)
				10.0 [5.5,16.0]

MEC Prosody Comprehension item, median [Q1,Q3]

	9.0 [8.0,11.0]

MEC Prosody Repetition item, median [Q1,Q3]

		11.0 [10.0,12.0]
	MEC Total, median [Q1,Q3]	21.0 [18.2,22.8]

BDAE command execution item, median [Q1,Q3] 14

	.0 [14.0,15.0]

Audiogram left-ear, median dBHL at 1000Hz [Q1,Q3]

	0.0 dBHL [0.0,15.0] 20.0 dBHL [15.0,30.0]

Audiogram right-ear, median dBHL at 1000Hz [Q1,Q3]

	5.0 dBHL [0.0,20.0] 15.0 dbHL [7.5,37.5]

Vocal audiogram, median% detection at 40dB [Q1,Q3]

	different written options (interrogative, imperative or affirmative). The linguistic prosody repetition
	subtest examines the ability to verbally reproduce linguistic intonations. It is formed of the same four
	sentences as the comprehension task. The previously recorded stimuli are presented in random order.
	99.0 % [94.0,100.0]

  ("internal noise"), and estimated the probability for that model to give the same response for identical trials (i.e. response consistency) and the probability of giving the first response option (i.e. response bias), for different standard deviations of that internal noise. For each

	participant, we then inverted that model and obtained the value of internal noise (by exhaustive
	search between 0 and +5 std) that minimized the error between the observed and predicted values
	for that participant's consistency and bias. As in previous studies 9 , we estimated internal noise
	conservatively between [0; +5 std] in order to avoid unreliable estimates at large values, a known
	problem with double-pass procedures (see Appendix A). Internal noise values in the upper side of
	that range (e.g. illustrated in Figure 3 between 4.8 and 5) may either correspond to true internal
	noise values, or to larger values for which we could not provide an exact estimate.

8 Adl Zarrabi et al. | 2023 additive gaussian noise

  -right). The two groups differed statistically for both

	representation typicality: M=-0.28 [-0.39;-0.17], Mann-Whitley's U(0.80)=25, p<0.000; and internal
	noise: M=2.06 [1.24; 2.86], U(-0.74)=134.00, p=0.004
	0.19, β=-0.31 [-0.62; -0.08], t(20)=-2.22, p=.038. Representation typicality also improved with better
	scores, albeit non-statistically (R 2 = 0.11, β=+0.03 [-0.003; +0.07], t(20)=1.57, p=.13). Second, both
	measures had also good symptom specificity, as strikingly neither correlated with the MEC score for
	prosody repetition (representation: R 2 =0.047, t(20)=-0.98, p=.34), while both MEC scores were
	themselves positively correlated (R 2 : 0.29, β=+0.32 [+0.08 -+0.56], t(20)=2.85, p=.01).

Within the patient group, internal noise values (and, to a lower extent, representation typicality) were statistically associated with scores of the current gold standard for assessing deficits of prosody perception (MEC), demonstrating good concurrent validity. First, larger internal noise values were associated with lower (more severe) scores on the MEC prosody comprehension scale: noise: R 2 =

Funding

This work was supported by grant from Fondation pour l'Audition (FPA RD 2021-12).

Data Availability

The data that support the findings of this study are available from the corresponding author, MV, upon reasonable request.

Ethics statement

The study was approved by Comité de Protection des Personnes CPP Ile-De-France V (ProsAVC, Decision of 22/07/2020). All participants provided informed consent.