Wijdane Kaiss
email: wijdane.kaiss@univ-ubs.fr

Khalifa Mansouri
email: khmansouri@hotmail.com

Franck Poirier
email: franck.poirier@univ-ubs.fr

Chatbot design to help learners self-regulte their learning in online learning environments

Keywords: self-regulated learning, e-learning, chatbot, moodle

E-learning environments are part of the selflearning ecosystem of which self-regulation is one of the main pillars. One of the key factors contributing to dropping out in an e-learning environment is the lack of self-regulatory skills. This skill is often absent in many learners, leading them to abandon their training. In this paper we propose a methodology, chatbot architectural design, allowing to help the learner to self-regulate in his/her learning by supporting him/her through a chatbot within the Moodle platform, which constitutes a metacognitive virtual assistant, to provide metacognitive messages. The aim is to improve the quality of learning in the e-learning environment.

I. INTRODUCTION E-learning environments require learners to be able to selfregulate their learning. The three pillars of self-learning are: Self-Regulated Learning (SRL), which involves the ability to set goals, plan and monitor progress, regulate emotions, and adapt strategies to optimize learning ; Metacognition, which involves the ability to reflect on one's own thinking and learning processes, to evaluate one's own strengths and weaknesses, and to identify and address areas for improvement ; and Motivation, which involves the drive and willingness to engage in learning activities, to persevere in the face of challenges, and to take ownership of one's own learning.

Self-Regulated Learning (SRL) is one of the three pillars of Self-Learning and it is demonstrated that, when mastering SRL skills, learners become more engaged in their learning and achieve stronger gains in learning [START_REF] Schunk | Motivation and self-regulated learning : theory, research, and applications[END_REF]. These skills must therefore be developed and promoted during learning [START_REF] Zimmerman | Des apprenants autonomes: Autorégulation des apprentissages[END_REF]. Its absence or weakness is seen as a factor that can lead to abandonment [START_REF] Khalil | MOOCs Completion Rates and Possible Methods to Improve Retention -A Literature Review[END_REF], [START_REF] Nawrot | Building engagement for MOOC students: Introducing support for time management on online learning platforms[END_REF].

The SRL skills refer to the learners' ability to define specific goals for their work, plan strategies to achieve those goals, and monitor and adapt those strategies as they progress.

Indeed, the teacher is not able to provide, in real time, help to learners to use self-regulation strategies, either because of the number of learners, as each demand requires an appropriate time for reflection and decision, or because of his availability. To solve this problem, we propose, in this paper, a design architecture of the chatbot that will be integrated in the Moodle platform to support learners to self-regulate during their learning process, to improve their learning and to stay motivated.

A chatbot is a new form of automated contextual communication between users and machines or systems, which exploits a conversational approach based on natural language. It refers to a software system, also known as a conversational agent, as it takes turns interacting with the user, through written messages [START_REF] Valtolina | Chatbots and Conversational Interfaces: Three Domains of Use[END_REF]. It is a technology that simulates human conversation by providing feedback and can interact intelligently through machine learning and artificial intelligence [START_REF] Parthornratt | A Smart Home Automation Via Facebook Chatbot and Raspberry Pi[END_REF].

The rest of the paper is organized as follows. The section 2 reviews our related works. The section 3 presents in detail our proposed chatbot in terms of architecture design. The chatbot service processes are shown in the section 4 and the section 5 concludes this work.

II. RELATED WORKS

A. Self-Regulated Learning (SRL)

Self-regulation generally refers to "self-generated thoughts, feelings, and actions that are planned and cyclically adapted to the attainment of personal goals" [START_REF] Zimmerman | Attaining self-regulation: A social cognitive perspective[END_REF]. Self-regulated learners "plan, set goals, organize, self-monitor, and self-evaluate at various points during the process of acquisition" [START_REF] Zimmerman | Self-Regulated Learning and Academic Achievement: An Overview[END_REF]. In [START_REF] Pintrich | Motivational and self-regulated learning components of classroom academic performance[END_REF], reviewed a variety of definitions of self-regulated learning and identified three common components of self-regulated learning:

(1) metacognitive strategies (e.g., planning, monitoring, and modifying cognition), (2) task management and control, and (3) actual cognitive strategies to learn the material. SRL can be understood as a learner's ability to self-monitor and adjust their learning strategies or behaviors in response to the current learning context [START_REF] Littlejohn | Learning in MOOCs: Motivations and self-regulated learning in MOOCs[END_REF]. Numerous studies have investigated the models, frameworks and fundamental phases of SRL [START_REF] Kirschenmann | Demands of modern PLEs and the ROLE approach[END_REF], [START_REF] Renzel | Tracing selfregulated learning in responsive open learning environment[END_REF]. On this basis, educators have made various attempts to help learners develop SRL skills and apply these strategies to their learning. However, studies have shown that students struggle with self-regulation in online learning [START_REF] Capuano | Learning Goals Recommendation for Self-Regulated Learning[END_REF]- [START_REF] Manso-Vazquez | Proposal of a learning organization tool with support for metacognition[END_REF].

It should also be noted that self-regulated learning support was more effective in the online learning condition than in the face-to-face setting. The researchers explained this with the nature of online learning, which requires the learner to monitor their own learning and demands metacognitive strategies more than face-to-face environments [START_REF] Kramarski | Investigating Preservice Teachers' Professional Growth in Self-Regulated Learning Environments[END_REF]. Providing metacognitive support in the form of feedback improved students' selfregulation in an online learning environment. The feedback helped students reflect on their learning processes and set goals for improvement. Also, providing opportunities for selfassessment and reflection was effective in supporting SRL in an online learning environment. Students who engaged in selfassessment and reflection had better learning outcomes and higher levels of SRL compared to those who did not. These examples reveal the potential of self-regulated learning support to increase students' self-regulated learning skills and learning performance.

The model of [START_REF] Zimmerman | Motivational Sources and Outcomes of Self-Regulated Learning and Performance[END_REF] was the most frequently accepted and we will use it, who postulated that SRL occurs in three cyclical phases: forethought (such as goal setting, planning), performance (such as time management, help seeking), and selfreflection (such as self-evaluation) [START_REF] Zimmerman | Attaining self-regulation: A social cognitive perspective[END_REF], [START_REF] Zimmerman | Motivational Sources and Outcomes of Self-Regulated Learning and Performance[END_REF]. Each phase of selfregulated learning consists of processes that self-regulated learners engage in, and each phase influences the processes of the next phase.

B. Supporting self-regulation through chatbots

Chatbots are machine agents that serve as natural language user interfaces to provide data and services [START_REF] Dale | The return of the chatbots[END_REF]. In recent years this technology has been employed for multiple purposes mainly in messaging applications. Chatbots can also serve various purposes such as customer care, emotional or social support, information providers, entertainment, etc [START_REF] Brandtzaeg | Why People Use Chatbots[END_REF].

Chatbot promises to solve a variety of problems in education today. One of the biggest advantages of chatbot is that it can support students individually and intently [START_REF] Winkler | Unleashing the Potential of Chatbots in Education: A State-Of-The-Art Analysis[END_REF]. Other research has confirmed that chatbots can be virtual companions for users intended to resolve availability issues, provide support and customer assistance [START_REF] Wu | Advantages and constraints of a hybrid model k-12 e-learning assistant chatbot[END_REF], [START_REF] Yang | Opportunities and Challenges in Using AI Chatbots in Higher Education[END_REF] powered by artificial intelligence.

In the higher-education, a chatbot can be trained from a wide variety of resources ranging from learning experiences to learning materials. According to [START_REF] Hubert | 6 Ways Artificial Intelligence and Chatbots Are Changing Education[END_REF], chatbots are trained to answer common questions about the study of a subject. This motivates learning supports more quickly and conveniently.

In [START_REF] Oliveira | IBM Watson Application as FAQ Assistant about Moodle[END_REF], the use of a chatbot in an educational context for the automation of higher education student care is presented. The chatbot interacts with students through text messages on topics in a closed context answering doubts about the "Higher Institution Course". However, to our knowledge, there are no chatbots that have been used in the educational context to support self-regulated learning strategies on the Moodle platform. In this subsection, we have discussed some scientific contributions on chatbots in the field of education.

III. PROPOSED CHATBOT ARCHITECTURAL DESIGN

Figure 1 shows the proposed design of our chatbot to support self-regulated learning strategies on the Moodle platform, inclusing the two main components (Learner message analysis and Response generator).

A. Learner Message Analysis

The Learner message analysis component uses natural language processing techniques to comprehend what learners are talking about. Specifically, it consists of two main tasks:

1) Intent matching: the aim of this component is to determine the intent of learners through their messages. It can be considered as the goal of the messages. The learner intent identification is studied from the perspective of the text classification. Correspondingly, learner text messages are classified into those intents: Engage in learning (which will allow the learner to establish his/her goal to start learning), Make appointment (to plan and manage his/her time in order to reach his/her defined goal), the intentions Exercise (to do an exercise on a part of the course chosen by the learner), My tasks (to see the tasks he/she has defined), My progress, Evaluation, which allows the learner to self-evaluate and self-monitoring (allow the learner to control and track his/her progress towards his/her goal), and ask for help via FAQ intent.

2) Entity Recognition: once the intent of the learner is determined, a specific set of context information is extracted from the message. Chatbots use this information to capture the current situation of the learner and generate the response accordingly. In the field of natural language processing, the researchers performed this work very effectively, known as Named Entity Recognition (NER) system [START_REF] Weiss | Text mining: Predictive methods for analyzing unstructured information[END_REF]. In essence, it is a model of classifying a word block in a message into a given entity (label). The main entities, the context information, that is usually extracted for most leaner intents are the domain, specialty, course, concept, tasks.

For example, when a learner message is "please, I want to know my progress in the pointers part of the C programming course.", the chatbot will automatically categorize the message to the intent of My progress. For this intent, two context information about the concept (pointers) and the course (C programming) is automatically determined.

In this paper, all the natural language processing operations mentioned above are implemented by Dialogflow. Dialogflow is a Google technology that supports human-computer interaction based on conversations in natural language. Unfortunately, Google has not published which algorithms are used in the Dialogflow. However, according to the Dialogflow documentation, the selected algorithms are most effective with the user message to the training phase.

B. Response Generator

Based on the intent and context information returned from the Learner message analysis component, the Response generation component constructs responses for the Learner. Three models : the Pattern-based model [START_REF] Joshi | SarcasmBot: An open-source sarcasm-generation module for chatbots[END_REF], Retrieval-based model [START_REF] Wu | Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots[END_REF], and Generative model [START_REF] Patidar | Correcting Linguistic Training Bias in an FAQ-bot using LSTM-VAE[END_REF], are utilized to generate the appropriate responses. For the Pattern-based model, chatbots match learner messages with each underlying question-answer pattern to create a response. Compared to the Pattern-based model, the Retrieval-based model offers much flexibility. Specifically, this model queries and analyzes available resources using APIs and databases. The Generative model is the smartest among the three models in terms of generating answers based on current user messages and previous messages. However, it encounters many difficulties in building and training. This means that it needs training with a very large set of data in order to achieve a good conversation. Because of this disadvantage, this model still has not been much used to build a chatbot in reality yet. In the proposed approach, the retrieval-based model will be used in our chatbot like most of the current chatbots in order to provide more flexibilities in our services.

Once the intent is detected by Dialogflow, an action to perform is chosen, the Webhook sends a formatted response corresponding to the intent. All actions will be coded in Node.js and hosted by the Webhook Fulfillment. The latter is a service that allows a dynamic response by searching for response elements in an external database. The database we will use is Mongodb which will contain data about the learner, to keep track of him/her, and about the moodle resources. And there is also the local database that will contain the answers recorded in Dialogflow. So, we will use external API google calendar to record the tasks (by date and time) that the learner has defined.

IV. CHATBOT SERVICE PROCESSES

In this section, we will present service processes for handling learner messages to help them self-regulate. Fig. 2 shows the process of the 'Engage in Learning' service, which helps the learner to set a goal by asking a set of questions. It responds to the SRL "Goal setting" strategy of the Fourthough phase. After the learner chooses to engage in a learning activity, the chatbot determines the intent (in this case, it will be Engage in learning intent) and recognizes the entities (e.g. Domain entity, Course entity, ... those defined in the design of the chatbot Fig. 1), in order to generate the set of questions from Dialogflow (will be questions, e.g. in which domain he/she wants to learn, which specialty, by which concept he/she wants to start, ...), so that the learner can clarify his/her goal and start learning. Fig. 3 shows the process of the 'make appointment' service, which helps the learner to plan the course parts in order to manage his/her time. It meets the SRL strategies "Planning" and "Time Management". After the learner chooses this service to organize his/her learning in order to reach his/her goal, the chatbot repeats the first phases (classification of the intention and recognition of the entities), to generate answers containing questions, for example, about the name of the task, the time to perform the task (date and time). After it confirms what it has defined, a check will be made to know if the chosen time or date is available, otherwise the chatbot will ask the learner to change. If it is available, the chatbot will record it in the learner's calendar, using the Calendar API, and keeps track of this in the MongoDB database. One of the self-regulation learning strategies is selfevaluation. The chatbot will help the learner to evaluate him or herself, by generating a test following a set of questions, generated from Dialogflow responses (see Fig. 4), which will determine the module on which he or she wants to pass the test. The latter will be generated from the moodle resource database using weebhook. At the end, the chatbot will generate a feedback, which will be personalized, on the parts that he/she has not answered correctly with corrections. This test and feedback will allow the learner to self-evaluate and self-monitor, since the learner will know the learning progress and its advancement to ensure the achievement of the objectives. In Fig. 5 shows the process of the 'FAQ' service. Instead students going to the website and long faq to ask for help (help seeking), they can simply ask a question to the chatbot and they can get an answer, by integrating data faq into Knowledge Base (KB). A Knowledge Base was populated with pertinent information regarding the topic at hand, to satisfy the requests for definitions and examples. Afterward, the chatbot generate an answer to satisfy the learner request from KB. The answer is sent back to the learner.

V. CONCLUSION Self-regulated learning has become a popular topic in the online learning research field. Self-regulation is not an inherent feature of human but the self-directive process. Despite the proliferation of participating in online courses and a large number of learning resources around the world, scant knowledge exists about how to support and scaffold self-regulated learners in online learning environments and how to guide instructors to lead to their successful online courses. The results of this study show the possibilities of using a conversational AI agent when scaffolding learners' self-regulated learning in online courses via our conceptual proposal. Future work will focus on the implementation and use of the chatbot, integrated into the Moodle platform, by learners who will participate in the experiment that we will do to validate our conceptual proposition.

Fig. 1 .

 1 Fig. 1. The proposed chatbot design.

Fig. 2 .

 2 Fig. 2. The service process « Engage in Learning».

Fig. 3 .

 3 Fig. 3. The service process « make appointment».

Fig. 4 .

 4 Fig. 4. The service process « self-evaluation ».

Fig. 5 .

 5 Fig. 5. The service process « FAQ».

ACKNOWLEDGMENT

The authors would like to express their gratitude to the editors and referees committee of the IEEE International Conference on Advanced Learning Technologies for their constructive comments and valuable feedback on this work.


Jérémy Boes
email: boes@irit.fr

Franc ¸ois Gatto Frédéric Migeon
email: migeon@irit.fr

Self-Organizing Agents for an Adaptive Control of Heat Engines

Keywords: Multi-Agent Systems, Self-Organization, Intelligent Control Systems

Controlling heat engines imposes to deal with high dynamics, non-linearity and multiple interdependencies. A way handle these difficulties is enable the controller to learn how the engine behaves, hence avoiding the costly use of an explicit model of the process. Adaptive Multi-Agent Systems (AMAS) are able to learn and to adapt themselves to their environment thanks to the cooperative self-organization of their agents. A change in the organization of the agents results in a change of the emergent function. Thus we assume that AMAS are a good alternative for complex systems control, reuniting learning, adaptivity, robustness and genericity. In this paper, we present an AMAS for the control of heat engines and show several results.

INTRODUCTION

Over the years, the complexity of car engines has been growing up, with new sophistications like Exhaust Gas Recirculation or particle filters. Moreover our expectations about their efficiency are becoming more important due to the strengthening of environmental norms, while the need for economic competitivity leads to a need for easily instantiable controllers. The emergence of software and electronic devices helps to tackle these challenges but the development of Engine Control Unit (ECU) remains a difficult and time consuming task.

The goal of this paper is to introduce a nonclassical approach of the control of car engines, based on Adaptive Multi-Agent Systems (AMAS), that aims to provide a quickly instantiable controller, avoiding a heavy parametrization work. Based on learning and adaptiveness skills of AMASs, our approach considers the engine and its ECU as a black box. The controller modifies ECU parameters in order to obtain the desired values on the engine outputs. This work is part of a national project named Orianne.

Our motivation for working on a non-classical approach comes from the difficulties to build and fit an efficient model of the engine we intend to control. Humans fulfill this control task in their everyday life. A child learning to drive a bike is a good example. He has the ability to move his legs in order to walk and run. He does not have any knowledge about bike gears or inertia theory. While he is experiencing his first ride, his most used skill is the ability to adapt and to coordinate his movements in a new situation. This adaptation requires learning, which is certainly not an instantaneous process. Following a practice period, the child eventually ends up acquiring the ability to control the bike but still did not get in touch with any theoretical concept and did not build any analytical model of the bike inside his head. Furthermore, he never stops learning: each new ride will improve his mastering of the bike, and he will be able to adapt to a new bike. This analogy is meant to explain the point of view we adopted: the use of an explicit analytical model is not necessary to the control of an engine.

Section 2 introduces usual control methods. We follow with an introduction to self-organization and to the Adaptive Multi-Agent Systems (AMAS) theory. Section 3 describes the design of a Multi-Agent System before going deeper in the agents behavior in section 4. Then section 5 shows some results, before section 6 concludes with our perspectives.

RELATED WORKS

In this section the main approaches of control are presented before self-organization and the Adaptive Multi-Agent Systems theory are briefly introduced.

Complex Systems Control

Controlling systems is a generic problem that can be expressed as finding which modifications are needed to be applied on the inputs in order to obtain the desired effects on the ouputs. The main approaches are presented in the next paragraphs.

PID

Proportional-Integral-Derivative (PID) controllers are well-known and widely used. They base the control on the calculation and the combination of three terms, corresponding to the error (P), its duration (I) and its variation rate (D) [START_REF] Astrom | PID Controllers: Theory, Design, and Tuning[END_REF]. The instantiation of a PID controller to a particular system is a heavy procedure despite adjustement methods like [START_REF] Tyreus | Tuning PI controllers for integrator/dead time processes[END_REF]. Besides, PID controllers are not always efficient with non-linear systems, they need to be complemented with other mechanisms like fuzzy logic [START_REF] Visioli | Tuning of PID controllers with fuzzy logic[END_REF]. Also, a PID controller can only handle one input with one retroaction signal, which is a severe drawback for complex systems control.

Adaptive Control

The dynamic nature of complex systems imposes to adapt the controller as the system evolves. Modelbased approaches like Model Identification Adaptive Control (MIAC) [START_REF] Soderstrom | System identification[END_REF] or Model Reference Adaptive Control (MRAC) (Kreisselmeier and Anderson, 1986) are regrouped under the name of Model Predictive Control (MPC) [START_REF] Nikolaou | Model predictive controllers: A critical synthesis of theory and industrial needs[END_REF]. A model able to forecast the behavior of the process is used in order to help the controller finding the optimal control scheme. These approaches successfully handle several inputs of dynamic nonlinear systems. They are used in particular in the domain of car engines [START_REF] Dabo | NCGPC with dynamic extension applied to a Turbocharged Diesel Engine[END_REF] but are limited by the construction of the model on one hand, and its parametrization on the other hand, which can take several years.

Intelligent Control

Intelligent control can be seen as a part of adaptive control. It regroups control approaches that use Artificial Intelligence methods to enhance the controller adaptiveness. Among these methods we can find neural networks [START_REF] Hagan | An introduction to the use of neural networks in control systems[END_REF], fuzzy logic [START_REF] Lee | Fuzzy logic in control systems: Fuzzy logic controller[END_REF], expert systems [START_REF] Stengel | Intelligent failure-tolerant control[END_REF] and bayesian controllers (Colosimo and Del Castillo, 2007). Neural networks provide great genericity but they require an offline learning step with a considerable amount of data. Even if this enable an efficient control, they remain sensible to perturbations and cannot adjust themselves to the evolution of the system.

Expert systems are composed of a knowledge base and an inference engine. They lack of genericity and are hardly adaptable to highly dynamic nonlinear systems. That is why they are often combined with fuzzy logic. Bayesian controllers are based on bayesian probabilities, the most common are the Kaman filter based controllers [START_REF] Lee | Extended kalman filter based nonlinear model predictive control[END_REF]). Yet, the bayesian approach does not deal with the problem of control itself but with a sub-problem which is the observation of the process. These methods can also be easily combined one with another, or with more classical control approaches.

Finally agents-based techniques have been applied in various fields, for example in the case of voltage control [START_REF] Wang | Multi-agent co-ordination for the secondary voltage control in power-system contingencies. Generation, Transmission and Distribution[END_REF] or manufacturing control [START_REF] Clair | Self-regulation in self-organising multi-agent systems for adaptive and intelligent manufacturing control[END_REF].

Self-Organizing Software

The next paragraphs introduce the notion of selforganization along with the Adaptive Multi-Agent Systems theory.

Self-Organization

The notion of self-organization refers to the fact that the organization of a system results from internal mechanisms and local interactions without any explicit external control [START_REF] Camazine | Self-Organization in Biological Systems[END_REF]. Various artificial systems show this property, such as swarms [START_REF] Montresor | Messor: Load-balancing through a swarm of autonomous agents[END_REF] or some multiagent systems [START_REF] O'hare | Foundations of distributed artificial intelligence[END_REF]. Selforganization brings robustness and scalability, and allows the engineering of underspecified software (Di Marzo Serugendo et al., 2011).

Adaptive Multi-Agent Systems

The Adaptive Multi-Agent Systems (AMAS) theory is a basis for the design of multi-agent systems where cooperation is the engine for self-organization [START_REF] Georgé | Cooperation[END_REF]. As cooperative entities, AMAS agents try to reach their own goals as well as they try to help other agents to achieve their goals. Moreover, an agent will modify its behavior if it thinks that its actions are useless or detrimental to its environment. Such situations are called Non-Cooperative Situations (NCS). Some behavioral rules are specific to NCSs, they allow agents to avoid or to solve these situations. By solving NCSs in regard to their own local goals, cooperative agents collectively find a solution to the global problem. Therefore one can consider the global behavior of an AMAS as emergent.

AMAS principles have been applied by [START_REF] Videau | Controlling Bioprocesses using Cooperative Self-organizing Agents[END_REF] for the control of bioprocesses, avoid- ing the use of a specific model and dealing with the evolution of the process over time, since an AMAS is perpetually adjusting itself to its environment.

The next two sections describes the agents of our controller, how they perform control and how they self-organize.

CONTROLLER OVERVIEW

In this section we present the required abilities of a complex system controller, and what are the agents of our controller that enable them.

Basic Behavior

The next paragraphs describe how our multi-agent system, called ESCHER (Emergent Self-adaptive Controller for Heat Engine Regulation), works when it is already adapted to the engine. The mechanisms that lead to this adaptation will be explained further.

Observing the Process

If we intend to control an engine, it is obvious that we need to be able to observe it. A specific agent type is in charge of perception, called Variable Agent. Each input and each output of the controlled black box has a corresponding Variable Agent. These agents perceive their value from the black box and send it to other agents that need this information. Also, Variable Agents can embed noise reduction algorithms if this problem is not handled by a third party system.

Representing Objectives and Constraints

The controller needs to know what is the desired state of the engine. This state is represented by a set of Constraint Agents and possibly by additional Variable Agents.

There are three types of Constraint Agents: Threshold, Setpoint and Optimization. A Threshold Constraint Agent expresses the will to keep a variable either below or above a threshold specified by a Variable Agent. This Variable Agent can either correspond to a process variable or to a user-defined value. In a similar way, a Setpoint Constraint Agent expresses the will to set a process variable to a particular value specified by a Variable Agent. Finally, an Optimization Constraint Agent represents the will to minimize or maximize a process variable.

Using the functions shown in figure 1, each Constraint Agent computes a critical level that varies from 0 (the agent is satisfied) to 100 (the agent is far from satisfied). The critical level depends on the value of the variable on which the constraint is applied and can be parametrized by a second Variable (in the case of a threshold or a setpoint constraint). The criticality function of an Optimization Constraint Agent is asymptotic to zero, since we always try to minimize or maximize the value. For instance, if the user desires to set the rotational speed to 5000rmp, the Variable Agent RotSpeedCommand (corresponding to "userdefined setpoint on rotational speed") will be set to 5000 and a setpoint Constraint Agent will be created. Its critical level will be zero if the value of the Variable Agent RotSpeed equals the value of RotSpeed-Command, and will increase with the difference between RotSpeed and RotSpeedCommand.

It is clear that decreasing the critical levels means solving the constraints, and the only way to do so is to perform the adequate actions on the engine inputs. Finding these actions implies to be able to analyze the current sate of the environment. The engine and the user desires are considered as the environment of the multi-agent system.

Analyzing the State of the Environment

As shown in Figure 2, ESCHER environment is the engine as well as the user-defined setpoints, thresholds and optimizations. Thanks to Variable Agents and Constraints Agents, the ESCHER has a representation of its environment. Before it can perform control, it must be able to extract relevant information from this representation. This is the role of agents called Context Agents.

A Context Agent memorizes the effects, on every critical level, of an action applied to one partic-ular input of the engine. It also memorizes the state of the environment when the action was applied. To represent this state the Context Agent maintains a set of validity ranges, containing one range per Variable Agent. A Context Agent also maintains a set of values that are the observed and memorized effects of the action on each of the critical levels. This set is called the forecasts. In other words, a Context Agent represents the information that if every Variable Agent value is inside the Context Agent validity range, and if its action is applied, then the effects on every critical level will be similar to the Context Agent forecasts.

A Context Agent is said valid when the environment is in a state that matches its validity ranges, i.e. every perceived variable is inside the corresponding validity.

When valid, a Context Agent sends a notification with its action and forecasts to the appropriate Controller Agent, which will be explained in the next part.

Selecting the Adequate Action

Each controlled input of the engine is associated with a Controller Agent. The role of a Controller Agent is to apply the most adequate action in order to reduce the critical levels. It will base the choice of the action (ie increment, decrement or stay put) on the information it receives from Context Agents, picking the action that will provoke the biggest decrease of the critical levels. When an action is picked, the Controller Agent notifies every Context Agents who proposed it.

For instance, the Controller Agent of the mass fuel injected may receive two notifications from two Context Agents. Context Agent A says if you apply a decrease of 0.5 the critical levels will not change except for the Constraint Agent Torque Optimization that will rise of 3.5. Context Agent B says if you apply an increase of 0.5 the critical levels will not change except for the Constraint Agent Torque Optimization that will drop of 1.5. In this case, the Controller Agent obviously chooses to increase the mass fuel injected.

There are several cases where the Controller Agent is unable to make a good decision, because of incomplete or incorrect information from Context Agents. These cases are Non-Cooperative Situations (NCS). They happen when ESCHER is not well adapted to the engine yet. When a NCS occurs, the cooperative behavior of involved agents is triggered in order to solve it. This will be explained in section 4. For now, we continue with the architecture of the system.

Topology

The four ESCHER's agent types were introduced in the previous paragraphs. Figure 3 shows the topology of their interactions. We can see that, while it is a MAS itself, ESCHER is also an aggregation of several MASs, each constituted by a Controller Agent and its set of Context Agents. The environment of these sub-MASs is composed of the Variable Agents, the Constraint Agents, and the environment of ESCHER. Indeed, each Controller Agent is not directly in interaction with other Controller Agents. The only link between them are their environment (the engine they are controlling, via the Variable and Constraint Agents). Actions from one Controller Agent will have consequences on the engine that will be reflected by Variable and Constraint Agents, and Controller Agents share the same environment, hence we can expect them to progressively synchronize their actions. Furthermore, Context Agents never directly interact with each other (even inside the same set) but only with their associated Controller Agent.

The question is now how can the system create and adjust Context Agents in order to deal with an unknown process? The next section gives an answer.

CONTEXT AGENTS LEARNING AND ORGANIZATION

This section explains how the set of Contexts Agents is built and how they self-organize to provide an accurate representation of the explored part of the process' state space.

Non-Cooperative Situations

In this part, we explain how agents solve the Non-Cooperative Situations they face. Since they provoke changes in the agents behavior, NCSs are the key of the adaptiveness of Adaptive Multi-Agent Systems. Each agent locally solves its own NCSs when they occur, by changing its own local behavior, and thus of the global behavior of the system. The NCSs of ESCHER concern the Controller Agents and the Context Agents, and trigger behaviors that correct Context Agents' forecasts and validity ranges.

No Adequate Action in Suggestions

This NCS occurs when the suggestions list of a Controller Agent contains only forecasts of increasing critical levels. Whatever the Controller Agent may chose, it believes it will deteriorate the constraints. There are two cases: either all the possible actions are already suggested (i.e. increase the value, decreasethe value and stay put) or some actions are not suggested. In the first case, the only choice left is to accept the suggestion with the less bad forecasts. In the second case, a new action is chosen among those which are not suggested, and a new Context is created with this action.

Empty Suggestions List

This NCS happens when a Controller Agent has to apply an action, but finds its suggestion list empty. It will be unable to find an adequate action with certainty, but it can make some hypothesis to try one. If the last applied action had reduced the higher critical level, the same action is reproduced. If not, the opposite action is applied (e.g. if the last action was an inscrease of the input value, the new action will be a decrease). If no action has ever been applied, the solution is to randomly chose the action. In any case a new Context Agent is created with the applied action. After its creation, a Context Agent extends its validity ranges as long as its action is applied and observe the variations of the critical levels to set its forecasts.

Wrong Forecast

When a Context Agent is selected and its action applied, it oberves the variations of the critical levels to check if they matches its forecasts. If the forecasts are wrong (i.e. a critical level evolves in the opposite direction of the forecast), the Context Agent considers that it should not have been valid when the action was suggested. Thus it reduces its validity ranges but does not modify its forecasts. A Context Agent dies if one of its ranges is reduced to an amplitude of zero. If the forecasts are simply inexact (i.e all the critical levels evolve in the forecasted direction, but not with the forecasted amplitude), the Context Agent considers that its validity was relevant. Thus it does not modify its validity ranges but adjusts its forecasts to match its observation.

This NCS also impacts the Controller Agent that had selected the Context Agent with the wrong forecasts. When a Controller Agent notices that the critical levels variations don't match the forecast, it stop applying the current action, notifies the Context Agent that its action is no longer applied, and looks for a new action in its suggestion list.

Self-Organization

Context Agents are able to evaluate their own behavior and to adjust it (by modifying their forecasts and their validity ranges) if necessary. Thus we can say that a Context agent is self-adaptive. Moreover, Context Agents are created at runtime. Each of them follows simple and local behavioral rules. These rules lead to the formation of a coherent set of Context Agents where each agent occupies a portion of the environment state space for which its forecasts on critical levels are correct. Thus we can say that the set is the result of the self-organization of the Context Agents. It is also intersting to note that all the Context Agents of a given set put together give a good approximation of the criticality functions.

There is another, more subtle, level of selforganization. Each Controller Agent makes its own decisions about the action to apply on its effector, other Controller Agents are never consulted. Nonetheless, it appears that they manage to synchronize their actions and efficiently reduce the critical levels, solving the constraints. This is possible because each process input corresponding to a Controller Agent is also represented as a Variable Agent. Context Agents suggestions for one Controller Agent are therefore conditionned by the current state of other Controller Agents corresponding process input. As they all try to lessen the critical levels, they eventually find a synergy and the global behavior of ES-CHER (the actions on all the controlled inputs) is coherent.

RESULTS

Generated black-boxes [START_REF] Boes | Mimicking Complexity: Automatic Generation of Models for the Development of Self-Adaptive Systems[END_REF] were used for the first tests to check ESCHER's abilities to deal with multiple inputs and outputs, to find compromises between several objectives, and to adapt to unexpected changes of behavior of the controlled process. These tests are not detailed here due to space limitations.

Several tests have been driven on a 125cc monocylinder fuel engine in various steady operating points of rotational speed and load. Those presented here have been driven at 5000rpm and 870mbar of pressure in the inlet manifold. The goal of ESCHER is here to maximize the indicated mean effective pressure (IMEP), an output variable that reflects the torque. In other words, the goal is to minimize the critical level of an Optimization Constraint Agent associated to the IMEP. Injected mass fuel (IMF) and ignition advance are controlled by ESCHER, which has no other previous information about the engine than the variation interval of the variables. Each test has been supervised and validated by domain experts. CHER does not know anything about the controlled process. It first decreases the IMF, but it is a mistake: the critical level rises (first lifecyle). The error is corrected by increasing the IMF. The critical level drops, so ESCHER keeps this action (lifecycles 2 and 3). Then it perceives that while the IMF is rising, the critical level of the IMEP is rising too. But it is due to the noise on the IMEP. This causes ESCHER to mistakenly decrease the IMF. The error is rapidly detected and ESCHER eventually finds that it has to increase the IMF in order to reduce the critical level (lifecycles 5 to 13). Once the maximal IMEP has been reached, ESCHER oscillates around the IMF value that keeps the critical level at its minimum. During this test, five Contexts Agents were created and the optimum value was reached in 12 lifecycles. The main challenge with the real engine compared to generated black boxes is to deal with noise and latency. As ESCHER quickly reacts to errors, the noise is handled naturally. To overcome the latency between an action and its effects, ESCHER waits during a parametrizable period of time between its actions and its observations. For each test on the engine, a lifecycle was executed every three seconds.

Controlling Mass Fuel Injection

Discussion

ESCHER finds optimum values for all of the tested operating points. As it is impacted by the noise on the variables, the time taken to converge may vary. The biggest differences were observed at 4000rpm and 700mbar of pressure in the inlet manifold where it took from 100 seconds up to 400 seconds to converge. But it remains quicker than human experts doing it by hand as it is currently the case in the industry. More tests are planned, with more objectives and more simultaneously controlled variables.

CONCLUSION AND FUTURE WORKS

In this paper, we presented an Adaptive Multi-Agent System, called ESCHER, which controls heat engines, basing its behavior on self-organizing agents. We detailed its architecture and the agents interactions.

ESCHER is based on the idea that the complete understanding of how an engine works is not necessary to its control. All the controller needs to know is how the engine behaves. In other words our controller is of black-box type: ESCHER only perceives the engine's inputs and outputs, but not its internal mechanisms. The basic principle of our controller is the following: the state of the engine inputs/outputs is memorized when an action is applied and the consequences of this action are recorded. This information will be used to decide whether this action was good or not in regard of the user-defined desired engine state. This means that the quality of the control improves over time: at the begining the controller knows nothing about the engine, but it perpetually learns from its actions and quickly manages to control it. The learning is parallel to the control, so ESCHER continuously self-adapts to the process.

No prerequisite knowledge other than the intentions of the user (i.e. some criticality functions) is needed by ESCHER. It is also able to satisfy multi-criteria constraints on multiple inputs and outputs. Moreover, the independence between Controller Agents gives a certain modularity to ESCHER. Each Controller Agent (and its related Context Agents) is a stand-alone MAS that can be plugged on a process input. Regardless on what is controlling the other inputs, it will be able to synchronize its actions to perform a correct control.

We showed that ESCHER is able to deal with nonlinearity and multiple inputs and outputs, and that it handles the noise and latency of a real engine, without heavy knowledge about the engine. The only required instantiation work is the translation of the user's objectives into criticality functions.

By focusing on its learning skills, we consider to make the ESCHER generic enough to be easily applied to all kind of systems. Multi-Agent Systems using a similar pattern have already been successfully applied to the control of the temperature of a bioprocess [START_REF] Videau | Controlling Bioprocesses using Cooperative Self-organizing Agents[END_REF] and are currently being applied in the contexts of ambiant systems [START_REF] Guivarch | Context awareness and adaptation in ambient systems by an adaptive multi-agent approach[END_REF] and energy management of buildings.

Secondly, the behavior itself needs some improvements. New mechanisms for a better resolution of the problem of latency between the actions and their effects are required. Moreover, in large processes, some inputs may not affect some outputs. The learning of the process behavior could be enhanced by taking into account this fact. Finally, the main limitation to the application of ESCHER is the need for predefined criticality functions. Self-defined criticality functions could simplify make our controller even easier to instantiate.

Figure

  Figure 1: Criticality Functions

Figure 2 :

 2 Figure 2: ESCHER and its Environment

Figure

  Figure 3: ESCHER Topology

Figure 4

 4 Figure4shows ESCHER controlling the IMF to maximize the IMEP. At the beginning of the test, ES-

Figure 4 :

 4 Figure 4: Controlling Injected Mass Fuel to Maximize IMEP

Figure 5 :

 5 Figure 5: Controlling Injected Mass Fuel and Ignition Advance to Maximize IMEP


Jérémy Boes
email: boes@irit.fr

Pierre Glize
email: glize@irit.fr

Frédéric Migeon
email: migeon@irit.fr

Mimicking Complexity: Automatic Generation of Models for the Development of Self-Adaptive Systems

Keywords: Complex Systems Modeling, Multi-Agent Systems, Self-tuning, Self-composition

Many methods for complex systems control use a black box approach where the internal states and mechanisms of the controlled process are not needed to be known. Usually, such systems are tested on simulations before their validation on the real world process they were made for. These simulations are based on sharp analytical models of the target process that can be very difficult to obtain. But is it useful in the case of black box methods? Since the control system only sees inputs and outputs and is able to learn, we only need to mimic the typical features of the process (such as non-linearity, interdependencies, etc) in an abstract way. This paper aims to show how a simple and versatile simulator can help the design of systems that have to deal with complexity. We present a generator of models used in the simulator and discuss the results obtained in the case of the design of a control system for heat engines.

INTRODUCTION

More and more artificial systems have to deal with real-world complex processes, like heat engines, bioprocesses or energy management. The design of such systems usually requires the use of the relevant corresponding models so they can be tested on simulations. Unfortunately, models for simulators are very time consuming to design by the domain experts. This is due to the great amount of parameters, their interdependencies, the dynamics and the non linearities between their parts.

Relating our experience in the case of the design of a heat engine controller, this paper aims to show that the use of an accurate model is not always required to ensure the efficiency of a system before its deployment in the real world. Indeed when the system is able to learn during its lifetime and to adapt itself to the process, the tests concern the learning skills and the adaptiveness, for which the simulation fidelity to the domain is not crucial. Following this idea, we designed simple and versatile abstract black-boxes to test our system during its development phase. They do not represent a specific real-world process, but only replicate the important features from the adaptiveness point of view (such as non-linearity, interdependencies, oscillations or latency). A generator has been developped to easily build these black-boxes in order to be able to perform every needed test.

Section 2 gives a short state of the art and explain why such black-boxes can be adequate, then section 3 details the method to generate them. Section 4 relates how we use the generated black-boxes and evaluates their efficiency. Finally, we conclude in section 5 with an analysis of our approach and future improvements.

CONTEXT

Our work on complex systems modeling was driven by our needs in a research project on complex systems control. This section gives a succinct state of the art of this field before speaking about complex system modeling and simulation.

Complex Systems Control

Controlling systems is a generic problem that can be expressed as finding which modifications must be applied on the inputs in order to obtain the desired effects on the ouputs. The next paragraphs describe the main approaches and challenges.

PID -Proportional-Integral-Derivative (PID) are widely used in the industry. They base the control on three terms related to the error between the current state and the desired state of the process [START_REF] Astrom | PID Controllers: Theory, Design, and Tuning[END_REF]. PID controllers need a heavy parametrization work to fit the controlled system. Moreover they are not efficient with non-linear systems and are difficult to deploy to handle more than one input, which is a severe drawback for complex systems control.

Adaptive Control -To deal with several inputs and non-linearity, model-based approaches like Model Predictive Control (MPC) [START_REF] Nikolaou | Model predictive controllers: A critical synthesis of theory and industrial needs[END_REF] use models to forecast the behavior of the process in order to find the optimal control scheme. These approaches are limited by the models they use. Building the models and finding the parameters that fit the actual controlled system is an open problem. This will be detailed in 2.2.

Intelligent Control -Intelligent control regroups approaches that use Artificial Intelligence methods to enhance existing controllers and possibly avoid the use of models. Among these methods we find fuzzy logic [START_REF] Lee | Fuzzy logic in control systems: Fuzzy logic controller[END_REF], expert systems [START_REF] Stengel | Intelligent failure-tolerant control[END_REF], neural networks [START_REF] Hagan | An introduction to the use of neural networks in control systems[END_REF], bayesian controllers (Colosimo and Del Castillo, 2007) and multiagent systems [START_REF] Wang | Multi-agent co-ordination for the secondary voltage control in power-system contingencies. Generation, Transmission and Distribution[END_REF] or [START_REF] Videau | Controlling Bioprocesses using Cooperative Self-organizing Agents[END_REF]. These methods often use a black-box approach where only inputs and outputs values (but not internal variables and mechanisms) of the process are known by the controller. Hence intelligent controllers mostly rely on their learning skills to find the optimal control.

The main challenges in complex systems control are to deal with interdependencies, non-linearity and high dynamics. But they also include latency and noise on the data.

Complex Systems Modeling

In the context of complex systems control, models can be used in two ways. Either they are incorporated in the controller to forecast the process behavior, or they are used in simulators to test the controller before its use on the real system.

For instance, many models of engines have been proposed, such as [START_REF] Borman | Mathematical simulation of internal combustion engine processes and performance including comparison with experiment[END_REF] or the Turbocharged Diesel Engine (TDE) model [START_REF] Jankovic | Constructive lyapunov control design for turbocharged diesel engines[END_REF]. The TDE model has seven parameters. Using this model in a controller involves to set them so the model fits the controlled engine. This is a very difficult task, therefore control methods like NCGPC (Nonlinear Continuous-time Generalized Predictive Control) use a simplified version with only three parameters [START_REF] Dabo | NCGPC with dynamic extension applied to a Turbocharged Diesel Engine[END_REF]. However, finding the proper values of these parameters remains time consuming and they might become inaccurate when the engine wears out. The same simplified TDE model is implemented and used in a simula- Other simulation tools focus on different goals such as finding optimal values of engine parameters [START_REF] Curto-Risso | Optimizing the operation of a spark ignition engine: Simulation and theoretical tools[END_REF], comparing fuel performances [START_REF] Zhang | Comparison of biodiesel performance based on hcci engine simulation using detailed mechanism with on-the-fly reduction[END_REF] or leading studies on the behavior of a specific type of engine [START_REF] Zhao | Modeling and simulation of a hydraulic free piston diesel engine[END_REF]. These models are highly specific and demands a heavy work of implementation and parametrization.

Simulation Needs

Our needs in term of simulation come from the development of a self-adaptive complex systems controller meant to be applied on heat engines. We want to be sure of the system's adaptation skills before applying it to real-world engines. Thus, we first identified the challenging characteristics of engines complexity in order to simulate them to test our system. In this context, the simulation does not need to be accurate but only to mimic the complexity of real-world engines.

In other words we want to check the behavior of our system under various levels of interdependency, various numbers of inputs and outputs, with linear or nonlinear outputs, with or without latency (or under any combination of the previous features), regardless of the relevance of the data relatively to the real engine. We also need several instances of each case. None of the existing heat engine models matches our required degree of modularity.

Furthermore, existing black-box generation techniques either focus on testing large software with a discrete state-transitions behavior [START_REF] Tahat | Requirement-based automated black-box test generation[END_REF] or are meant to test software components and not the whole system [START_REF] Edwards | A framework for practical, automated black-box testing of component-based soft-ware[END_REF]. We need to run tests on the system level with a continuous process.

Hence we use simple yet versatile black-boxes that mimic the complexity of a real-world process and that can be automatically generated. The models they use and their generator are presented in the next section.

Constraints Associated Behaviors

Bind Constraint: The agent must bind its output port to at least one Function Agent. Critical level: 1.

Search for Function Agent with a free input port and bind to it.

Constraints

Associated Behaviors Bind Constraint: The agent must bind its input port to one (and only one) Function Agent. Critical level: 2.

Search for Function Agent and bind to it.

Path Constraint: The agent must link to every Input Agents of a specified list, through a chain of Function Agents. Output Agents are initialized with a Path constraint regarding to the user specifications. Critical level: 2*size of the list.

Search for a Function Agent linked to some of the targeted Input Agents (if possible), bind to it, and delegate the Path Constraint.

ABSTRACT BLACK-BOXES

In this section we present our abstract blackboxes: how they are used, what they are made of, and how they can be easily generated thanks to a multiagent system.

Utilization and Description

Abstract black-boxes are used to simulate a realworld process in order to test the learning abilities of an artificial system during its development. They allow to focus on a specific feature (or on a combination of them) of complexity, and to make available a large collection of different instances of the same feature.

They are composed of inputs, outputs and mathematical functions. Only inputs and outputs are observable from the outside. Each of the inputs are linked to at least one output through a chain of one or more functions, and vice versa. The chains of functions can be intermingled and can presents cycles. The role of inputs and outputs is obvious, they are the link between the functions and the environment of the black-box. At each timestep, inputs get a value from the environment and forward it to their linked functions, functions compute a value and forward it to their linked neighbors (either other functions, itself or outputs), and outputs get the calculated value of their linked function and forward it to the environment. To ease the automated generation presented in 3.2, it has been decided to limit the functions to two arguments and one calculated value. Figure 1 shows an example of a 4 inputs -2 outputs generated blackbox, composed of five functions, containing one cycle and where each of the four inputs impacts both of the outputs.

Generation

A Multi-Agent System, based on the Adaptive Multi-Agent System theory [START_REF] Georgé | Cooperation[END_REF] and called BACH (Builder of Abstract maCHines), has been developped to easily and rapidly generate abstract black-boxes given user-defined constraints. The user gives the number of inputs, outputs and functions, defines the variation range of each input and output, and which input impacts which output. He also sets a loop factor which determines the percentage of functions that have to be in a cycle in the generated black box. Then agents are created for each component of the abstract black-box (Input Agents, Output Agents and Function Agents). There are two main steps to the generation. First, the self-composition is in charge of the respect of the structural constraints (interdependencies and cycles). Then the self-tuning of the functions ensures that the outputs stay in their specified range. The self-composition and self-tuning phases are explained in the next two paragraphs.

Self-Composition

BACH is initialized regarding to the user specifications: Inputs, Outputs and Functions Agents are created and each of them is given a set of individual constraints. They have to cooperate and form links between them to solve these constraints. The resulting organization is forwarded to the second step of the generation. The next paragraphs describe the agents, their constraints and how they solve them. Architecture and Skills -Agents have inputs and/or output ports. They can bind their output port to other agents input ports (and the other way around) but input ports can only be bound once while output ports Constraints Associated Behaviors Bind Constraint: The agent must bind its output port to at least one agent and bind each of its input ports to one agent. Critical level: 5.

First search a partner to bind the output port, then find a partner for each of the input ports. Partners can be Input Agents, Output Agents, Function Agents or itself. Path Constraint: The agent must link to every Input Agents of a specified list, either by direct binding or through a chain of other Function Agents. Function Agents are not initialized with Path Constraints, but they eventually receive it from the binding partners on the output port. Critical level: 4*size of the list.

If the agent's input ports are both free and if the size of the list is less than or equal to 2, then bind the input ports to the specified Input Agents. The constraint is then solved. Else, if the size of the list is greater than two, bind one of the input port to another Function Agent, split the list in half and send one of the half to the new partner, keep the other. If both of the input ports are already bound, split the list and send a half to each of the partners. Cycle Constraint: The agent must be in a cycle. Critical level: 7

Bind the output port to an agent of its input path, or bind one of its input ports to an agent in its output path, or bind one of the input port to its own output port. can be bound as many times as needed. Input Agents do not have any input port and can bind their output port to Function Agents only. Output Agents do not have any output port and can bind their input port to Function Agents only. Finally, Functions Agents have two input ports, one output port and can bind them to Input Agents, Output Agents or Function Agents (including itself). Constraints -There are three types of constraints with which agents can be initialized: Bind Constraints represent the need for an agent to have all of its input and output ports bound to another agent, Path Constraints force agents to link to a specific list of Input Agents (if needed through a chain of Function Agents), and Cycle Constraints express the need for a Function Agent to be part of a cycle. Behavior -Each constraint has a critical level representing its relative significance. Agents try to solve their most critical constraint first. They have a specific behavior for each type of constraint, described in tables 1, 2 and 3.

The choice of a binding partner is based on two main criteria: its usefulness and its critical level. First, the agents are sorted between potential and nonpotential partners, whether they can fill the needs of the selecting agent or not. Then the potential partner for which the gain in terms of critical level will be the best is selected. During the self-composition, an agent may not find any potential partner. This happens when the user did not specified enough functions regarding to the number of inputs and outputs and their interdependencies. In other words the user specified an overconstrained problem. In this case, the system must release a constraint: the number of functions. Hence the lonely agent creates a new Function Agent and they bind immediately.

Self-Tuning

Once the structure of the black-box is set by the selfcomposition phase, interdependencies and cycles answer a part of the user requirements. Now the selftuning of the functions ensures that outputs stay in the user-specified variation ranges.

Function Agents embed a matrix that is a discrete representation of their function. The first row and the first column are the axis (for the arguments) while other cells contain the value of the function, thus are called function cells. Table 4 shows an example of a function f(x,y) where x varies from 1 to 10, y varies from 0 to 70 and f from 0 to 100. A linear interpolation is perfomed when needed for the computation of a value. For instance in table 4, f(3.5,60)=4.5. The self-tuning of the functions is merely the correct filling of the matrices.

To do so, each Output Agent sends to its bound Function Agent a message containing its variation range. The Function Agent randomly puts each bound in one of the function cells of its matrix and calculates the other function cells value v using the formula

v = a * d + m (1)
where m is the value of the closest bound, d is the Manhattan distance to this bound and a is a positive coefficient if m is the minimum, negative otherwise.

If the computed v is out of bounds, the value of the closest bound is affected to the cell instead of v. Once the function cells are filled, the variation range of the Output Agent is respected. Then, the Function Agent sends a message containing this range to every Function Agent (possibly itself) bound to its output. Every agent receiving this message will consequently adjust the corresponding axis in its matrix. Finally, Input Agents send their variation range to their bound Function Agents, who adjust their corresponding axis.

Generated Black-Boxes

Figure 1 shows the result of the self-composition of four Input Agents, two Output Agents and five Function Agents while table 4 shows the result of the selftuning process for a Function Agent. In addition to the required characteristics of interdependency and non-linearity, generated black-boxes exhibit oscillations (periodic or aperiodic), latency and progressive stabilization, as shown in figure 2 where inputs are manually controlled. We can see that the black-box first takes some time to stabilize itself, then a modification is applied on input In4. This causes perturbations on outputs Out1 and Out2, then they eventually stabilize until another modification applied on input In3 causes them to move again.

The next section explains how we used the abstract black-boxes during the development of a heat engine controller, and compares the results of the tests on generated black-boxes with those obtained on a real engine.

CASE STUDY: THE DESIGN OF A CONTROL SYSTEM FOR HEAT ENGINES

Involved in a national project about heat engine control named Orianne 1 , our role was to develop a self-calibration algorithm for the engine control unit. This section explains how abstract black-boxes were used and describes the results obtained both on simulators and on real engines. In the following paragraphs ESCHER is the name of the self-adaptive controller being developed and tested, and stands for Emergent Self-adaptive Control for Heat Engine cali-bRation.

Method

We defined a two phases process for the development of ESCHER. During the first phase, ESCHER was de-1 French acronym for Digital Tool for the Design of Engine Control Functions. signed and implemented, and all tests were conducted on abstract black-boxes. Then the second phase consisted in replacing the black-boxes by a real 125cc gasoline engine. The goal of this second phase is to validate the use of abstract black-boxes. Indeed, our software is able to learn the behavior of the generated black-boxes to find the best actions to apply to their inputs, hence being able to do the same with a real engine would mean that the use of abstract black-boxes is relevant.

Results

In this section, more information about ESCHER is given before results on abstract black-boxes and on a real engine are shown.

ESCHER

ESCHER is a Multi-Agent System based on the Self-Organizing Controller pattern [START_REF] Boes | Self-Organizing Agents for an Adaptive Control of Heat Engine[END_REF]. Its function is to find the adequate values of a process inputs in order to make the outputs fulfilling user-defined objectives. These objectives are various (thresholds, optimizations or setpoints), can change over time and are expressed as criticality functions that the system tries to lower. ESCHER has no previous knowledge about the process other than its inputs and outputs and their variation ranges. It learns in real-time from the effects of its actions to adjust itself to the process and finds the most adequate actions to apply. In our case, the process is an engine equipped with its control unit (ECU): the inputs are some of the parameters of the ECU and the outputs are values measured on the engine.

In the next paragraphs, we show an example of test we ran using abstract black-boxes.

Results on abstract black-boxes

For this example, we use a black-box with two inputs (Var0 and Var1) and two outputs (Var2 and Var3). The objective is to set Var2 and Var3 to the middle of their range of variation, thus each of the outputs is associated with a criticality function. ESCHER must find the value of the inputs for which both of the criticality functions are at their minimum.

Figure 3 shows the critical levels of the outputs and the values of the inputs over time. At first ES-CHER increases only Var1. This causes the critical levels to drop, so ESCHER keeps on this action until the critical levels begin to increase. Then ESCHER changes its behavior, modifying its actions (it starts to increase Var0, lowers and stabilizes Var1, etc) to maintain the lowering of the critical levels. Finally both critical levels reach zero, which means the objectives have been reached. ESCHER now maintains the black-box in this state.

We have seen that ESCHER is able to find the most adequate values for the inputs of abstract blackboxes (i.e. for which the critical levels are the lowest). Now we see in the next paragraph how it behaves with the real engine.

Results on the real engine

The following paragraphs describe an example of the tests conducted on an engine and validated by domain experts. The engine is put in a steady operating point (defined by its rotational speed and its load). Then ESCHER has to optimize the engine behavior by modifying specific parameters of the ECU. ESCHER controls the injected fuel mass along with the ignition advance. Its goal is to maximize the torque. Figure 4 shows the torque critical level and both of the controlled parameters. At first, ESCHER decreases both of the controlled parameters, causing the critical level to rise. It corrects this initial mistake, increasing the injected fuel mass first, then the ignition advance. It eventually finds the optimum value for which the critical level is the lowest.

Before analysing these results, it is important to note that alhough the test shown in 4.2.2 is simple and easily explainable, BACH was used to generate heavier black boxes that allowed us to enhance ESCHER.

Analysis

ESCHER is able to adapt itself and to find an optimum for abstract black-boxes as well as for the real engine. This means that the models used in the black-boxes correctly replicate the main aspects of the engine in terms of non-linearity and degree of interdependency. The behavior of ESCHER faced to these main aspects of complexity was properly tested thanks to the generated abstract black-boxes. However, some adjustements in the way ESCHER interacts with the process had to be made before obtaining these results.

This was due to the fact that two important features had not been tested with the black-boxes: the noise on the perceived data, and the latency between an action and its effects. As ESCHER reacts quickly to correct its mistakes, the noise did not prevent the system to work properly, it only made it make a few more mistakes. But the latency caused more difficulties, the system was not able to find the correlation between its actions and its observations of the process behavior. We overcomed this problem by slowing down ESCHER, making it wait three seconds between its actions and its perceptions.

As most of the work had been done thanks to the black-boxes, the second phase of development lasted about three months (versus 12 months for the first phase) and was only about minor modifications. In conclusion, the abstract black-boxes were sufficient for a very large part of the development of ESCHER but still need some improvements to be more efficient. This will be discussed in the next section.

CONCLUSION AND FUTURE WORKS

In this paper, we presented models that replicate some aspects of complexity and a way to simply generating them and use them as black-boxes. We showed that they can be used during the development of self-adaptive systems to test the adaptiveness skills because they properly imitate the main features of complex systems. The easy generation allows to produce tailored simulators. Interdependencies are ensured by the slef-composition process while nonlinearity is provided by the self-tuning process. Latency can be obtained by varying the number of functions in the model. As abstract black-boxes only rely on general properties of the real world processes they represent, they remain generic and can be used in various domains. They allowed us to abstract the specifics of engines and thus to obtain a generic controller.

We claim that this approach avoids a time consuming work: the creation of a fitted model of the system we have to control. We show that the self-adaptive control system tested on abstract black-boxes gives interesting results when applied on a real heat engine. Nevertheless abstract black-boxess are only interesting when we have to develop a software able to learn.

Our future works will focus on two main drawbacks: the absence of noise simulation and the lack of control on the generation process. The first one should be dealt by adding some classical noise generation algorithms on the Output Agents, such as [START_REF] Taralp | Efficient fractional gaussian noise generation using the spatial renewal process[END_REF]. The second one leads to the generation of models that are too intricate when the number of functions is too high. The outputs stay in their user-defined variation range but are unable to really reach their minimum and maximum. This should be handled by finding better behaviors for the agents of BACH during the self-composition process. Finally, the self-tuning process can also be improved by adding different methods for the calculation of the matrices, such as b-splines surfaces [START_REF] Catmull | Recursively generated b-spline surfaces on arbitrary topological meshes[END_REF].

Figure 1 :

 1 Figure 1: Inside an abstract black-box

Figure 2 :

 2 Figure 2: Inputs and outputs variations over time of a generated abstract black-box.

Figure 3 :

 3 Figure 3: Test on an abstract black-box: outputs critical levels (up) and inputs values (bottom)

Table 1 :

 1 Input Agents: constraints and behavior.

Table 2 :

 2 Output Agents: constraints and behavior.

Table 3 :

 3 Function Agents: constraints and behavior.

Table 4 :

 4 A Function Agent's function matrix: axis (grey) and function cells (white).