Mathieu Bourgais

Patrick Taillandier

Laurent Vercouter

BEN : An Agent Architecture for Explainable and Expressive Behavior in Social Simulation

Keywords: Social Simulation, Agent Architecture, Cognition, Emotions, Evacuation

Social Simulations are used to study complex systems featuring human actors. This means reproducing real-life situations involving people in order to explain an observed behavior. However, there are actually no agent architectures among the most popular platforms for agent-based simulation enabling to easily model human actors. This situation leads modelers to implement simple reactive behaviors while the EROS principle (Enhancing Realism Of Simulation) fosters the use of psychological and social theory to improve the credibility of such agents. This paper presents the BEN architecture (Behavior with Emotions and Norms) that uses cognitive, affective and social dimensions for the behavior of social agents. This agent architecture has been implemented in the GAMA platform so it may be used by a large audience to model agents with a high level explainable behavior. This architecture is used on an evacuation case, showing how it creates believable behaviors in a real case scenario.

Introduction

These last years, agent-based simulation has been used to study complex systems featuring human actors ; the community is now speaking of social simulation [START_REF] Gilbert | Simulation for the social scientist[END_REF]. The main goal is to reproduce real life situations involving hundreds or thousands of simulated humans in order to better understand interactions leading to an observed result.

To be as close as possible to case studied, social simulations have to integrate social agents with a behavior as close as possible to the human behavior. The creation of believable social agents implies the reproduction of complex processes simulating the human reasoning [START_REF] Sun | Cognition and multi-agent interaction: From cognitive modeling to social simulation[END_REF]. Such systems lead to a closer behavior to the one expected. However, the obtained behavior may be hard to explain and to express with high level concepts. This is the meaning of the KISS (Keep It Simple, Stupid) principle [START_REF] Axelrod | Advancing the art of simulation in the social sciences[END_REF] which invites modelers to keep a simple behavior model so it can be explainable by simple rules at any moment in the simulation. This principle has been discussed over the years, leading to the KIDS (Keep It Descriptive, Stupid) principle [START_REF] Edmonds | From kiss to kids-an 'anti-simplistic'modelling approach[END_REF] that favors more descriptive models to gain realism and then the EROS (Enhancing Realism Of Simulation) principle [START_REF] Jager | Enhancing the realism of simulation (eros): On implementing and developing psychological theory in social simulation[END_REF] which calls for the use of cognitive, affective and social dimensions to improve the credibility of social agents. Therefore, the problem is to model realistic simulated humans with an explainable behavior at a high level.

To tackle this issue, this paper presents BEN (Behavior with Emotions and Norms), a modular agent architecture integrating cognition, emotions, emotional contagion, personality, norms and social relations. Each of these components relies on psychological or social theories, helping a modeler to improve the credibility of simulated humans and ensuring an explainable behavior with high level concepts [START_REF] Kaptein | Self-explanations of a cognitive agent by citing goals and emotions[END_REF].

This architecture is implemented and integrated within GAMA [START_REF] Taillandier | Building, composing and experimenting complex spatial models with the gama platform[END_REF], a modeling and simulation platform aiming to be used by a large audience. The goal is to create a tool that may even be used by modelers who are not expert in programming, without loosing the expressivity for the behavior developed. This implementation is explained in this paper through the example case of the evacuation of a nightclub, showing it succeeds to handle a real-life scenario and still provide a behavior with a high degree of explainability.

This paper is structured as follows: Section 2 reviews existing works to create social agents with a cognitive behavior, an emotional engine or social relations but also the existing agent architecture in popular simulation platforms. In Section 3, a formalism is proposed to deal with the mental state of the agent in terms of cognition, emotion and social relations. Section 4 describes the BEN architecture which relies on the aforementioned formalism. Section 5 presents the implementation of BEN through an example to illustrate how it can be used on a model of evacuation to create an explainable and believable behavior for agents simulating humans. Finally, Section 6 serves as a conclusion.

Related Works

Creating a believable social agent with an explainable behavior may be complex [START_REF] Langley | Explainable agency for intelligent autonomous systems[END_REF]. To ease this process, simulation platforms and behavior architecture have been developed by the community. These existing works are presented in this section.

Frameworks and platforms for simulations

Among the various agent-based platforms [START_REF] Kravari | A survey of agent platforms[END_REF], some like JACK [START_REF] Howden | Jack intelligent agents-summary of an agent infrastructure[END_REF] or Jadex [START_REF] Pokahr | Jadex: A bdi reasoning engine[END_REF] implement the BDI (Belief Desire Intention) [START_REF] Bratman | Intentions, plans, and practical reason[END_REF] paradigm, giving a cognitive behavior to agents, based on modal logic [START_REF] Philip R Cohen | Intention is choice with commitment[END_REF]. The addition of cognition helps creating more believable agents [START_REF] Adam | Bdi agents in social simulations: a survey[END_REF] but these platforms are not suitable for thousands of agents required in simulation.

To overcome this problem, Sing and Padgham [START_REF] Singh | Opensim: A framework for integrating agentbased models and simulation components[END_REF] propose to connect a simulation platform to an existing BDI framework (like JACK or Jadex) and, with the same idea [START_REF] Padgham | Integrating bdi agents into a matsim simulation[END_REF], the Matsim platform [START_REF] Balmer | Multi-agent systems for traffic and transportation engineering[END_REF] has been linked with the GORITE BDI framework [START_REF] Rönnquist | The goal oriented teams (gorite) framework[END_REF]. These works require a high level in computer science, making it difficult to use by modelers with low level programming skills.

Frameworks like Repast [START_REF] Collier | Repast: An extensible framework for agent simulation[END_REF] or MASON [START_REF] Luke | Mason: A multiagent simulation environment[END_REF] are dedicated simulations tools which improve existing programming languages to ease the development of agentbased simulations. Agents are described by Java classes and the framework is used to describe the scheduling of the execution of all the classes and the output of the simulation. This means these softwares do not offer specific agent architecture to control or to explain the agents' behavior.

On the other hand, simulation platforms like Netlogo [START_REF] Wilensky | Netlogo: Center for connected learning and computer-based modeling[END_REF] or GAMA [START_REF] Taillandier | Building, composing and experimenting complex spatial models with the gama platform[END_REF] are dedicated softwares with their own programming language, their own interface and their own interpreter and compiler. They are made to be easy to use by people with low level in programming skills and they can handle thousands of agents during simulations, making them usable for the definition of explainable models made by experts of the studied fields.

By default, these platform do not propose any particular architecture for the agent behavior: modelers have to define these behaviors with "if-then-else" rules. However, there exist plugins, for NetLogo [START_REF] Sakellariou | Enhancing netlogo to simulate bdi communicating agents[END_REF] and for GAMA [START_REF] Taillandier | A bdi agent architecture for the gama modeling and simulation platform[END_REF], to use agent architectures based on the BDI paradigm in order to create simulated humans with a more complex and more believable behavior. They both provide the agents with high level concepts such as beliefs and intentions and GAMA's plugin goes beyond, offering a reasoning engine, leading agents to make decisions based on the perception of its environment.

Agent Architectures for Social Simulations

Using behavioral architectures enables modelers to define more easily credible and explainable social agents as these architectures offer high level concepts from works in psychology and sociology for the decision making. Among the numerous agent architectures [START_REF] Balke | How do agents make decisions? a survey[END_REF], some of the most known are presented in this section. SOAR [START_REF] John E Laird | Soar: An architecture for general intelligence[END_REF] and ACT-R [START_REF] Byrne | Perception and action. The atomic components of thought[END_REF] are two cognitive architectures, grounded on works from psychology. Agents have access to a long and a short term memory, making a decision based on the previous experiments in a given context. These approaches are more complex than the BDI paradigm, making the decision making process more credible . However, they require heavy computation time, which makes them less pertinent for social simulations involving thousand simulated actors.

CLARION [START_REF] Sun | The importance of cognitive architectures: An analysis based on clarion[END_REF] represents another proposition of cognitive architecture. The agent's reasoning is divided between four sub-systems, each one manipulating explicit and implicit elements to make a decision in a given context. To our knowledge, CLARION is still a theoretical architecture which has not been implemented in any simulation platform.

Another approach consists in building the reasoning engine around the emotions of the agents. For example, EMA [START_REF] Gratch | A domain-independent framework for modeling emotion[END_REF] is based on the cognitive appraisal theory of emotion [START_REF] Arnold | Emotion and personality[END_REF] developed by Smith and Lazarus [START_REF] Smith | Emotion and adaptation[END_REF] while DETT [START_REF] Van Dyke | A model of emotions for situated agents[END_REF] is based on the OCC [START_REF] Ortony | The cognitive structure of emotions[END_REF] theory of emotions. Both those systems creates emotions by assessing the perceptions of the environment and then infer a behavior from the emotional state of the agent.

eBDI [START_REF] Jiang | Ebdi: an architecture for emotional agents[END_REF] relies on OCC theory too but it also uses a BDI architecture to make decisions. This means emotions are created through perceptions and then act upon beliefs, desires and intentions. Finally, these modified mental states are used to make a decision. This proposition has not been yet integrated to a simulation platform.

Finally, some researchers propose to rely on the social context of the agent to describe its behavior : this is done with normative architectures. EMIL-A [START_REF] Andrighetto | Emergence in the loop: Simulating the two way dynamics of norm innovation[END_REF] and NoA [START_REF] Josef | Norm-governed practical reasoning agents[END_REF] describe the agent's behavior with social norms, obligations and sanctions. In other words, an agent makes a decision depending on the state of the normative system at the level of a society of agents.

BOID [START_REF] Broersen | The boid architecture: conflicts between beliefs, obligations, intentions and desires[END_REF] and BRIDGE [START_REF] Dignum | Towards agents for policy making[END_REF] propose to combine a normative architecture with a BDI paradigm, leading the agent to take into account the social system when making a decision. However, contrary to EMIL-A and NoA, the agent has personal beliefs, desires and intention, creating a more heterogeneous and credible behavior. But, to our knowledge, these architectures have not been implemented in simulation platform in order to deal with thousand of simulated actors.

Synthesis

To comply with the EROS principle, modelers need architectures proposing as much psychological and social dimensions as possible. Currently, as shown in this section, there does not exists a single architecture proposing at the same time, cognition, affective dimensions and social dimension for simulation. The only attempts, to our knowledge, to combine more than two traits have used the notion of personality, to combine cognition with emotions and emotional contagion [START_REF] Lhommet | Never alone in the crowd: A microscopic crowd model based on emotional contagion[END_REF] or to combine cognition with emotions and social relations [START_REF] Ochs | Simulation of the dynamics of nonplayer characters' emotions and social relations in games[END_REF].

In this paper, we tackle this issue by proposing BEN (Behavior with Emotions and Norms), an agent architecture featuring cognition, emotions, personality, emotional contagion, social relations and norm management. To implement it, we have based our work on the existing cognitive architecture provided by GAMA. To ease the use of BEN, we have implemented it using the principles of GAMA that has proved its ease of use [START_REF] Eg Macatulad | 3dgis-based multi-agent geosimulation and visualization of building evacuation using gama platform[END_REF][39] thanks to its modeling language GAML that we extended.

Formalization of Mental States

With the BEN architecture, an agent manipulates cognitive mental states, emotions and social relations to make a decision. These notions, and the formalism used to represent them, are presented in this section.

Representing the World with Predicates

Predicates are used to unify the representation of the information from the world from the agent's point of view. P j (V) represents a general predicate with the following elements :

-P : the identifier of the predicate.

-j : the agent causing the information. -V : a set of values stored by the predicate.

Depending on the context, this general representation may change. P j represents an information with no particular value attached, P(V) represents an information caused by no particular agent and P stands for an information with no particular value, caused by no particular agent.

For example, the information there is a fire in the environment is represented by the predicate fire. If this fire is caused by agent Bob, it is represented by fire Bob . Finally, if this fire caused by Bob is at a location (x;y), this information is represented by fire Bob (location::(x,y)).

Reasoning according to Cognitive Mental States

With BEN, an agent has cognitive mental states representing its thoughts on the world. M i (PMEm,Val,Li) represents a general cognitive mental state possessed by agent i with the following elements :

-M: the modality, indicating the type of the cognitive mental state (e.g. a belief). -PMEm: the object of the cognitive mental state. It could be a predicate (P), another cognitive mental state (M) or an emotion (Em). -Val: a real value which meaning depends on the modality. It enables to compare two cognitive mental states with the same modality and the same object. -Li: a lifetime value indicating how long the cognitive mental state stays in the agent's knowledge.

In BEN, cognition is based on the BDI model [START_REF] Bratman | Intentions, plans, and practical reason[END_REF], stating an agent needs beliefs, desires and intentions. Also, to link cognition with affective and social dimensions, BEN features 6 different modalities :

-Belief: represents what the agent beliefs about the world. The meaning of the attached value is the strength given by the agent to the belief. -Uncertainty: represents an uncertain information about the world. The meaning of the attached value is the importance given to that uncertainty by the agent. -Desire: represents a state of the world the agent want to reach. The meaning of the attached value is the priority of this desire, compared to other desires.

-Intention: represent a state of the world the agent is willing to achieve. The meaning of the attached value is the priority of this intention, compared to other intentions. -Ideal: represents an information which is socially appraised by the agent.

The attached value is a praiseworthiness value given by the agent. -Obligation: represents a state of the world the agent ought to reach. The meaning of the attached value is the priority of this obligation, compared to other obligations.

Representing Emotions and Social Relations

Emotions in BEN are based on the OCC theory [START_REF] Ortony | The cognitive structure of emotions[END_REF] which means emotions are valued answers to the appraisal of a situation. Em i (P,Ag,I,De) represent an emotion possessed by agent i with the following elements :

-Em: the name of the emotion.

-P: the predicate about which the emotion is felt.

-Ag: the agent responsible for the emotion.

-I: the intensity, positive or null, of the emotion.

-De: the decay value for the emotion's intensity.

This representation enables the agent to have multiple emotions at the same time, all on different predicates. Also, this representation can be adapted, with Em i (P,Ag) representing an emotion with no particular intensity nor decay value.

Finally, each agent may store social relations with other agents simulating human actors. These relations are based on the work of Svennevig [START_REF] Svennevig | Getting acquainted in conversation: a study of initial interactions[END_REF] who identifies four minimal dimensions to describe a social relation between two people. In BEN, R i,j (L,D,S,F,T) represents a social relation, from agent i towards agent j with the following elements :

-R: the identifier of the social relation.

-L: a real value between -1 and 1 standing for the degree of liking. A value of -1 indicates agent j is hated, a value of 1 indicates agent j is liked. -D: a real value between -1 and 1 standing for the degree of dominance. A value of -1 indicates agent j is dominating, a value of 1 indicates agent j is dominated. -S: a real value between 0 and 1 standing for the degree of solidarity. A value of 0 indicates no solidarity with agent j, a value of 1 indicates a complete solidarity with agent j. -F: a real value between 0 and 1 standing for the degree of familiarity. A value of 0 indicates no familiarity with agent j, a value of 1 indicates a complete familiarity with agent j. -T: a real value between -1 and 1 standing for the degree of trust. A value of -1 indicates mistrust against agent j, a value of 1 indicates a complete trust towards agent j.

With this definition, social relations do not have to be symmetric ; between two agents i and j, R i,j (L,D,S,F,T) is not obviously equal to R j,i (L,D,S,F,T).

An Agent Architecture with Cognitive, Affective and Social Dimensions

The BEN architecture represents the main contribution of this article. In this section, we explain how an agent using BEN makes a decision with cognition, emotions, emotional contagion, personality, social relations and norms. With these dimensions, an agent simulating an actor may react to a change in the environment and still explain its behavior with high level concepts. The personality component is based on the OCEAN [START_REF] Robert R Mccrae | An introduction to the five-factor model and its applications[END_REF] theory which defines fives parameters (Openness, Consciousness, Extroversion, Agreeableness, Neurotism) that are sufficient to represent a personality. To ease the use of BEN, these personality traits are the only parameter a modeler may access ; they are used to compute all the other parameters needed by the various processes : probability to remove a plan or an intention unfulfilled in the cognitive part, charisma and emotional receptivity for the emotional contagion, initial intensity and decay for emotions created by the engine, update values of social relations obtained with the social engine, and obedience for the normative engine.

Global Presentation of the Architecture

The agent's knowledge is composed by cognitive bases, containing cognitive mental states as formalized in section, an emotional base, a social base and a base of norms. This knowledge may evolve through the simulation, which is not the case of plans for the cognitive engine and sanctions for the normative engine, which are stored in dedicated bases, out of the agent's knowledge, as seen on Figure 1.

Each module, and each process of each module, may be mandatory (in plain line on Figure 1) or optional (in dash line on Figure 1) ; the optional modules and processes may be deactivated by modelers if not necessary in the case studied. For example, if the social engine is not useful from the modeler's point of view, this process is not used in the definition of the agent's behavior as it is an optional process.

Finally, some processes are executed automatically (in blue on Figure 1) and some others need to be defined manually by the modeler (in pink on Figure 1). This manual definition enables the architecture to be adapted to each case study while the automatic processes ease the use of BEN as the modeler does not need to be an expert in emotional creation or social relations update.

Making Decisions in an Evolving Environment

On Figure 1, every module has a number, indicating its order during the execution. Every time an agent is activated, it perceives the environment, it manages its knowledge based on the new perceptions, it makes a decisions and finally it gives a temporal dynamism to its knowledge. This section explains briefly how each process of each module works, more details can be found in a previous work [START_REF] Bourgais | Towards cognitive, affective and social agents in the simulation[END_REF].

Perceptions The First step in BEN, corresponding to the module number 1 on Figure 1, consists in perceiving the environment. This step is used to make a link between the world and the agent's knowledge, by creating beliefs and uncertainties on information from the environment, by defining emotional contagion with other agents or by creating new social relations. These three processes are defined manually which means the modeller has to indicate what information is transformed as a predicate and which cognitive mental state is build upon that predicate, which emotion is subjected to an emotional contagion and what is the initial value for each dimension of a new social relation. The last process of this module enables an agent to execute sanctions during the enforcement done on the other agents perceived.

Adding a belief is an important process in BEN as it triggers different rules. Precisely, adding a belief Belief A (X) :

removes belief Belief A (notX).

removes intention Intention A (X).

removes desire Desire A (X) if intention Intention A (X) has just been removed.

removes uncertainty U ncertainty A (X) or U ncertainty A (notX).

removes obligation Obligation A (X).

With the same principle, adding uncertainty U ncertainty A (X) :

removes uncertainty U ncertainty A (notX).

removes belief Belief A (X) or Belief A (notX) All these processes are defined inside a perception, which may be parameterized. A modeller indicates a distance of perception or a geometry inside which the perception is done, but also specifies which agents are perceived. As it is, this module and all its processes may adapt to any case study in social simulation.

Managing Knowledge

The second step of BEN, corresponding to the module number 2 on Figure 1, enables the agent to manage its knowledge after the perception and before making decision. In this phase, modelers may define inference rules, which enable to create or remove any cognitive mental state depending on the actual status of the agent's knowledge. For example, a modeler may define that an agent has a new desire Desire A (Y ) if this agent has a belief Belief A (X). On the same model, laws may be defined to create obligations if the obedience value of the agent, computed from its personality, is great enough.

During this second step, an emotional engine creates emotions based on the agent's knowledge. This process is done according to the OCC theory [START_REF] Ortony | The cognitive structure of emotions[END_REF] and its formalism with the BDI model [START_REF] Adam | Emotions: from psychological theories to logical formalization and implementation in a BDI agent[END_REF]. For example, an emotion of joy about a predicate P is created according to the following rule: Joy i (P j ,j) def = Belief i (P j ) & Desire i (P). The complete process and all the rules to create 20 emotions automatically, with no intervention from the modeler, are detailed in a previous work [START_REF] Bourgais | Towards cognitive, affective and social agents in the simulation[END_REF].

Finally, a social engine may be executed during this second step of the architecture. It updates the social relation with the other agents perceived, depending on the knowledge previously acquired. All this process and the equations to compute automatically the new value for each dimension of a social relation are explained in a previous work [START_REF] Bourgais | Towards cognitive, affective and social agents in the simulation[END_REF].

Making Decision

The third step of BEN, corresponding to the module number 3 on Figure 1, is the only mandatory part. This module enables the agent to make decisions and then execute an action, all of this through a cognitive engine over which a normative engine may be added. It is executed automatically, with no need of intervention from the modeler.

The cognitive architecture is based on the BDI model [START_REF] Bratman | Intentions, plans, and practical reason[END_REF] : the agent has intentions based on its desires and one of the intentions as a current intention. The modeler defines plans of action that indicates what action an agent has to do for a particular current intention in a given context; the plan chosen is kept as the current plan. The normative engine works the same way as the cognitive engine, with obligations as desires and norms as plans. The only difference is an obedience value that can be added to norms and obligations. More details may be found in previous works [START_REF] Bourgais | Towards cognitive, affective and social agents in the simulation[END_REF].

Temporal Dynamics The final part of the architecture, corresponding to the module number 4 on Figure 1, gives a temporal dynamic to the agent's behavior. This is done automatically by degrading the cognitive mental states and the emotions and by updating the status of each norm.

The degradation of cognitive mental states decreases the lifetime of each cognitive mental state stored by the agent. This mechanism enables an agent to forget, after a certain time, a belief, a desire, etc. The degradation of emotions consists in subtracting the decay value of each emotion to each intensity. With this process, an emotion fades away, unless it is created again, for example with the emotional engine or through the emotional contagion process.

Finally, the last process updates the status of each norm, indicating if it was usable in the current context, and in the case it was usable, has it been used or not by the agent. This system enables to ease the enforcement of norms in a later perception, as each norm indicates its status instead of computing this status in another context.

Simulating the Evacuation of a Nightclub

The architecture defined in Section 4 has been implemented in the modeling and simulation platform GAMA [START_REF] Taillandier | Building, composing and experimenting complex spatial models with the gama platform[END_REF], extending the GAML programming language to help modelers define social agents with cognitive, affective and social dimensions to express their behavior. This implementation is used on the example case of an evacuation of a nightclub in fire, as detailed in this section.

Presentation of the Example Case

The 27th of January 2013, the Kiss Nightclub in Santa Maria, Rio Grande do Sul state is Brazil, was set in fire at the end of a show lead by a local music band. The ceiling caught fire because of fireworks, emitting toxic smokes which lead to the death of 242 people. The official investigation put light on various factors which aggravated the tragedy: there were between 1200 and 1400 people in the building that could normally handle 641 people, there was only one entrance/exit door, there was no smoke detector nor alarms and finally, the exit signs were showing the direction of the restrooms. The investigation also shows that most of the deceases were due to asphyxia, near the restrooms [START_REF] Atiyeh | Brazilian kiss nightclub disaster[END_REF].

Our goal here is to reproduce the behavior of people caught in this tragedy in the most credible way possible. In other words, we are using the BEN architecture to create the agents' behaviors in order to get a result as close as possible as what happened in this nightclub during the fire.

Modeling the behavior of human actors with BEN

The initial agent's knowledge, at the start of the simulation, can be divided into three types : beliefs about the world, initial desires and social relations with friends. Also, each agent has a personality. The first step of BEN is the perception of the environment. We need to define what an agent perceives and how it affects its knowledge. Here are examples of the agent's perceptions :

-Perceiving the exit door updates the beliefs related to it.

-Perceiving the fire adds the belief there is a fire.

-Perceiving the smoke adds the belief about the level of smoke perceived.

-Perceiving other agents enables to create social relations with them. An emotional contagion about the fear of a fire is also defined.

Once the agent is up to date with its environment, its overall knowledge has to adapt to what it has perceived. This is done with the definition of inference rules and laws:

-A law creates the obligation to follow the exit signs if there is a reasonable doubt (modeled by the obedience value attached to the law and the quantity of smoke perceived) of a catastrophe. -An inference rule adds the the desire to flee if the agent has a belief there is fire. -An inference rule adds an uncertainty there is a fire if the agent has a belief there is smoke. -An inference rule adds the desire to flee if the agent has a fear emotion about the fire with an intensity greater than a given threshold.

With the execution of inference rules and laws, each agent creates emotions with the emotional engine. In this case, the presence of an uncertainty about the fire (added through the inference rule concerning the belief about smoke) with the initial desire that there is no fire produces an emotion of fear, which intensity is computed depending on the quantity of smoke perceived.

Once the agent has the desire to flee (because it perceived the fire or its fear of a fire had an intensity great enough), it needs action plans and norms to indicate what it has to do. Table 2 shows the definition of some action plans and norms used by the agent to answer its intention to flee, depending of the context it perceives. The agent has a good visibility and has a belief on the exact location of the exit door

The agent runs to the exit door

In this plan, the agent runs to the exit door following the shortest path. The agent has a good visibility and has no belief about the location of the door the agent follows the agent in its field of view with the highest trust value among its social relations This norm works with the trust value of social relations created during the simulation.

The agent has a bad visibility and has the obligation to follow signs

The agent goes to the restrooms In this norm, the agent comply with the law that indicates to follow exit signs. The agent has a bad visibility and has a belief exit signs are wrong

The agent moves randomly

In this plan, the agent moves randomly in the smoke.

The social relation defined with a friend may also be used to define plans to help one's friend if it is lost in smoke. This plan consists in finding the friend and telling him the location of the exit door.

As the situation evolves during the simulation, an agent may change its current plan. For example, if an agent leaves the smoke area while fleeing to the restroom, it may perceive the exit, and go there instead of following the signs.

The complete model can be found at this address: https://github.com/mathieuBourgais/ExempleThese

Results and Discussion

At the start of the simulation, agents are placed randomly in the recreated Kiss Nightclub with a personality initialised by a Gaussian distribution centered on 0.5 and with a standard deviation of 0.12 for each dimension. The spread of the smoke is modeled according to an official report from the french government [START_REF] Chivas | Formalisation du savoir et des outils dans le domaine des risques majeurs (dra-35) -toxicité et dispersion des fumées d'incendie phénoménologie et modélisation des effets[END_REF]; an agent is considered dead after 50 seconds in the heavy smoke.

Figure 2 shows a visual result of the simulation where the black lines represent the walls of the nightclub, the grey squares represent the smoke and the triangles represent the simulated actors. The color of each triangle indicates the plan followed. A video of the simulation can be found on the following address: https://github.com/mathieuBourgais/ExempleThese As the exact number of people in the nightclub is not known, we tested three cases : 1200 people at the beginning, 1300 people at the beginning and 1400 people at the beginning. The statistical results obtained in Table 3 are computed from 10 simulations for each scenario. Statistical results indicate our model is well calibrated to reproduce the real life case where 242 people died. However, the main result concerns the explainability and the expressivity of the model. The video of the simulation shows various behavior patterns which may be expressed with high level concepts thanks to BEN.

For example, a lot of agents leave the club at the beginning of the simulation because they directly perceived the fire. This behavior seems corresponding to a real life case where people seeing a fire in a nightclub would flee. On the other hand, agents which do not perceive the smoke or the fire are fleeing later. During that time, they forgot the location of the exit so they had to follow the official exit signs, leading them, in this case, to the restrooms.

Thanks to the BEN architecture, we were able to translate a behavior expressed in common language into an actual behavior for simulated actors. At any moment, it is possible to pause the simulation to inspect the behavior of an agent; this behavior will be expressed in terms of cognitive mental states, emotions, social relations, norms and plans which is, from our point of view, easier to read and understand than equations. This point is supported by the fact that BEN and its cognitive part relies on folk psychology [START_REF] Norling | Folk psychology for human modelling: Extending the bdi paradigm[END_REF]. Also, some works [START_REF] Broekens | Do you get it? user-evaluated explainable bdi agents[END_REF] [START_REF] Kaptein | The role of emotion in self-explanations by cognitive agents[END_REF] have shown that using BDI and emotions helps explaining the agents' behavior.

Conclusion

This article presents the BEN architecture, which enables to model agents simulating human actors with cognitive, affective and social dimensions. All the features of the architecture are based on theories coming from psychology and social sciences and are formalised in the same frame to interact between each other without being dependent. This allows the architecture to be domain independent and modular, so it can be used and adapted on different contexts. This architecture is currently implemented in the modeling and simulation platform GAMA and this implementation is used, in this article, on the case study of the evacuation of a nightclub in fire. This example shows BEN achieves to produce a complex and credible behavior but maintaining its high level explainability.

Figure 1 Fig. 1 .

 11 Figure1represents the theoretical BEN architecture, providing cognitive, affective and social dimensions to agents simulating human actors. It is made up of four modules, each composed of several processes, communicating with the agent's knowledge, all of this seating on the agent's personality.

Fig. 2 .

 2 Fig. 2. Simulation of the Kiss Nightclub's evacuation

Table 1 .

 1 Table indicates how few of these initial knowledge are formalized with BEN. Example of agent's initial knowledge

	statement	formalisation	description
	A belief on the exact po-	Beliefi(exitDoor, lif etime1) Each agent has a belief about
	sition of the exit door		the precise location of the exit
			door with a lifetime value at
			lif etime1.
	A desire there is no fire Desirei(notF ire, 1.0)	Each agent wish there is no
			fire in the nightclub with a
			priority of 1.0. This desire
			cannot lead to an action (no
			action plan are defined to an-
			swer it).
	a relation of friendship	Ri,j(L, D, S, F, T )	Each agent i is likely to have
	with another agent		a social relation with agent j,
			representing its friend.

Table 2 .

 2 Action plans and norms answering the fleeing intention

	conditions	actions	commentaries

Table 3 .

 3 Number of agents dead in the simulation of the Kiss Nightclub fire

	number of agents 1200 1300 1400
	mean value	230.2 237.7 249.4
	standard deviation 20.1 15.6 32.6

Acknowledgment

This work is partially supported by two public grants overseen by the French National Research Agency (ANR) as part of the program PRC (reference: ESCAPE ANR-16-CE39-0011-01 and ACTEUR ANR-14-CE22-0002).


The ALICE Muon Forward Tracker commisionning: first beam tests Beam test setup (June 2018 at CERN PS)

• ALPIDE Telescope: 3 chips (at 220, 160 and 140 mm from disk front plane).

• Trigger signal: plastic scintilliators upstream.

First hit maps from MFT disk protoype

Physics motivations

Main ALICE improvements with MFT

• Extend the open heavy flavour Physics program at large pseudo-rapidity, measurements down to low p T .

• Increase S/N ratio matching tracks with MUON Spectrometer.

• Add high-precision vertexing capabilities to the MUON spectrometer (Currently limited by the front absorber). e.g. prompt/non-prompt charmonium discrimation.

New high-precision measurements accessible thanks to MFT upgrade are summarised in the Table 1.1.

Schematic view of B decay J/ψ identification

Main beam test motivations

• Test readout and tracking capabilities from a half disk prototype using MOSAIC readout boards (1 board/ladder).

• Estimate resolution on track reconstruction (Expected sensor intrinsic spatial resolution: ∼5µm).

• Estimate detection efficiency (Expected detection efficiency > 99.5%).

Beam test data and methods

Collected data specifications

• Total Number of reconstructed tracks: ∼3.10 7 .

• 2 acquisition configurations: 1 front ladder + 1 back ladder (17 runs) and 1 front + 2 back (38 runs).

• 2 back-bias voltage configurations: 0V and -3V.

Resolution and efficiency calculations

• Alignment performed using Millepede with x, y, z and θ z coordinates as free parameters.

• Resolution σ int x,y calculated in the transverse plan in each direction (x or y).

• Horizontal direction (y):

σ int y = σ with y * σ without y
, where "with" and "without"mean taking into account or not the ladder point to do the tracking.

• Efficiency defined as the ratio between the number of tracks found in 2 ladders (1 back + 1 front) and the number of tracks found in the reference ladder (e.g back = ref to estimate front eff).

Cluster pattern study

• Cluster pattern distribution for one chip on a ladder. Distributions consistent for all chips.

Results of beam test data analysis Resolution

• Estimated Resolution for beam test data ∼ 7µm for y direction (∼ 7.5µm for x direction).

• Consistent with expected resolution ∼ 5µm considering multiple-scattering effects.

Efficiency

• Estimated detection efficiency consistent with expected efficiency (> 99.5%).

  Manuel GUITTIERE, ALICE Collaboration SUBATECH, CNRS/IN2P3, Université de Nantes, IMT Atlantique, Nantes, France, (Manuel.Guittiere@subatech.in2p3.fr) The ALICE Muon Forward Tracker commisionning: first beam tests Manuel GUITTIERE, ALICE Collaboration SUBATECH, CNRS/IN2P3, Université de Nantes, IMT Atlantique, Nantes, France, (Manuel.Guittiere@subatech.in2p3.fr) MFT upgrade overview Muon Forward Tracker technology • 5 disks equipped on both sides with silicon pixel sensors (920 ALPIDE sensors). Ladder structure: FPC + sensors.• ALPIDE technology: Monolithic Active Pixel Sensor (MAPS), CERN/CEA development.


Julien Champ

Titouan Lorieul

Pierre Bonnet

Najate Maghnaoui

Christophe Sereno

Thierry Dessup

Jean-Michel Boursiquot

Laurent Audeguin

Thierry Lacombe

Alexis Joly
email: alexis.joly@inria.fr

Categorizing plant images at the variety level: Did

teaching and research institutions in France or abroad, or from public or private research centers.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. (2016). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022 To create your highlights, please type the highlights against each \item command.

It should be short collection of bullet points that convey the core findings of the article. It should include 3 to 5 bullet points (maximum 85 characters, including spaces, per bullet point.)

• two new datasets were created for the evaluation of plant varieties recognition

• an experimental study was conducted with today's best performing techniques

• recognizing rice seeds variety appears to be feasible in controlled environment

• recognizing grape varieties from their leaves is still an open problem

• results show that convolutional neural networks perform the best on such problems

Introduction

Sustainable development of agriculture as well as biodiversity conservation are strongly related to our knowledge of the identity, geographic distribution and uses of plants. Unfortunately, such basic information is often only partially available for professional stakeholders, teachers, scientists and citizens, and often incomplete for ecosystems that possess the highest plant diversity (such as Mediterranean and tropical regions). One of the big challenge, expressed as the taxonomic gap, is that identifying plants is usually impossible for the general public, and also often a difficult task for professionals such as farmers or foresters and even for the botanists themselves. Using image-based identification and collaborative data management tools is considered as one of the most promising solution to help bridging the taxonomic gap (Joly et al. (2014b); [START_REF] Caputo | Imageclef 2013: the vision, the data and the open challenges[END_REF]; Cai et al. (2007a); Joly et al. (2014a); [START_REF] Spampinato | Automatic fish classification for underwater species behavior understanding[END_REF]; [START_REF] Kumar | Leafsnap: A computer vision system for automatic plant species identification[END_REF]). For centuries, plants have actually been classified according to their morphology, and then in many cases on their visual appearance. With the recent advances in digital devices/equipment, automatizing such classification process thanks to computer vision techniques therefore appears as the most straightforward solution.

The study of the visual appearance of plants is however much less advanced when speaking about agricultural varieties and genetic resources. Indeed, plants breeding has been practiced since thousands of years by farmers, nurserymen and others plant producers. Theses centuries of selection has allowed to develop specific plant characteristics (from wilds plants populations) for specific needs, such as better plant Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. (2016). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022

ACCEPTED MANUSCRIPT

A C C E P T E D M A N U S C R I P T 3 production, pest resistance, or water use efficiency. Due to this work, humanity is now able to store, access and exchange thousands of crops varieties or horticultural plants. Describing and analysing the morphological, physiological or ecological variations of the closely related varieties belonging to a single original species is however much more difficult than studying classical inter-species variations. This problem is increasingly studied with the recent advances in plant genomic and the availability of related data but it is usually not studied from the perspective of image data and automatic visual analysis tools. The emergence of automatized varieties identification tools based on visual contents might be useful in many different use cases such as (i) living or dry plant collections management (for field conservatory, nursery plant production), (ii) control of plant material transfer, (iii) field prospecting (with the aim to well characterize the terroir of a plant production), etc. However, categorizing plant images at the variety level is a problem that is much harder than the more classically studied problem of identifying plants at the species level. Working at such a finer taxonomic grain is actually much more difficult because of the very low inter-class variability and the fact that the visual patterns to be used for discriminating the varieties can be very specific and very different from a species to another one. The real grand challenge is thus not to design ad-hoc computer vision techniques for each use case but to evaluate in what way generic image classification techniques can deal with very different contexts and acquisition protocols set up by the biologists themselves. In this paper, we therefore introduce two image collections of agro-biodiversity interest with the aim to evaluate the feasability of automated plant varieties recognition from their visual appearance. They have been chosen so as to involve very different acquisition protocols and visual patterns in order to evaluate if state-of-the-art image classification techniques can generalize to such specific contexts and avoid the cost of building specific ad-hoc solutions. The acquisition protocol of each collection was carefully designed jointly with biologists and stakeholders so as to target innovative usage scenarios within existing workflows. The produced image datasets exhibit a very low inter-class variability resulting in two challenging fine-grained classification tasks, even for expert human operators (see section 3 and 4). As a second contribution, we then present the results of an experimental study that was conducted on the two datasets using the two effective families of classification techniques for solving finegrained image classification problems, i.e. convolutional neural networks on one side and fisher vectors-based discriminant models on the other side (see section 6). More generally, the scientific contribution of this paper lies in two main points. First of all, the production of specific visual training data and knowledge is now becoming one the most central problem in computer vision. Image classification has actually now been proved to be solved on large generalist corpora, but there is still a gap before being able to recognize the spectrum of millions or even billions of entities lying in the long tail of data occurrences (i.e classes with very few or none training samples available). Experimental studies such as the one conducted in this paper provides some essential findings on the genericity and transfer learning abilities of state-of-the-art computer vision techniques in this regard. Secondly, the paper proves for the first time that automatically identifying plant varieties from their visual appearance is feasible for some groups and usage scenarios and this opens the door to further investigations in the domain.

Related works

Content-based image retrieval and computer vision approaches are considered as one of the most promising solutions to help bridging the taxonomic gap, as discussed in [START_REF] Gaston | Automated species identification: why not?[END_REF]; [START_REF] Cai | Sensor network for the monitoring of ecosystem: Bird species recognition[END_REF]Trifa et al. (2008); [START_REF] Spampinato | Multimedia analysis for ecological data[END_REF]; Joly et al. (2014a). We therefore see an increasing interest in this trans-disciplinary challenge in the multimedia community (e.g. in [START_REF] Nilsback | Automated flower classification over a large number of classes[END_REF]; [START_REF] Goëau | Visual-based plant species identification from crowdsourced data[END_REF]; [START_REF] Cerutti | A Parametric Active Polygon for Leaf Segmentation and Shape Estimation[END_REF]; [START_REF] Mouine | Advanced shape context for plant species identification using leaf image retrieval[END_REF]; [START_REF] Kebapci | Plant image retrieval using color, shape and texture features[END_REF]; [START_REF] Hsu | An interactive flower image recognition system[END_REF]. Some recent studies have been done on closely related species (based on leaves analysis such as bean species [START_REF] Larese | Automatic classification of legumes using leaf vein image features[END_REF], or seed analysis such as in the studies of Acacia species and sub-species [START_REF] Sivakumar | Discrimination of acacia seeds at species and subspecies levels using an image analyzer[END_REF], Diplotaxis species [START_REF] Grillo | Seed image analysis and taxonomy of diplotaxis dc.(brassicaceae, brassiceae)[END_REF], or pea varieties [START_REF] Smykalova | Morphocolorimetric traits of pisum seeds measured by an image analysis system[END_REF]). [START_REF] Orru | Computer vision as a method complementary to molecular analysis: Grapevine cultivar seeds case study[END_REF] have meanwhile invested on the efficiency evaluation of image analysis methods (for morpho-colorimetric feature extraction) compare to molecular analysis on grape variety seeds in the aim to discriminate taxonomical groups. Nevertheless, most of the analysis actually realised are not able to deal with an important number of varieties, such as in our case. Beyond the raw identification performances achievable by state-of-the-art computer vision algorithms, recent visual search paradigms actually offer much more efficient and interactive ways of browsing large flora than standard field guides or online web catalogs [START_REF] Ellison | Nextgeneration field guides[END_REF]). Smartphone applications relying on such image-based identification services are particularly promising for setting-up massive ecological monitoring systems, involving thousands of contributors at a very low cost. A first step in this way has been achieved by the US consortium behind LeafSnap1 , an i-phone application allowing the identification of 184 common american plant species based on pictures of cut leaves on an uniform background (see [START_REF] Kumar | Leafsnap: A computer vision system for automatic plant species identification[END_REF] for more details). Then, the French consortium supporting Pl@ntNet (Joly et al. (2014a)) went one step beyond by building an interactive image-based plant identification application that is continuously enriched by the members of a social network specialized in botany. Inspired by the principles of citizen sciences and participatory sensing, this project quickly met a large public with more than 300K downloads of the mobile applications (Goëau et al. (2013a(Goëau et al. ( , 2014a))). A related initiative is the plant identification evaluation task organized since 2011 in the context of the international evaluation forum CLEF 2 and that is based on the data collected within Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. (2016). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022 In 2011In , 2012In and 2013 respectively 8, 11 and 12 respectively 8, 11 and 12 international research groups participated to this large collaborative evaluation by benchmarking their images-based plant identification systems (see Goëau et al. (2011a[START_REF] Goëau | The ImageCLEF 2012 Plant Identification Task[END_REF]Goëau et al. ( , 2013b) ) for more details). The data used during these 3 first years can be accessed online3 . Contrary to previous evaluations reported in the literature, the key objective was to build a realistic task very close to real-world conditions (different users, areas, periods of the year, important species number, etc.). The 2014th and 2015th edition of the task were organized within a newly created lab of CLEF called LifeCLEF4 and dedicated to the identification of living organisms in general (with an audio-based bird identification task and a video-based fish identification task in addition to the image-base identification task). Details of the participants and the methods used in the runs are synthesised in the overview working notes of the task (Goëau et al. (2014b[START_REF] Goëau | Lifeclef plant identification task 2015[END_REF]).

ACCEPTED MANUSCRIPT A C C E P T E D M A N U S C R I P T 4 Pl@ntNet.
From a computer vision and technological perspective, our work is more generally related to image classification. Most popular methods for this problem are typically based on the pooling of local visual features into global image representations and the use of powerful classifiers in the resulting high-dimensional embedded space such as linear support vector machines [START_REF] Lazebnik | Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[END_REF]; [START_REF] Perronnin | Improving the fisher kernel for large-scale image classification[END_REF]). The Bag-of-word representation (BoW) notably remains a key concept although the raw initial scheme of [START_REF] Sivic | Video google: A text retrieval approach to object matching in videos[END_REF]) is now outperformed by several alternative new schemes [START_REF] Lazebnik | Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[END_REF]; [START_REF] Jiang | Towards optimal bag-of-features for object categorization and semantic video retrieval[END_REF]; [START_REF] Perronnin | Fisher kernels on visual vocabularies for image categorization[END_REF]; van Gemert et al. ( 2010); [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF]). Its principle is to first train a so called visual vocabulary thanks to an unsupervised clustering algorithm computed on a given training set of local features. The produced partition is then used to quantize the visual features of a given new image into visual words that are aggregated within a single high-dimensional histogram. Partial geometry can be embedded in the image representation by using the Spatial Pyramid Matching scheme of [START_REF] Lazebnik | Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[END_REF]). As it relies on vector quantization, the BoW representation is however affected by quantization errors. Very similar visual features might be split across distinct clusters whereas more dissimilar ones might be affected to the same visual word. This results in both mismatches and potentially irrelevant matches.

To alleviate this problem, several improvements have been proposed in the literature. The first one consists in expanding the assignment of a given local feature to its nearest visual words [START_REF] Jiang | Towards optimal bag-of-features for object categorization and semantic video retrieval[END_REF]; [START_REF] Philbin | Lost in quantization: Improving particular object retrieval in large scale image databases[END_REF]; [START_REF] Van Gemert | Visual word ambiguity. Pattern Analysis and Machine Intelligence[END_REF]; [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF]). This allows reducing the number of mismatches without degrading much the encoding time. Other researchers have investigated alternative ways to avoid the vector quantization step, using sparse coding (Yang et al. ( 2009)) or locality-constrained linear coding (Wang et al. (2010)). Such methods optimize the affectation of a given local feature to a few number of visual words thanks to sparsity or locality constraints on the global representation. Another alternative is to use aggregation-based models such as the improved Fisher Vector of [START_REF] Perronnin | Fisher kernels on visual vocabularies for image categorization[END_REF] or the VLAD encoding scheme [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF]). Such methods do not only encode the number of occurrences of each visual word but also encode additional information about the distribution of the descriptors by aggregating the component-wise differences. When used with discriminative linear classifiers, such high-dimensional representations benefit of both generative and discrimination approaches leading to state-of-the-art classification performances on fine-grained classification benchmarks [START_REF] Gosselin | Revisiting the fisher vector for fine-grained classification[END_REF]). A radically different approach to image classification is the use of deep convolutional neural networks. Rather than extracting the features according to hand-tuned or psychovision oriented filters, such methods directly work on the image signal. The weights learned by the first convolutional layers allows to automatically build relevant image filters whereas the intermediate layers are in charge of pooling these raw responses into high-level visual patterns. The last fully connected layers work more traditionally as any discriminative classifier on the image representation resulting from the previous layers. Deep convolutional neural networks have been recently proved to achieve better results on large-scale image classification datasets such as ImageNet [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]) and do attract more and more interest in the computer and multimedia vision communities. A known drawback of Deep Convolutional Neural Networks is however that they require a lot of training data mainly because of the huge number of parameters to be learned. Their performances on fine-grained classification are consequently more controversial and they are still often outperformed by local features based approaches, as shown in our experiments. Besides, it is important to notice that they inspire the investigation of new deep learning models making use of more traditional visual features embedding methods (e.g. [START_REF] Simonyan | Deep Fisher networks for large-scale image classification[END_REF]).

Rice seeds dataset

Context

The French research unit on Genetic Improvement and Amelioration of Mediterranean and Tropical Plants (Agap) manages some major collections of plant genetic resources (lucerne, grapevine, rice, sorghum, cotton, groundnut, etc.), amounting to date to almost 60,000 accessions. These collections are made up of different categories of genetic resources: French national collections and heritage resources, original materials, species related to the cultivated or wild species worked on, resources of a scientific nature. In particular, these collections contain many materials intended for genetics and genomics studies, notably for the model species Medicago truncatula Gaertn., along with collections of rice insertion lines. The facilities managing these collections are labelled as biological resource centres through the ISO 9001 certification initiatives and the extension of standard Afnor NF 96900.

Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. (2016). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022

ACCEPTED MANUSCRIPT A C C E P T E D M A N U S C R I P T 5
The seed and biological resources laboratory (LSRG), in Montpellier, more specifically manages a large collection of 1717 varieties of Oryza sativa L. coming from more than 50 countries world wide. This large collection host nevertheless a small fraction of the 40 000 rices varieties used across the world. Each variety is preserved through one or few accessions, each accession being formed by a set of rice seeds kept in their hulls (from few tens of seeds to several thousands depending on the variety). The seeds are regularly renewed through germination allowing to keep a high average germination rate of about 90%. Before the image acquisition campaign initiated by the work presented in this paper, only few illustrations of the seeds in collection did exist.

Usage scenario

Automatically identifying the rice seeds varieties thanks to their visual appearance might be useful for several scenarios. First of all, a visual control is already performed by the operators in charge of managing seeds collections. This allows detecting mistakes or inconsistencies in the labelling of the new received accessions (i.e. new in-the-field samples). This process is however very imprecise as it relies on rough visual attributes of the seeds such as color:brown, shape:long or hairiness:hairy. Secondly, the automatic identification of the variety could be useful for entity resolution purposes (taxonomic name resolution). In many field contexts, the variety of a cultivated plant is actually only known through its vernacular name, i.e the local name given by the farmers of that region of the world. Mapping that names on a taxonomic referential is usually a very hard task and prevent from collecting accurate data from that cultures (which represent a potentially huge source of information).

Image acquisition

The image acquisition set up (illustrated in Figure 1) was a standard reflex camera fixed on a tripod at a fixed distance (DIST=10.4cm) from the petri box containing the rice seeds. The camera was equipped with a Compact Macro Lens (EF 50mm 1:2.5, Sigma), an MR-14 EX flash and a remote trigger. The images used for the training set (see next section for more details) were acquired by a different person, a different day, and with a different camera than the ones used as test images. This allows making the benchmark more realistic and closer to a real-world scenario where a person would have to reproduce the image acquisition protocol. For the selection of the varieties to be included in the dataset, we focused only on the ones having their rough visual attributes informed in the database (i.e. 550 varieties). We then selected 95 varieties through a randomized greedy algorithm guarantying that all visual attributes are represented in the dataset. The table 1 gives a synthesis of the visual and morphological diversity of this dataset, as for example the rounded shape of the variety 16, the brown color of the variety 23, or the presence of long hairs on variety 25.

Evaluation dataset

The final evaluation dataset to be shared with the scientific community, referred as CiradRiceSeed is composed of two

Grape Leaves dataset

Context

The French Vine and Wine Institute (IFV), through the intermediary of the National Plant Material Pole located in Southern France, is a vine selection establishment that coordinates clonal selection and conservation actions carried out throughout the country. It ensures the conservation of a quite unique collection in the world of hundreds of vine varieties at the "Domaine de l'Espiguette", and about 4000 clones. A large part of this material is inscribed in the national official catalogue5 . The main objectives of the IFV is to improve the potential of the wine plant material while maintaining all of the characteristics and identity of each variety. The IFV contributes to the conservation of all vine varieties grown in France by both the sanitary quality of plants, and the consideration of the diversity of varieties and types of wines. This activity allows a strong link between research and production. The IFV selection center is located in the "Domaine de l'Espiguette", occupying 80 hectares in the town of Grau du Roi, Languedoc-Roussillon. This site was chosen for the national conservatory clones because its soils consist only of pure Mediterranean sand, containing neither phylloxera or nematode vectors of fanleaf virus. More than 38 hectares are planted with clones collections. Based on DNA results, it is estimated that around 5000 varieties exist worldwide, and many of them are closely related [START_REF] This | Historical origins and genetic diversity of wine grapes[END_REF]). This important number, reflect the long history of domestication that probably started 8000 BP (before present). Morphological identification of this varieties is mainly done with adult leaves analysis, with some specific normalized leaf descriptors, as described in OIV (2009).

Usage scenario

Ampelography is a field of botany dedicated to the identification and classification of grapevines. Traditionally this has been done by comparing the shape, colour and texture of the vine leaves and grape berries. Recently, DNA fingerprinting has played a major role in the development of this discipline, nevertheless it has not permitted to widely extend grapevines identification to non-expert peoples. Automatically identifying the grape varieties thanks to the visual appearance of their leaves might be for this reason useful for several scenarios. First of all, it is important to remember that grape identification may have a very important impact, notably on financial aspect, as most of the wines in the world are based on the cultivars names. Vine variety identification is then practice, (i) by state agents in charge of the reglementation in vigor, (ii) nurserymans that want to confirm names of the commercialized material, (iii) winegrowers who want to identify old plants conserved in unused plots, etc... All theses contexts could get a significant benefit of an automated multimedia identification system.

Image acquisition

In the aim to evaluate such solution, we organised in June 2012 the collect of visual data on a selection of grape varieties growing at the Espiguette estate. 11 different people, each of them with different camera, were mobilized for this experimentation. They were organised by binomials, in order to collect by binomial about 60 different images of laves for each variety. Variety were selected according to their economic importance in French viticulture. Leaves were selected at the adult stage, in healthy condition, and the provided instructions were to (i) try avoiding strong lightness contrasts (with a part of the leaf in shadow and and other one in full sun condition) (ii) try to avoid the visual overlap between several leaves. Photographers were free to choose their camera parameters, but to pay attention to put one leaf at the image center, to photograph it in landscape format, to maximize it surface on the picture (in order to ovoid to take picture from a too long distance). Most of the pictures were taken directly on the living plants, but a part of them were took on a cut leaf put on the ground (with pure sand in this case).

Evaluation dataset

The final evaluation dataset to be shared with the scientific community, referred as IFVGrapeLeaves is composed of two parts, a training set and a testing set. The table 1 give a synthesis of the visual and morphological diversity of this dataset. It is then possible to see variation (i) between leaves in terms of leaf shape, lobes number, or tooth size on the leaf margin, (ii) but also in terms of background color, light conditions, or image quality. The full dataset is compound of 2037 images of 34 grape varieties. In order to divide it in train an test sets, we kept all the pictures of one photograph for train (generally the photograph who took the most important Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. ( 2016 number of images for or specific variety), and we putted all the others pictures of the binomial in test dataset. This avoids for the same variety, to have picture of the same author in both datasets. The varieties list is provide in table 3.

Baseline fine-grained image classification systems

We did experiment two families of image classification techniques that are known to provide state-of-the-art classification performances in fine-grained recognition challenges [START_REF] Gosselin | Revisiting the fisher vector for fine-grained classification[END_REF]; Joly et al. (2014b)) and that perform the best within the two last editions of the PlantCLEF international evaluation campaign Goëau et al. (2014b[START_REF] Goëau | Lifeclef plant identification task 2015[END_REF] (whatever the type of view or acquisition protocol e.g. leaf scans, in-the-field leaf pictures, flower pictures, bark pictures, etc.).

Fisher vectors & Support Vectors Machine

Fisher vectors (FV) were first introduced in image classification by [START_REF] Perronnin | Fisher kernels on visual vocabularies for image categorization[END_REF] and proved to be very efficient in the fine-grained classification task later on [START_REF] Gosselin | Revisiting the fisher vector for fine-grained classification[END_REF]). According to recent surveys such as [START_REF] Huang | Feature coding in image classification: A comprehensive study[END_REF], they are the best performing pooling strategy currently available. We will only recall here the main steps used to extract Fisher vectors, for detailed explanations of the theoretical derivation and for performance analysis we redirect the readers to [START_REF] Sánchez | Image classification with the fisher vector: Theory and practice[END_REF]. The pipeline for computing the Fisher vector describing an image consists in:

• Dense extraction of local features: descriptors, usually SIFT descriptors, are extracted on densely sampled overlapping patches on several scales.

• PCA transformation: the descriptors are then decorrelated and compressed using a Principal Component Analysis reducing the dimensionality (usually to 80 for SIFT features).

• Feature space density estimation: the distribution of features is modeled as a Gaussian Mixture Model (GMM) that is learned using the popular Expectation Maximisation algorithm. We thus obtain a probability distribution of the form of u(x) = K k=1 w k u k (x) where u k follows a Gaussian distribution, u k ∼ N(µ k , Σ k ) with µ k the mean and Σ k the covariance matrix which is diagonal because the features are decorrelated, and w k is the weight of the k-th Gaussian, they satisfy k w k = 1.

• Encoding: the features are encoded and pooled using

G µ k = 1 √ w k N i=1 γ k (x i ) x i -µ k σ k G σ k = 1 √ w k N i=1 γ k (x i ) √ 2 x i -µ k σ k 2 -1
where all the dicitions and squaring are element-wise operation and where x) . Theses 2K vectors

γ k (x) = w k u k (x) K k =1 w k u k (
Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. ( 2016 are concatenated to produce the final representation of dimension 2dK.

• Post-processing: the vectors are L2-normalized and then they are power-normalized using sign(x).|x| γ where the power parameter γ is found by cross-validation.

Usually, the supervised classification of Fisher Vectors is performed by using a linear classifier as it has been shown that using kernelized techniques on such high-dimensional features does not improve the performances. In our experiments, we used the Support Vector Machine (SVM) algorithm implemented within the LibLinear library [START_REF] Fan | Liblinear: A library for large linear classification[END_REF]).

Convolutional neural networks

Convolutional Neural Networks (CNN) have been mainly used since the 90's for their performances in digit classifica- 2012)).

In these experimentations we have used Caffe [START_REF] Jia | Caffe: Convolutional architecture for fast feature embedding[END_REF]), a Deep Learning Framework, allowing us to use state of the art CNN architectures and models. For the two datasets presented in this work, we have chosen in the Caffe model Zoo the "GoogLeNet GPU implementation" model, based on Google winning architecture in the ImageNet 2014 Challenge [START_REF] Szegedy | Going deeper with convolutions[END_REF], and we finetuned this model on our datasets.

The GoogLeNet architecture consists of a 22 layers deep network with a softmax loss as the classifier on top, and is composed of three "inception modules" stacked on top of each other. Each intermediate inception module is also connected to an auxiliary classifier during training, so as to encourage discrimination in the lower stages in the classifier, increase the gradient signal that gets propagated back, and provide additional regularization. These auxiliary classifiers are only used during the training part, and then discarded.

Experiments

Setup

For a fair evaluation of both visual classification methods, we first cropped and resized all images to a resolution of 256x256 pixels, that is the size of the images given as input to the first layer of the convolutional neural network as described in section 5.2. For the Fisher Vector's method, we used a sampling step of 3 pixels and 5 different scales to extract the patches on which the SIFT descriptors were computed. These descriptors were L2normalized and square-rooted before being reduced to 80 dimensions using PCA similarly to [START_REF] Gosselin | Revisiting the fisher vector for fine-grained classification[END_REF]. The Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. (2016). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022 2014), we applied a power-law normalization of the fisher vectors and did learn the value of the parameter α (i.e. the power value) for each dataset by cross-validation. The regularization parameter C of the SVM (which controls the trade-off between achieving a low error on the training data and minimising the norm of the weights) was also learned by cross-validation for each dataset. The cross-validation was performed using a stratified shuffling procedure with 5 iterations: during each iteration, 10% of the data was randomly sampled to constitute the validation set and the rest was kept for training while preserving the proportion of the different classes in each set.

ACCEPTED MANUSCRIPT A C C E P T E D M A N U S C R I P T
For the Convolutional Neural Network, we used several strategies that appeared to improve the results within our experiments. The first one (CNN 1) consisted in training the CNN described in section 5.2 from scratch, exclusively on the training data of each dataset and without the use of any external data. As for all other configurations, it makes use of the data augmentation technique implemented within Caffe library and consisting in cropping randomly a 224x224 pixels image, and eventually mirroring it horizontally. The second strategy (CNN 2) rather consisted in fine-tuning a previously trained CNN. We therefore started with the CNN described in 5.2, trained on the popular generalist ImageNet dataset. We then removed its top layers (the fully connected ones) and train this new model using the desired dataset.

The third strategy (CNN 3) is similar to the second one but integrates an additional data augmentation step aimed at improving the robustness of the classifier to the heterogeneous acquisition conditions and camera settings (such as the white balance, the use of the flash, etc.). For each training image, 8 new images were generated by applying a set of colorimetric transformations with randomized parameters, i.e. brightness & saturation modulation in the HSL color space (multiplier factor randomized between 0.8 and 1.2), and contrast modulation (multiplier factor randomized between 0.7 and 1.3). The fourth strategy (CNN 4) targets the same objective of increasing the robustness to colorimetric variations but through a different approach consisting in pre-processing all images (train and test images are both modified) with an histogram equalization (on each RGB channel) instead of using data augmentation. As this strategy provided the best results in the experiments, we also evaluated it for the Fisher Vector method so as to allow a fair comparison. For all the training strategies of the CNN, the batch size was fixed to 32, and the gamma value to 0.9. Other specific parameters can be found in table 5. During the training, at each iteration, the learning rate is updated according to : base lr * gamma #iterations/step Note that for the CNN1 configuration, for which the training phase is started from scratch, the base learning rate parameter (base lr) was multiplied by 100.

Results

Tables 6 and7 present the synthesized results of the experiments for respectively the CiradRiceSeed dataset and the IFVGrapeLeaves dataset. For each run, we provide the average success rate of the classifier on all images of the test set, considering either the best prediction for each test image (Top 1), or the 3 best predictions (Top 3), or the 5 best predictions (Top 5). A first global conclusion that we can derive from the comparison of the two tables is that the performances achieved on the CiradRiceSeed dataset are much better than the ones achieved on the IFVGrapeLeaves dataset. Whatever Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. ( 2016 On the other side, the best performances on the CiradRiceSeed dataset are very good reaching up to 88.67% good classification rate, whereas the number of classes is much higher (95 vs. 34). This shows that the scenario addressed through the IFV-GrapeLeaves dataset, i.e. the leaf-based identification of the varieties in the field, is a much more challenging problem than the scenario addressed through the CiradRiceSeed dataset (inlab identification). Note that this does not mean that the grape's varieties could never be automatically discriminated based on the visual appearance of their leaf. We actually know that it is to some extent possible thanks to existing morphological identification keys OIV (2009). But it shows that the acquisition protocol should be more constrained (typically by scanning the leaves instead of targeting in the field photographs) or enriched by the use of other traits (e.g. grapes images). We thus plan to enrich the IFVGrapeLeaves dataset with such new contents for further evaluations. On the other side the performances on the CiradRiceSeed dataset are clearly better to what we could expect. The visual variability across the 95 species is actually globally low and there are many varieties are almost impossible to distinguish to the naked eye. Back to the scenarios described within section 3.2 (i.e. labelling control and entity resolution issues), we can even argue that the achieved performances might be sufficient for a practical usage. Rising an alert when the label of a given accession does not belong to the 5 best automatic predictions might for instance be a very effective control tool (the Top 5 classification rate of the best experimented classifier is actually equal to 99.29%). Beyond these scenarios, the achieved performances show that the visual features learned by the classifiers are informative enough to characterize the morphological variability of the different varieties. This open the door to new phenotyping scenarios in the future, for instance if we consider combining such analysis of the visual appearance with the analysis of genomic data or the analysis of other traits. Let us now look more in details to the scores obtained by the different classification methods on the CiradRiceSeed dataset, starting with the different CNN strategies. The weak performances of CNN 1 first show, as one could expect, that the convolutional neural network is not able to learn effective visual features when it is trained on the targeted data only. It is actually well known that this technology requires much more larger training materials. The scores obtained by CNN 2 show that using a network trained beforehand (on a generalist dataset such as ImageNet) provides better performances, even if our fine-grained classification problem is much more specific. The achieved performances are however still lower than the ones achieved by the Fisher Vectors. The most likely reason, confirmed by the better performances of CNN 3 and CNN 4, is that the visual features trained by the CNN on ImageNet are still less robust than the hand-crafted SIFT features to the colorimetric variations occurring in our data (resulting from the heterogeneous acquisition conditions and camera settings of the different observers). The score of CNN 3 shows that using data augmentation as a way to increase the robustness to such transformations does improve the results. A simple histogram equalization (on each RGB channel) applied to all pictures as performed in CNN4 provides an even better improvement and allows the CNN to reach a very high classification score equal to 88.67%. This shows that the visual features trained by the CNN on ImageNet are generic enough to well characterize the pre-processed pictures even if their colorimetric distribution is far from the natural images of ImageNet.

Conclusion and perspectives

This paper addressed the problem of categorizing plant images at the variety level, i.e. at a finer taxonomic grain than state-of-the-art studies usually working at the species level. It therefore introduced two new evaluation datasets of agrobiodiversity interest, each being related to concrete scenarios on large-scale resources. A baseline experimental study was conducted on the two datasets to assess whether state-of-the-art classification techniques such as convolutional neural networks and fisher vectors are performant enough to answer the targeted use cases. It did show that the achieved classification performance is very different between the two problems. It is actually pretty bad for the grape leaves collection but much better in the case of the rice seeds collection for which the acquisition protocol was much more constrained and the morpholigical variability between varieties more visible. The conclusion we can draw from these raw results is that automatically identifying plant varieties might already be feasible for some specific scenarios and in controlled environments but that it is still an open problem in the general. In further works, we attempt to explore more in depth the confusion matrix of variety-level visual classifiers in order to correlate the visual similarity of the varieties with genomic data or with other phenotypical properties of the plants.

Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. (2016) , Kirschel, A.N.G., Taylor, C.E., Vallejo, E.E., 2008. Automated species recognition of antbirds in a Mexican rainforest using hidden Markov models. Journal of The Acoustical Society of America 123. doi:10.1121/ 1.2839017. Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y., 2010

Fig. 1 .

 1 Fig. 1. Image acquisition setup for the CiradRiceSeed dataset

  ). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05

  ). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022

  ). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022 approach and training strategy, the classification rate on the IFVGrapeLeaves dataset actually remains very low, with a maximum of 11.50% good classification rate.

  . Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022 Comment citer ce document : Champ, J., Lorieul, T., Bonnet, P., Maghnaoui, N., Sereno, C., Dessup, T., Boursiquot, J.-M., Audeguin, L., Lacombe, T., Joly, A. (2016). Categorizing plant images at the variety level: did you say fine-grained ?. Pattern Recognition Letters, 81, 71-79. DOI : 10.1016/j.patrec.2016.05.022

Table 1 . Image samples of the CiradRiceSeed dataset for 5 random varieties

 1

	Train	Test

Table 4 . Image samples of the IFVGrapeLeaves dataset for 5 random varieties

 4

	Train	Test

Table 2 . Variety list of the CiradRiceSeed dataset

 2

	ID	Variety name	ID	Variety name
	0	KANIRANGA	48	LAC 3 (1)
	1	DA 9	49	P 335
	2	DA 5	50	9 AB
	3	T 1	51	T 1 (IRR 74)
	4	CO 18	52	NHTA 13
	5	SURJAMKUHI	53	T 43
	6	DHOLA AMAN	54	DJUBUH
	7	JC 73-4	55	SAN THOU GEE
	8	RTS 5	56	KHAO TONG
	9	KUN MIN TSIEH HUNAN	57	MACK FAY DENG
	10	CO 25	58	KH CHETANG
	11	JHONA 349	59	KHAO XENG
	12	BAGUAMON 14	60	LA KHONE DENG
	13	MALAGKIT PIRURUTONG	61	9 D
	14	RTS 16	62	13 A
	15	PADI RAOEKANG	63	257
	16	HU LO TAO	64	483
	17	JC 91	65	KU 48
	18	LAMBAYQUE 1	66	IRAT 8
	19	ARC 10497	67	RAM TULASI SEL.
	20	KAKANI 2	68	HERIKA (LONG FORM)
	21	ACHILLE	69	CALORO
	22	63-105	70	SEMO
	23	ZAKPALE	71	ZAKPALE 4
	24	268 B / PR 22-3-2	72	KESSIRIME B
	25	377	73	KEDIALA OUADEO B
	26	TAITUNG 328	74	MAROVEL BEIDARI B
	27	KOIRAO BALEO	75	PAGAIYAHAN
	28	COLUMBIA 2	76	EH IA CHIU
	29	SHINHAKABURI (1)	77	DA 23
	30	ZAKPALE 3	78	DA 16
	31	HAO MET NHAY	79	RATHUWEE
	32	MACK HING HOM	80	JC 148
	33	CHIANAN 8 MA-3	81	JC 178
	34	SOMCAU 70 A	82	NAM SA GUI 19
	35	9 A	83	MOROBEREKAN
	36	P 269	84	ARC 10317
	37	8 A	85	DA 13
	38	3 E	86	BASMATI 370
	39	14 SEN 8 A	87	PRATAO
	40	AGNAKE G	88	TA MAO TAO
	41	DV 29	89	JC 1
	42	HAGINOMAE MOCHI	90	JC 92
	43	MALADY (MDG 439)	91	TSIPALA 421
	44	GOUE L	92	DOM ZARD
	45	GOUE CC	93	MEHR
	46	KOREMOUTOU L	94	ARELATE
	47	TELE 1

Table 3 . Variety list of the IFVGrapeLeaves dataset

 3

	ID	Variety name
	1	Arrufiac.B	18	Mauzac.B
	2	Bourboulenc.B	19	Merlot.N
	3	Cabernet.franc.N	20	Meunier.N
	4	Chardonnay.B	21	Mourvedre.N
	5	Chenin.B	22	Négrette.N
	6	Clairette.B	23	Petit.Courbu.B
	7	Colombard.B	24	Petit.Manseng.B
	8	Duras.N	25	Piquepoul.blanc.B
	9	Fer.N	26	Roussanne.B
	10	Gamay.N	27	Sauvignon.B
	11	Grenache.gris.G	28	Semillon.B
	12	Grenache.N	29	Syrah.N
	13	Gros.Manseng.B	30	Tannat.N
	14	Lauzet.B	31	Tempranillo.N
	15	Len.de.l'El.B	32	Ugni.Blanc.B
	16	Macabeu.B	33	Vermentino.B
	17	Marsanne.B	34	Viognier.B

Table 5 . Parameters of the train stage for the CNN base lr max iterations Step size CiradRiceSeed

 5 But since a few years, they appear to have now surpassed all state of the art methods for large-scale image classification(Krizhevsky et al. (

		0.00001	10000	1000
	IFVGrapeLeaves	0.0001	2000	200
	tion.

Table 6 . Classification rates of the evaluated runs on the CiradRiceSeed dataset

 6

	System used

Table 7 . Classification rates of the evaluated runs on the IFVGrapeLeaves dataset

 7

	System used

  . Localityconstrained linear coding for image classification, in: Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, IEEE. pp. 3360-3367. Yang, J., Yu, K., Gong, Y., Huang, T., 2009. Linear spatial pyramid matching using sparse coding for image classification, in: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEE. pp. 1794-1801.

http://leafsnap.com/

http://www.clef-initiative.eu/

http://publish.plantnet-project.org/project/plantclef

www.lifeclef.org

http://plantgrape.plantnet-project.org/

Acknowledgments

This work has been done throughout the ARCAD-FEDER project, supported by Agropolis Fondation, the French foundation for Agricultural Sciences and Sustainable Development, the European Union through the ERDF, and the Région Languedoc-Roussillon.