Greqam

Nicolas Jacquemet
email: nicolas.jacquemet@univ-paris1.fr

Stéphane Luchini
email: stephane.luchini@univmed.fr

Jason F Shogren

Adam Zylbersztejn
email: adam.zylbersztejn@malix.univ-paris1.fr

Document de Travail n°2011-49 Coordination with Communication under Oath

Keywords: Coordination game, Cheap talk communication, Oath. JEL Classification: C72, D83

Herein we explore whether the social psychology theory of commitment via a truth-telling oath can reduce coordination failure. Using a classic sequential coordination game, we ask all players to sign voluntarily a truth-telling oath before playing the game with cheap-talk communication. Three results emerge with commitment-via-the-oath: (1) coordination increased by nearly 50 percent; (2) senders' messages were significantly more truthful and actions more efficient, and (3) receivers' trust of messages increased.

Introduction

A coordination game captures the idea that value can be created when people coordinate their noncooperative actions in a strategic environment (see [START_REF] Schelling | The strategy of conflict[END_REF][START_REF] Cooper | Selection Criteria in Coordination Games: Some Experimental Results[END_REF]. If people coordinate their otherwise sovereign actions, they can achieve the first best equilibrium among many suboptimal ones. Coordination failure arises when people fail to realize the first best outcome due to strategic uncertainty -the risk associated with not knowing how your opponent will play the game (see, e.g., the survey by [START_REF] Devetag | When and why? A critical survey on coordination failure in the laboratory[END_REF]. Better communication between players is the most frequently prescribed institution to overcome coordination failure. Accumulated evidence shows (i) communication can improve efficiency; but (ii) many coordination failures still remain because the messages are non-binding cheap talk, i.e., there are no real economic consequences to players who do not act in accordance to the message (see, e.g., [START_REF] Cooper | Communication in Coordination Games[END_REF][START_REF] Parkhurst | Repetition, Communication, and Coordination Failure[END_REF].

Herein we explore whether we can create a coordination game environment in which people are committed to match words with actions. Following the social psychology theory of commitment, we use a truth-telling oath procedure introduced by Jacquemet, [START_REF] Jacquemet | Eliciting Preferences under Oath[END_REF] to strengthen the link between players' communications and actions: each player voluntarily signs an oath to tell the truth before he or she enters the lab. Commitment theory and experimental evidence has shown people are more likely to match actions with words when they have freely chosen to commit themselves to doing them through a prior action (see [START_REF] Joule | La soumission librement consentie[END_REF]. 1 Our hypothesis is that the prior action -an oath to tell the truth-will create commitment that will enhance the power of cheap talk communication thereby reducing strategic uncertainty and increasing coordination.

In an experimental design using the classic sequential coordination game of a sender and a receiver (see [START_REF] Selten | Reexamination of the perfectness concept for equilibrium points in extensive games[END_REF][START_REF] Rosenthal | Games of perfect information, predatory pricing and the chain-store paradox[END_REF], 2 our results are that coordination increases by over 50 percent within the oath treatment. The oath increases coordination by changing the behavior of the sender-his messages are now more truthful and focused on efficiency, and his choices are more efficient. These findings are in line with related economic experiments. People who make promises about future actions after pre-play communication are more likely to keep them (see e.g. [START_REF] Ellingsen | Promises, Threats and Fairness[END_REF][START_REF] Charness | Promises and Partnership[END_REF][START_REF] Vanberg | Why Do People Keep Their Promises? An Experimental Test of Two Explanations[END_REF]. The oath also changed the behavior of the receiver. Although many remained cautious, more receivers trusted the messages about how the sender will play the game. This finding reveals the relative nature of commitment and communication: nearly all senders committed to matching deeds with words, whereas fifty percent of the receivers still remained wary.

1 In so called low-ball experiments, for instance, subjects are asked their willingness to perform a target behavior before knowing the full costs of the target behavior. Data show that low-balling significantly increases compliance relative to cases in which individuals are asked to perform the target behavior directly. See [START_REF] Cialdini | Low-ball procedure for producing compliance: Commitment, then cost[END_REF] for the seminal experiment, [START_REF] Cialdini | Interpersonal influence[END_REF] for an overview and Joule and Beauvois (1998) for a comprehensive work on procedures that create commitment.

2 In sequential games, coordination failures arise from strategic uncertainty; See for instance [START_REF] Beard | Do People Rely on the Self-Interested Maximization of Others? An Experimental Test[END_REF]; [START_REF] Beard | Reliant behavior in the United States and Japan[END_REF]; [START_REF] Goeree | Ten Little Treasures of Game Theory and Ten Intuitive Contradictions[END_REF]; [START_REF] Jacquemet | What drives failure to maximize payoffs in the lab? A test of the inequality aversion hypothesis?[END_REF]. 2 Description of the experiment

The game

Coordination game. We rely on the normal form game presented in Table 1. 3 The game involves two players: player A, who chooses between actions R and L, and player B, who chooses between actions r and l. If R is chosen by player A, player B can maximize both players' payoffs by selecting action r. Alternatively, player B may choose action l, which slightly undermines her own payoff and furthermore sharply decreases player A's payoff. If, in turn, player A chooses L, then both players' payoffs do not depend on player B's decision -payoffs are the same whatever action is chosen.

From the theoretical standpoint, (R, r) is a Pareto dominant and a pure-strategy perfect Nash equilibrium. (L, l) also happens to be a Nash equilibrium, which is nonetheless imperfect -since it involves a weakly dominated strategy l from player B. Therefore, player A's best choice is R should player B be an own-payoff maximizer who seeks to use the dominant strategy r, and L otherwise. Furthermore, action L involves the least strategic uncertainty for player A. Depending on the stakes, player A's expected payoff from an unreliant decision L can dominate the expected payoff of a reliant decision R, even if the probability that player B uses dominated strategies is very low. 4

Pre-play communication. Prior to decision making, player B (hereafter, the sender) transmits a cheap talk signal to player A (hereafter, the receiver), indicating (truthfully or not) her intended 3 This payoff structure appears as Treatment 1 in the genuine experiment of [START_REF] Beard | Do People Rely on the Self-Interested Maximization of Others? An Experimental Test[END_REF]. Among several reported payoff schemes, this one induces the most striking behavior among participants: (i) the frequency of player As' unreliant choices related to this treatment is remarkable: 65.7%, and (ii) this is the only treatment where deviations from the dominant strategy by player Bs were observed (in 17% of all cases where player A made a reliant decision R). [START_REF] Jacquemet | What drives failure to maximize payoffs in the lab? A test of the inequality aversion hypothesis?[END_REF] report that roughly 50% of all player As' decisions are unreliant, and that player Bs use dominated strategy in about 20% of all cases, when this simultaneous-move game is repeated. [START_REF] Jacquemet | What drives failure to maximize payoffs in the lab? A test of the inequality aversion hypothesis?[END_REF] show that these results are robust to a change in the payoff structure that eliminates inequality in payoffs between players.

4 The cut-off probability of decision l by player B which makes player A indifferent between actions L and R equals 0.036.

decision. We are interested in a fixed-form communication, in which senders choose between three messages: "I will choose r", "I will choose l", "I will either choose l or r". The first two messages are informative, while the last one is uninformative in what concerns senders' intentions. The first of these signals has important theoretical properties: namely, it is self-committing, but not self-signalling [START_REF] Farrell | Cheap Talk[END_REF][START_REF] Aumann | Nash Equilibria Are Not Self-Enforcing[END_REF]. It is self-committing because, if trusted, it induces the receiver to choose R, to which the sender's best response is r -exactly as the message announces. However, it is not self-signalling, since it does not reveal sender's true intentions: she may use message "I will choose r" to persuade her partner to choose R, and take any action afterwards. Hence, communication in our game provides some reassurance to the receiver about the rationality of the sender. If individuals have a weak preference for truthfulness, senders always reveal their true intention and use message "I will choose r" to signal that they disregard the weakly dominated strategy. One-way communication between partners is generally found to improve efficiency in coordination games by reducing strategic uncertainty. For an insightful review of related literature see, e.g., [START_REF] Crawford | A Survey of Experiments on Communication via Cheap Talk[END_REF]; [START_REF] Ellingsen | When Does Communication Improve Coordination?[END_REF].

Experimental design

Baseline treatment (NoOath). Each experimental session consists of 10 rounds of the game presented in Table 1. Roles are fixed, so that each participant takes 10 decisions as either receiver or sender. After each interaction, pairs are rematched according to a perfect stranger round-robin procedure that guarantees that each pair of subjects may interact at most once throughout the entire experiment.5 For the sake of avoiding the end-game effects, we do not reveal to participants the exact number of rounds, but only inform in the experimental instructions that there will be several of them. At the beginning of each round, the sender sends one of the three messages to the receiver by clicking on a relevant button on her computer screen. We explain to the subjects that messages sent by senders do not affect their payoffs, and that they can be followed by any decisions. Once the receiver has confirmed receiving the message, the game moves to the decisionmaking stage, where the receiver chooses between R and L, while the sender -between r and l.

Instructions inform the participants that decision is first taken by the receiver, then by the sender, and the final payoff depends only on the receiver's decision should L has been chosen, and on both partners' decisions otherwise. At the end of every round, each subject is only informed about her own payoff. Next, individuals are informed that either another round is about to start, or that the experiment ends.6 Oath treatment. This treatment uses an identical experimental environment as the baseline treatment, except that each subject is asked to sign an explicit oath before entering the lab (see (see [START_REF] Jacquemet | Eliciting Preferences under Oath[END_REF][START_REF] Jacquemet | Referenda under oath[END_REF]. The oath is implemented as follows: each subject enters alone and is directed to a monitor at the front of the laboratory. The monitor then offers each subject a form to sign entitled "solemn oath" as presented in Figure 1. 7 The Paris School of Economics logo on the top of the form and the address at the bottom indicate that it is an official paper; the topic designation and the research number were added so to ensure the credibility. The monitor explicitly points out to the subject before she reads the form that she is free to sign the oath or not, and that participation and earnings are not conditional on signing the oath (subjects are, however, not informed about the topic of the experiment when asked to take the oath). The subject reads the form, which asks whether she agrees "to swear upon my honor that, during the whole experiment, I will tell the truth and always provide honest answers" (in bold in the original form). Regardless of whether the subject signs the oath, he or she is thanked and invited to enter the lab. The exact wording used by the monitors to offer the oath to respondents was scripted to standardize the phrasing of the oath. One monitor stayed in the lab until all subjects had been presented with the oath, to avoid communication prior to the experiment. Subjects waiting their turn could neither see nor hear what was happening at the oath-desk.

Experimental procedures

We run six experimental sessions (three for each experimental condition), each of them involving 20 subjects -10 receivers and 10 senders. 8 In the oath treatment, subjects first come one by one to the oath-desk and are exposed to the above described procedure. Although signing the oath was not mandatory, a large majority of subjects accepted to do so. Six subjects did not take an oath. This leads to a 90% acceptance rate. This is in line with previous experiments involving an oath procedure (see [START_REF] Jacquemet | Eliciting Preferences under Oath[END_REF][START_REF] Jacquemet | Referenda under oath[END_REF]. 9 We apply an intention to treat strategy, so that these six subjects are kept in the statistical analysis -given the number of observations this does not affect the statistical results, though.

In both treatments, subjects sign an individual consent form (mandatory and distributed publicly) before entering the lab. Participants are then randomly assigned to their computers and asked to fill in a small personal questionnaire containing basic questions about their age, gender, education, etc. The written instructions are then read aloud. Players are informed that they will play some (unrevealed) number of rounds of the same game, each round with a different partner, and that their own role will not change during the experiment. Before starting, subjects are asked to fill in a quiz assessing their understanding of the game they are about to play. Once the quiz and all remaining questions are answered, the experiment begins. Prior to the first round, players are randomly assigned to their roles -either receiver or sender. Subsequently, they are anonymously and randomly matched to a partner. Then, the sender sends a message to the receiver, after which they are both asked for their choices, R or L for receivers, and r or l for senders. At the end of 8 The data for the baseline treatment come from the communication treatment of [START_REF] Jacquemet | What drives failure to maximize payoffs in the lab? A test of the inequality aversion hypothesis?[END_REF].

9 This is also a standard acceptance rate for commitment experiments (see [START_REF] Joule | La soumission librement consentie[END_REF][START_REF] Burger | The Foot-in-the-Door Compliance Procedure: A Multiple-Process Analysis and Review[END_REF]. every round, each participant is informed solely about her own payoff. Once all pairs complete a round of the game, subjects are informed whether a new round starts. If this is the case, pairs are rematched. Otherwise, a single round is randomly drawn and each player receives the amount in Euros corresponding to her gains in that round, plus a show-up fee equal to 5 Euros. All sessions took place in the lab of University Paris 1 (LEEP) in between June 2009 and January 2012. The recruitment of subjects has been carried out via LEEP database among individuals who have successfully completed the registration process on Laboratory's website. 10 The experiment involved a total group of 120 subjects, 72 males and 48 females. 102 of them are students, among which 58 subjects are likely to have some background in game theory due to their field of study. 11 28% never took part in any economic experiment in LEEP before. Participants' average age is roughly 24. No subject participated in more than one experimental session. Each session lasted about 45 minutes, with an average payoff of approximately 12 Euros.

Results

Table 2 summarizes aggregate results by round from both treatments. We see our first key result: the likelihood that players coordinated to the optimal Pareto-efficient outcome increased by nearly 50 percent due to oath -to 75.0 percent optimal coordination with the oath from 52.7% without.

10 The recruitment uses Orsee [START_REF] Greiner | An Online Recruitment System for Economic Experiments[END_REF]; the experiment is computerized through a software developed under Regate [START_REF] Zeiliger | A presentation of Regate, Internet based Software for Experimental Economics[END_REF].

11 Disciplines such as economics, engineering, management, political science, psychology, mathematics applied in social science, mathematics, computer science, sociology, biology. Each type of suboptimal coordination -either (L; r) or (R; l)-was less likely in the oath treatment than in the baseline. Accounting for both optimal and suboptimal coordination, the commitmentvia-the-oath contributes to coordinating players' actions -Nash equilibria increase to 79.3 percent in the oath treatment from 66% in the baseline.

Result 1 Commitment via the truth-telling oath increased coordination on the socially optimal Nash equilibria to 75 percent of games from 53 percent without the oath.

The increase of coordination induced by the oath is explained by significant changes in individual behavior. First, we look at the patterns of communication by senders. Figures 2.a, 2.b and 2.c show the empirical distribution functions (EDF) of the number of messages sent by subjects in the role of senders over all 10 decision periods. Each bullet inside the graph indicates an individual, on the x-axis we present the number of messages she has sent (between 0 and 10), the y-axis represents the probability of observing an individual who has sent at most a given number of messages. Figure 2.a depicts the empirical distribution of informative messages ("I will choose r" and "I will choose l") in both treatments. In both cases communication is widely used by senders, since no subject has abstained from sending at least one informative message. The number of messages sent by each subject is relatively high, ranging from 5 messages to 10 messages in the baseline treatment and from 8 to 10 messages in the oath treatment. In particular, 14 subjects out of 30 (46.7%) in the baseline treatment send 10 informative messages, i.e. one such message in every round; 22 subjects out of 30 (73%) do so in the oath treatment. The difference is significant according to a bootstrap test for equality of proportions (p = .067). Overall, although senders in the oath treatment seem to use communication more often than senders in the baseline treatment, first-order dominance of the EDF of informative messages in the oath treament over informative

Number of decisions r

No oath Oath messages in baseline is not statistically significant (p = .118).12 Figure 2.b, in turn, presents the analogous EDF exclusively for messages "I will choose r". We find that subjects in the oath treatment display a stronger willingness to signal their credibility than subjects in the baseline treatment: for this class of messages, the EDF from the oath treatment first-order dominates the EDF from the baseline treatment (p = .035). Finally, Figure 2.c focuses on messages "I will choose l". These messages are seldom used -only 3 subjects out of 30 send them at least once in the oath treatment, whereas 12 of them out of 30 do so in the baseline; this difference is significant at the 5% level (p = .012). Altogether, we find that the individual communication behavior does not vary much in quantity -since senders use communication to a similar extent in both treatmentsbut does change in quality -in the oath treatment messages "I will choose r" are sent significantly more often, while messages "I will choose l" are sent significantly less often.

Result 2 The oath both (1) increases the informativeness of transmitted signals, and (2) changes the signal's structure by substantially increasing the use of reassuring announcements.

Figures 3.a and 3.b summarize both players' decision-making patterns. The degree of player receivers' reliance is larger in the oath treatment: out of 10 decisions, an average of 5.9 decisions R is observed in the baseline treatment, and 7.7 in the oath treatment. A mean difference bootstrap test indicates that the difference is significant (p = .029). Both EDF presented in Figures 3.a are similar on the low end, i.e for players who do not play R often, while discrepancies are visible on the upper end, where the oath treatment induces subjects to play R more often. We observe that 43.3% of receivers in the oath treatment choose to play R in all rounds while none of them do so in the baseline treatment (bootstrap proportion test: p = .000). The differences between the two EDF are highly significant: the EDF of receivers' behavior in the oath treatment firstorder dominates the EDF of receivers' behavior in the baseline (p = .003). Substantial differences also appear in senders' behavior, as revealed in Figure 3.b. Based on the empirical frequencies of decisions r, we find that the EDF from the oath treatment first-order dominates the EDF from baseline (p = .017). Therefore, in the presence of oath senders are more likely to cooperate with receivers. In particular, 80% of senders choose to play r in all 10 rounds in the oath treatment, while only 43.3% do so in the baseline. A bootstrap test for equality of proportions indicates that the difference is significant at the 1% level (bootstrap proportion test: p = .006).

Result 3 The oath shifts sender's actions towards more payoff maximizing decisions.

We now explore the link between messages and actions through the truth-telling behavior of senders. For sender we calculate the proportion of cases in which the action is coherent with the message (given it is informative) within all the cases in which an informative message is sent. We call this the truth-telling ratio; Figure 4.a presents the EDF from both treatments. Looking at the figure, we find striking evidence that misinforming one's partner about intended move is substantially more widespread without the oath. First, 26 of 30 senders (87%) always reveal their actual intentions when sending an informative message in the oath treatment relative to only 17

Trust ratio

No oath Oath of 30 player Bs (57%) in the baseline treatment. The difference is statistically significant with p = 0.016 according to a bootstrap proportion test. Second, the EDF from the oath treatment significantly first-order dominates the EDF from baseline (p = 0.067). In Figure 4.b, we represent the same truth-telling ratio using only messages "I will choose r". Here the difference between the two treatments is more ambiguous. The EDF of the truth-telling ratio from the oath treatement still first-order dominates the EDF from baseline, but the result is not statistically significant (p = 0.586).

Result 4 The oath improves the truthfulness of announcements.

Last, in Figures 5.a and 5.b, we look at receivers' perception of information obtained from their partners, conditional on the presence of oath. More precisely, for each individual in the role of the receiver we calculate the proportion of cases in which she gives the best response to an informative signal (that is, when the receiver plays R should the signal from the sender announce action r, or receiver's move L follows message announcing move l by sender) among all the cases when the received message happens to be informative. We call this proportion the trust ratio, and present its EDF in Figures 5.a and 5.b. The first figure suggests that oath has an influence on how receivers react to messages: the EDF obtained in the oath treatment first-order dominates the EDF obtained in the baseline (p = .035). This remains true when restricting attention to messages "I will choose r", as it can be seen in the second figure. The differences in behaviors are less pronounced but still significant (p = .067).

Result 5 The oath improves the receiver's trust toward the announcement.

Conclusion

Overcoming coordination failure through economic design can be enhanced with more insight into non-monetary commitment devices. If people can freely choose their coordination partner, they can choose people they already know are trustworthy, so commitment is guaranteed and coordination is efficient (see [START_REF] Riedl | Efficient Coordination in Weakest-Link Games[END_REF]. But in many cases, people must coordinate with people they do not know. Now the commitment to backing words with deeds is unclear, which leads to more coordination failure. This paper investigates whether the social psychological idea of commitment via a truth-telling oath can improve how communication reduces strategic uncertainty, thereby increasing coordination on efficient outcomes. We create commitment by having our players sign a voluntary oath to tell the truth prior to entering the laboratory [START_REF] Jacquemet | Eliciting Preferences under Oath[END_REF]. Our results suggest that the truth-telling oath significantly increased optimal coordination by nearly 50 percent.

But the oath had an asymmetric effect on behavior. A sender was more likely to send informative and reassuring messages, and then back up his words with actions. In contrast, a receiver was less likely to change his behavior. A few more receivers trusted the messages received under oath than without, but still half of them remained wary. The oath appears to create stronger commitment in senders' signals/actions than in receivers' responses. Commitment theory reveals the limits of communication as a coordination device. Commitment increased the efficiency of communication by making the supply more trustworthy and reliable. But improved supply-side communication did not translate into 100% efficiency in the coordination game itself. This suggests limits still exist to communication from the demand side.

Figure 1 :

 1 Figure 1: Oath form used in the experiment

Figure 2 :

 2 Figure 2: Communication behavior of senders by treatment (a) Informative messages

Figure 3 :

 3 Figure 3: Decisions of receivers and senders by treatment

Figure 4 :

 4 Figure 4: Truthfulness of senders by treatment (Empirical distribution function)

Figure 5 :

 5 Figure 5: Messages trusted by player As (Empirical distribution function)

Table 1 :

 1 The experimental game

		Sender
		l	r
	Receiver	L (9.75; 3) (9.75; 3) R (3; 4.75) (10; 5)

Table 2 :

 2 Aggregate results

						Round					Overall
		1	2	3	4	5	6	7	8	9	10
	No oath
	Coordination (L, l) or (R, r) 56.7 46.7 60.0 63.3 66.7 70.0 80.0 66.7 66.7 83.3	66.0
	Efficient outcome (R, r)	43.3 26.7 50.0 50.0 50.0 60.0 66.7 60.0 56.7 63.3	52.7
	Miscoordination (L, r)	36.7 43.3 40.0 33.3 16.7 20.0 20.0 23.3 30.0 10.0	27.3
	Miscoordination (R, l)	6.7 10.0	0.0	3.3 16.7 10.0	0.0 10.0	3.3	6.7	6.7
	Oath
	Coordination (L, l) or (R, r) 73.3 80.0 83.3 86.7 80.0 80.0 76.7 80.0 83.3 70.0	79.3
	Efficient outcome (R, r)	70.0 70.0 73.3 80.0 76.7 73.3 76.7 76.7 83.3 70.0	75.0
	Miscoordination (L, r)	26.7 20.0 16.7 10.0 16.7 16.7 23.3 20.0 13.3 20.0	18.3
	Miscoordination (R, l)	0.0	0.0	0.0	3.3	3.3	3.3	0.0	0.0	3.3 10.0	2.3

Note. Columns 1-10 summarize the frequencies of outcomes (defined in rows) as % of all outcomes observed in each round of a given experimental treatment. The last column provides overall results.

This procedure, also known as rotation matching, is optimal for our experimental design: for a given number of players and the one-shot nature of each interaction between subjects, it maximizes the number of rounds. See[START_REF] Kamecke | Rotations: Matching schemes that efficiently preserve the best reply structure of a one shot game[END_REF] and[START_REF] Duffy | Do Actions Speak Louder Than Words? An Experimental Comparison of Observation and Cheap Talk[END_REF] for a related discussion.

The material used in the experiment is available from the authors upon request.

The word "oath" is written on the form and read by the subject, but never said aloud.

This result comes from a bootstrap version of the univariate Kolmogorov-Smirnov test. This modified test provides correct coverage even when the distributions being compared are not entirely continuous and, unlike the traditional Kolmogorov-Smirnov test, allows for ties (see[START_REF] Abadie | Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Model[END_REF][START_REF] Sekhon | Multivariate and Propensity Score Matching Software with Automated Balance Optimization[END_REF].

* We thank Tore Ellingsen and the participants of the Environmental & Resource Economics Seminar at the University of Manchester School of Social Science for valuable comments, and to Maxim Frolov and Ivan Ouss for their assistance in running the experiments. Financial support of the chair "Economie Publique et Développement Durable" (Aix-Marseille University) is gratefully acknowledged. Nicolas Jacquemet acknowledges the Institut Universitaire de France. Stéphane Luchini thanks the MIT Sloan School of Management for its hospitality. Adam Zylbersztejn is grateful to the Collège des Ecoles Doctorales de l'Université Paris 1 Panthéon-Sorbonne, the Alliance Program and the Columbia University Economics Department for their financial and scientific support. † Paris School of Economics and University Paris I Panthéon-Sorbonne. Centre d'Economie de la Sorbonne, 106


Julien Hirel

Philippe Gaussier
email: gaussier@ensea.fr

Mathias Quoy

Biologically inspired neural networks for spatio-temporal planning in robotic navigation tasks

In this paper we present a biologically-inspired model of spatio-temporal learning in the hippocampus and prefrontal cortex which can be used in tasks requiring the behavior of the robot to be constrained by sensory and temporal information. In this model chains of sensory events are learned and associated with motor actions. The temporality of these sequences is also learned and can be used to predict the timing of upcoming events. The neural network acts as a novelty detector and can modulate the behavior of the robot in case its actions do not have the expected consequences. The system is used to solve two different robotic navigation tasks involving an alternation between random exploration, goaldirected navigation and waiting periods of various lengths.

I. INTRODUCTION

The hippocampus is an essential part of the brain when it comes to spatial navigation. Its role in navigation has been brought to light with the discovery of place cells [START_REF] O'keefe | The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat[END_REF], neurons firing at specific locations in the environment. More recently, some results have suggested that the hippocampus encodes not only places but also accessible paths [START_REF] Alvernhe | Different ca1 and ca3 representations of novel routes in a shortcut situation[END_REF]. One particular experimental protocol has allowed to bring to light the spatio-temporal role of the hippocampus in goaloriented navigation. Solving this task for rats is done in 3 steps: i) navigate toward a goal zone ii) wait without moving for 2 seconds before a food pellet is dropped from the ceiling iii) explore the area to find the food. Physiological recordings have provided evidence of spatial but also temporal information about the timing of the waiting period being present at the level of the hippocampus [START_REF] Hok | Goal-related activity in hippocampal place cells[END_REF]. Burton et al. [START_REF] Burton | Lesion of the ventral and intermediate hippocampus abolishes anticipatory activity in the medial prefrontal cortex of the rat[END_REF] suggest that this activity propagates to the prefrontal cortex and is necessary for the animal to solve the task with good performances. The hippocampal spatiotemporal activity seems to be a key component in the time estimation system allowing the animal to predict the delayed consequences of its own actions.

The place preference task also presents some interesting challenges for robotic navigation. It requires goal-oriented navigation skills, a time-estimation mechanism linked with action selection and the ability to alternate between different behavioral strategies (move toward a known location, wait, explore) depending on the phase of the experiment. The time estimation is crucial for the animal to be able to resume its motion at the end of the waiting period, especially in the absence of the rewarding stimulus. To do so, some knowledge of the length of time spent waiting before obtaining the food must be acquired. The animal must then be able to adapt its behavior depending on the correspondence between its temporal predictions and what it observes. Integrating a temporal aspect is crucial when planning trajectories in a dynamic environment, especially for multi-robot systems where each agent must plan to avoid static obstacles as well as other robots [START_REF] Quottrup | Multi-robot planning : a timed automata approach[END_REF]. Predicting when to expect sensory events and detecting their absence could help a robot know when to "replan". The learned temporal characteristics of the task could thus be used to avoid the computational cost of constant replanning or the need for user-defined heuristics to control when to replan [START_REF] Liu | Adaptive replanning in hard changing environments[END_REF], [START_REF] Philippsen | Mobile robot planning in dynamic environments and on growable costmaps[END_REF].

In an attempt to have a better understanding of the mechanisms involved in spatio-temporal processing in the mammal brain and to apply this knowledge to robotic applications, we developed a biologically-inspired neural network model. We discuss how such a neural architecture can give a robot the ability to control its behavior depending on the timing of perceived sensory events. The robot can learn temporal predictions about the delayed consequences of its actions. As a result, unexpected consequences can be detected and the behavior of the robot adapted thereof. We show how we can use our model to solve the place preference task with a mobile robot. We discuss how this system can be generalized to perform well in any task requiring both navigation and time-estimation skills. An example is given by training the robot to perform a patrol with several checkpoints where it must stop for various periods of time.

II. MODEL

Banquet et al. [START_REF] Banquet | Space-time, order and hierarchy in fronto-hippocamal system: A neural basis of personality[END_REF] presented a theoretical model which predicted the need for transition cells in order to plan with a cognitive map. Transition cells were capable of learning temporal relations between past and present sensory signals. This learning consequently allowed transition cells to predict future sensory events. In [START_REF] Andry | Learning and communication in imitation: An autonomous robot perspective[END_REF], sequences of motor actions were learned and reproduced by learning the timing between motor commands. In [START_REF] Cuperlier | Neurobiologically inspired mobile robot navigation and planning[END_REF], transitions between visual place cells were learned and used in navigation with a planning strategy based on a cognitive map of transitions. However no timing was learned for the spatial transitions between places. The cognitive map provided information about the available paths between spatial states but not about the time needed to shift from one state to the next. In an attempt to obtain a unified model for different robotic applications, a learning equation (1) based on a Normalized Least Mean Square (NLMS) algorithm was developed. It allows the model to learn and predict various types of upcoming signals, coming from several modalities (visual and proprioceptive) [START_REF] Hirel | Model of the hippocampal learning of spatio-temporal sequences[END_REF].

Fig. 1 shows the neural network for the learning of the associations. Multi-modal state information is computed from sensory input coming from visual, auditory, vestibular systems etc. A winner-take-all (WTA) competition takes place between the state neurons. Any change in the winning state triggers the spectral timing memory. Each state has a corresponding battery of neurons in the memory. These neurons are activated with different timings, forming a spectral decomposition of time [START_REF] Grossberg | Neural dynamics of adaptive timing temporal discrimination during associative learning[END_REF]. Only one battery can be activated at any time, and its activity gives information about the time passed since the last sensory event (i.e. a change in the sensory state competition). When a new sensory event occurs, the association of this new information (coming from the state neurons) with the spectral timing memory of the last sensory event is learned by state prediction cells. As a result, the system learns to connect the first event with the second and memorizes the timing between the two events. Later when the first event is experienced again, it activates its corresponding spectral timing neurons and predictions for the second event will arise from transitions cells. The shape of the prediction activity looks like a Gaussian curve which reaches its maximum value just before the expected timing of the next event. The equation used for learning the sensory association shown in fig. 1 is the following:

dW i j (t) dt = α • η i (t) • (X US i (t) -X i (t)) • X CS j (t) ∑ X CS k (t) 2 + σ 1 (1)
η i (t) = |X US i (t) -m US i (t)| + σ 2 (2)
m US i (t) = γ • m US i (t -dt) + (1 -γ) • X US i (t) (3)
X i is the activity of neuron i, α is the learning rate, η i a learning modulation , X US i is the unconditional signal for the LMS. σ 1 is a small value used to avoid the divergence of the synaptic weights for very low memory values. m US i is a sliding mean of X US i , σ 2 is a low value setting a minimal learning rate and γ a parameter controlling the balance between past and current activities in the computation of the sliding mean.

Sensory events can include the recognition of a new visual place cell, the detection of a color on the floor signaling an important place, the sound being made by the release of the food reward or a change in the proprioceptive information coming from odometry. At any time transition cells can predict reachable sensory states, depending on the current state. The sensory transitions can then become motor

Visual place

Food found

Food dispenser

Goal location

Colored zone

Food dropped

Sound

Planning Wait Exploration Fig. 2. The place task as a sequence of behavioral strategies cued on multi-modal perceptive events.

transitions when associated with motor actions. Here, the motor actions of the robot can be controlled in terms of orientation and speed, allowing it to control whether or not it is moving. The place preference task can then be characterized by a series of behaviors controlled by sensory events (fig. 2). The prediction of the perception of the sound signaling the reward has to be associated with the immobility of the robot, since the reward will only be delivered after a certain period of time spent on the goal location. Other purely spatial predictions are associated with directions to take with a particular speed (a default moving speed). Motor actions associated with each transition are computed from path integration, using the odometry of the robot. A navigation strategy (e.g. planning with a cognitive map) can use the information about available transitions and bias the selection of a transition and its associated action [START_REF] Cuperlier | Neurobiologically inspired mobile robot navigation and planning[END_REF]. The cognitive map forms a topological representation of the environment by linking subsequent transitions. In the map, goals are represented by rewarding transitions (e.g. arriving at a food or water source etc.). Also, each received reward (e.g. food) has an associated drive (e.g. hunger) controlling the tendency of the robot to look for that reward. The reward signal satisfying a particular drive triggers the association in the cognitive map between this drive and the last transition performed. The drive activity can then be propagated through the map (with synaptic weights < 1) using a max operator (in a fashion similar to the Bellman-Ford algorithm). The gradient of activity in the map gives information about the distance to rewarding transitions.

Another pathway associates transition activity with satisfaction signals (fig. 3). The predictions emitted by these neurons correspond to actions which should lead to a satisfying state. The equations for the learning of the association (4) and the satisfaction prediction activity (5) are: Transitions predicting satisfying stimuli and neutral stimuli can thus be differentiated. A neural network is used to memorize the maximal activity of the bell-shaped predictions. When the prediction activity falls back under a fraction λ of this maximum, it means that the expected timing of the satisfaction has passed and it will probably not happen. The parameter λ controls the tendency of the system to wait more or less for expected satisfactions after their predicted timing. The closer λ is to 1, the faster the system will signal the non-occurrence of a prediction. As long as the prediction activity is close to the max, the non-occurrence signal is inhibited but when the difference becomes significant non-occurrence neurons are activated. When the prediction stops, the memories of the maximal prediction activity is reset by inhibitory connections. The non-occurrence signal is used to inhibit the drive that was supposed to be satisfied. α is a learning rate, γ a decay rate and θ an activity threshold.

dW T P i j (t) dt = α • (1 -W T P i j (t)) • X T j (t) • X S j (t) -γ • X T j (t) (4) X P i (t) = ∑ k W T S ik (t) • X T k (t) -θ (5)
The system also detects when a predicted drive satisfaction does not occur and consequently inhibits the corresponding drive. This will modify the behavior of the robot which will attend to other objectives (i.e. seek to satisfy other active drives or explore its environment). By inhibiting the drive we tell the robot that it can currently not be satisfied. By doing so we allow the robot to resume its movement, even in the absence of any explicit stimulus terminating the waiting period. If no drive is active then there is no activity in the cognitive map and no particular transition is selected. As a result, the transition architecture does not select any particular action to perform. A small amount of noise in the action selection system will lead to the selection of a random direction. The robot will thus start a random exploration behavior. Inhibiting the different drives is a way to modulate the behavior toward different goals and to start exploring the environment if all those goals have been fulfilled or are not reachable at this time. Drives build up over time (with a speed regulated by the ε parameter). This controls the frequency with which the robot needs to fulfill each objective. The complete model is shown in fig. 4.

III. ROBOTIC EXPERIMENTS

A. Place preference task

Experiments are conducted on a Robosoft robulab 10 mobile robot1 in an enclosed environment of 5x5m. A colored zone of 1x0.5m marks the goal location. A color detector placed under the robot allows the detection of the goal. Ultrasound sensors act as proximity detectors to provide lowlevel obstacle avoidance. In a preliminary phase, the robot is allowed to explore the environment for 40 minutes. A camera mounted on a pan-tilt system allows the autonomous learning and recognition of visual landmarks from the log-polar transform of 2D local views centered on the maxima of a DOG (Difference of Gaussians) filtering performed on the gradient of the image [START_REF] Cuperlier | Neurobiologically inspired mobile robot navigation and planning[END_REF]. New visual place cells, represented by a constellation of visual landmarks, are autonomously learned based on a minimum activity threshold. Sensory events are characterized by changes in spatial places (based on visual information), detection of a colored zone or perception of a simulated sound representing a reward. Through random exploration, the cognitive map linking the different states is learned (fig. 5). The robot learns a topological representation of its environment and how to reach any state using the shortest path possible. The robot is controlled by the system in terms of linear and angular velocity.

The second step is to teach the robot how to perform the place preference task. The training of the robot can be facilitated by an intuitive and natural interaction with the human. In order to train the robot to wait for the sound we first need it to go to the goal location. Even though we could just wait for the robot to reach the goal by random exploration, we use a leash strapped to an artificial neck to guide the robot. This system provides an easy way for a human to interact with the robot and guide it to a location of interest. For the experiment, we set a 7s delay during which the robot has to stay in the colored zone to hear a simulated sound signaling the delivery of a food reward. The length of the delay is chosen to be significantly longer than the time it takes for the robot to cross the goal zone at its default speed. On its own, the robot cannot learn that being on the goal location could lead to receiving a reward since its default behavior (i.e. to move) does not permit it to stay long enough in the colored zone. After having guided the robot to the colored zone, we then use the low-level obstacle avoidance to make the robot stop, by placing ourselves in front of its ultra-sound sensors. Once again, this provides a natural way to interact with the robot and make it stop moving. Extra human-robot communication devices are not needed. The robot will eventually hear the sound after 7s of waiting. Proprioception about its current speed (i.e. no movement) is associated with the prediction of the sound.

In the original experiment with the rats, finding the food pellet is the actual reward and waiting at the goal location The circles represent the center of the place cells and the lines transitions between places. b) Trajectories taken by the robot while planning its path to the goal using the cognitive map. Starting points are indicated by crosses and the goal by a red rectangle. The paths are not optimal (i.e. straight lines) because the algorithm selects transitions based on their distance to the goal in terms of number of nodes in the graph, which can lead to some sub-optimal decisions in the Cartesian space. Since the competition only selects one transition, the robot follows the edges of the transition graph. Smoother trajectories could be obtained by increasing the number of place cells, at the expense of memory, computational power and an increased learning time. Another solution could be to have a soft competition between transitions to select some average direction predicted by several good transitions.

is just a necessary step to obtain it. However after repeated trials, the sound of the food pellet being releasing after the delay must become associated with the delivery of the food. In this robotic experiment, we assume that this conditioning has already happened and that hearing the sound represents by itself a goal that needs to be satisfied. Whereas the reward should be the actual food pellet, we consider that hearing the sound is a goal in itself and has a corresponding drive that needs to be satisfied. Upon perception of the sound, the corresponding drive is inhibited and the robot starts to explore. The drive level then builds up over time until it reaches a value high enough to direct the robot toward the goal again. The time needed by the drive to reach that value simulates the time needed by the rat to find the food. The robot needs to find its way back to the goal autonomously, using the cognitive map (trajectories in fig. 5). The arrival on the goal triggers the prediction of the arrival of the sound and activates the corresponding action (stop moving). Each success reinforces this behavior. Fig. 6 shows the speed profiles and sound prediction activity around and at the goal location for rewarded and extinction trials. It shows that the model successfully predicts the perception of the sound and controls the movement of the robot until then. In extinction trials, the prediction system is Fig. 6. Speed profile and sound prediction activity at the goal. a) Rewarded trials. The prediction has a bell shaped activity with a maximum value preceding the expected timing of the satisfying stimulus. Once the sound is heard, the prediction stops and other transition cells start predicting the next stimuli. b) Extinction trials. The reward is not given, so no sensory stimulus signals the end of the 7s delay. Shortly after the prediction has peaked, the non-occurrence neurons are activated and start inhibiting the drive controlling the goal-oriented behavior and the robot switches to random exploration. The system has a little tolerance to delays in the occurrence of a predicted event and waits for a few seconds more after the end of the 7s waiting period before the movement of the robot is resumed. Without the non-occurrence system, the robot would wait indefinitely for the sound. necessary for the robot to resume its movement after the end of the 7s delay. Without it the robot would wait forever for the sound, without any clue as to when it should be expecting it, and when it can consider that it will not be delivered. Additionally, the reward timing can later be modified to longer or shorter delays and the system will learn to adapt the timing of its predictions.

B. Automated security patrol

Our model is able to perform the place preference task and gives good explanatory and predictive information about how this task is solved by rodents. However its performances are not limited to this task and they can be generalized to any navigation task involving sequences of sensory-motor behaviors with temporal constraints. One application where this type of behavior could be needed is automated security patrols. In this case the robot needs to navigate toward a sequence of checkpoints. It may have to stop at some of the checkpoints and perform some kind of behavior like looking around for intruders, for a precise amount of time, before proceeding to the next checkpoint.

Using the same neural architecture as in the place preference task, we now teach our robot to follow a given trajectory. During the first phase of the learning, the robot has absolutely no knowledge of its environment and is guided by a human using the leash. The trajectory forms a loop that the robot must complete. Place cells are autonomously learned based on a minimum activity threshold and the actions to move from one place to the next are associated to the transition between these places. When the loop is completed and the robot is back at its starting point, the leash is dropped and we let the robot reproduce the trajectory to verify that the loop was correctly learned. Because no checkpoint has been learned yet, no drive is active in the cognitive map. However the robot only knows one path and will select the Trajectories of the robot during the task. The dark trajectory corresponds to the learned path, when guided by the human. The light trajectories are the autonomous reproduction. Due to the size of place fields, the reproduced loop can be shifted compared to the original learned trajectory. This is a consequence of the prediction system that begins to predict the movement to reach the next place once it enters a new place. Since place cells can have broad place fields, this prediction can occur before the robot reaches the center of the field, which is where the robot was originally guided. The architecture offers a trade-off between the computational power required and the precision of the reproduction in the number of place cells learned on the trajectory. Indeed only movements between place cells are learned, so too few place cells will lead to a very coarse reproduction of the learned trajectory, whereas a fine reproduction can be achieve with many place cells but will lead to a longer learning phase and a memory and computationnally expensive system. only available transition in each place (the noise on the action selection system has been set to a minimal level for this task).

During the second reproduction of the patrol, the human places himself in front of the robot to make it stop. The operation is repeated at 3 different locations. The robot associates these checkpoints with the nearest place cell. The spatial accuracy of the checkpoints depends on the distance between learned place cells. The human then removes himself from the front of the robot and lets it resume the patrol. Since no visual or auditory stimulus marks the end of the waiting delay, our system needs additional information to characterize its state during and after the waiting period. We add proprioceptive information coming from odometry to the multi-modal state integration. A sensori event is triggered each time the robot resumes its movement. So in addition to transitions predicting the next place from the current checkpoint place, the robot can predict a perceptive event signaling a change from immobile to mobile. Since the robot has been immobile for the duration of the transition, the action associated is to wait without moving. The dura-tion of the pause is learned by the system as a temporal prediction. Each checkpoint is associated with a particular drive (representing the need of the robot to inspect each checkpoint) that increases over time. When the robot arrives on a checkpoint place, if the corresponding drive is strong enough, the cognitive map will bias the selection of the waiting transition rather than the transition to the next place and the motion will stop. Once the prediction has peaked, the drive is inhibited by the non-occurrence system and the transition to the next place will win the competition and trigger its associated action: to move to the next place. The robot thus proceeds to the next phases of the patrol. Fig. 7 shows the learned cognitive map and the trajectories for the patrol during learning and reproduction phases. In addition to the results in the previous experiment showing that a robot can learn to efficiently reach a goal by planning its path, we see here that the cognitive map can be used with good performances for the fast learning and reproduction of a trajectory. Here the learned cognitive map is rather incomplete, the robot only knows the path it has been taught. If the robot is kidnapped, by taking it and placing it far from the patrol, it will be lost. It can however be taught with the leash how to return to the patrol path and will remember it. Kidnapping the robot by placing it somewhere else on the patrol path will lead to the robot resuming the patrol and heading toward the nearest checkpoint with a high drive value. If we let the robot explore the whole environment and build a comprehensive map, it can use more direct paths between each of the checkpoints. No specific order for the checkpoints is defined in the system. The cognitive map strategy solely optimizes the paths of the robot to satisfy the strongest drives. It also takes into account the distance to the corresponding checkpoints. The frequency with which each checkpoint is visited could be controlled by adjusting the speed with which each drive rises after having been satisfied.

Table I shows the duration of the immobility of the robot at the three checkpoints. The learned duration corresponds to the time spent by the human in front of the robot. The reproduction values are calculated over 5 loops. The robot manages to reproduce the waiting behavior that was learned at the 3 checkpoints and shows consistency in the length of the reproduced waiting periods. The robot has correctly learned to suppress its movement upon entry on a particular place and resume it after a particular length of time that can be different from place to place. However another observation is that the period during which the robot is not moving can be different during the learning and reproduction phases. This is due to two reasons: (i) Our system learns to associate a place with a waiting period of a particular duration. The duration learned by the system is the length of time between the entry on the place and the resumption of movement. When the training is done by the human, all the time spent in a place by the robot before the human stops it is added, for the learning, to the time spent waiting. To solve this issue, our system could include an additional state marking the beginning of the immobility. This would allow the learning of a timed transition between the entry on a place and the inhibition of the movement. The robot could then learn not only to wait at a particular place, but also during any movement between two places.

(ii) The prediction activity has a peak preceding the exact timing of the expected resumption of movement, learned during the first loop. However the non-occurrence neurons are only activated when the prediction activity falls back to lower values. This period between the peak of activity representing the learned timing and the drive inhibition signal, in addition to the time needed by the system and robot to transmit the motor command to the wheels, delays the restarting of the robot. Also, our predictions follow the Weber law [START_REF] Grossberg | Neural dynamics of adaptive timing temporal discrimination during associative learning[END_REF]: their peak of activity preceding the expected timing of an event is less and less precise has the learn timing gets longer, as observed in animals. This explains why the long timings which are predicted well in advance by the system tend to lead to shorter reproductions.

The timing system learns continuously, not only during the first loops. The experimenter can always modify what the robot has learned. New checkpoints can be added by humans by placing themselves in front of the robot. The timings learned for existing checkpoints can be made longer in the same way. They can also be made shorter by using the leash to restart the movement of the robot before the end of the learned delay. The architecture will then learn to adapt the predicted timings.

IV. CONCLUSIONS AND FUTURE WORKS

We have presented a biologically plausible neural network capable of learning temporal associations between consecutive sensory events. The neural network approach allows the system to learn and adapt those timings based on task contingencies. The associations are linked to motor actions, meaning that the robot learns which actions will allow it to move from one state to another. At any time the system provides information about reachable states and about the expected timing of the arrival in a new state. A complementary network detects the non-occurrence of expected events and can modify the behavior of the robot accordingly. The latter ability forms a kind of novelty detection, allowing the robot to know when conditions in the tasks or environment may have changed. The integration of the temporal aspect of transitions between sensory events, which represents the novelty of this work, has been done with an architecture that has been successfully used in several navigation experiments, using a cognitive map [START_REF] Cuperlier | Neurobiologically inspired mobile robot navigation and planning[END_REF] or Q-learning [START_REF] Hirel | Why and how hippocampal transition cells can be used in reinforcement learning[END_REF]. Several works have used biologically plausible models of place cells in robotic navigation and use reinforcement learning to adapt strategies to changing task contingencies [START_REF] Arleo | Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity[END_REF], [START_REF] Foster | A model of hippocampally dependent navigation, using the temporal difference learning rule[END_REF], [START_REF] Dollé | Path planning versus cue responding: a bio-inspired model of switching between navigation strategies[END_REF]. However these models fail to take into account the temporality of action-consequence pairings. In cases where delayed rewards which are not the immediate consequence of an action of the robot are delivered, a more complex model integrating time is required.

The non-occurrence signal really gives an internal negative feedback about the current actions of the robot. Here it is used to control the behavior of the robot by acting on the drive activity to modulate the tendency of the robot to try to reach particular goals. Yet an alternative could have been to use the signal to inhibit the currently selected transition, telling the cognitive map to look for another path to the goal. It could also be used to suppress the current navigation strategy altogether, telling the system to switch from a planning to a sensori-motor strategy. A modulation of the learning of actions, signaling that the current sensorimotor association should be suppressed because its outcome is not the predicted one, would also be an option. With such a variety of choices, we would need a high-level metacontroller to appropriately use this negative feedback. The bases of this meta-controller have already been implemented using frustration signals which built up over time if the robot stopped making progress [START_REF] Hasson | Frustration as a generical regulatory mechanism for motivated navigation[END_REF]. However the system needed to be temporally calibrated for the task. With the temporal predictions, the need for this calibration would disappear.

Fig. 1 .

 1 Fig. 1. Hippocampal associative learning architecture

Fig. 3 .

 3 Fig.3. Detail of the prediction, non-occurrence and drive systems. Transition activity is associated with satisfaction signals. Transitions predicting satisfying stimuli and neutral stimuli can thus be differentiated. A neural network is used to memorize the maximal activity of the bell-shaped predictions. When the prediction activity falls back under a fraction λ of this maximum, it means that the expected timing of the satisfaction has passed and it will probably not happen. The parameter λ controls the tendency of the system to wait more or less for expected satisfactions after their predicted timing. The closer λ is to 1, the faster the system will signal the non-occurrence of a prediction. As long as the prediction activity is close to the max, the non-occurrence signal is inhibited but when the difference becomes significant non-occurrence neurons are activated. When the prediction stops, the memories of the maximal prediction activity is reset by inhibitory connections. The non-occurrence signal is used to inhibit the drive that was supposed to be satisfied.

Fig. 4 .

 4 Fig. 4. Integrated model combining the timed transition learning architecture with a cognitive map navigation strategy. A detail of the new nonoccurrence system represented by darker boxes is shown in fig 3.

Fig. 5 .

 5 Fig. 5. a) Cognitive map of the environment learned during the exploration phase.The circles represent the center of the place cells and the lines transitions between places. b) Trajectories taken by the robot while planning its path to the goal using the cognitive map. Starting points are indicated by crosses and the goal by a red rectangle. The paths are not optimal (i.e. straight lines) because the algorithm selects transitions based on their distance to the goal in terms of number of nodes in the graph, which can lead to some sub-optimal decisions in the Cartesian space. Since the competition only selects one transition, the robot follows the edges of the transition graph. Smoother trajectories could be obtained by increasing the number of place cells, at the expense of memory, computational power and an increased learning time. Another solution could be to have a soft competition between transitions to select some average direction predicted by several good transitions.

Fig. 7 .

 7 Fig.7. a) Places and transitions learned for the patrol. Place field centers are signaled by black disks. Black squares represent the association between places and proprioceptive events. They point out the 3 checkpoints. b) Trajectories of the robot during the task. The dark trajectory corresponds to the learned path, when guided by the human. The light trajectories are the autonomous reproduction. Due to the size of place fields, the reproduced loop can be shifted compared to the original learned trajectory. This is a consequence of the prediction system that begins to predict the movement to reach the next place once it enters a new place. Since place cells can have broad place fields, this prediction can occur before the robot reaches the center of the field, which is where the robot was originally guided. The architecture offers a trade-off between the computational power required and the precision of the reproduction in the number of place cells learned on the trajectory. Indeed only movements between place cells are learned, so too few place cells will lead to a very coarse reproduction of the learned trajectory, whereas a fine reproduction can be achieve with many place cells but will lead to a longer learning phase and a memory and computationnally expensive system.

TABLE I DURATION

 I OF IMMOBILITY AT EACH OF THE CHECKPOINTS.

	Checkpoint	1	2	3
	Learned (s)	24.1	19.1	11.6
	Reproduced (mean±sd)

22.3 ± 0.7 19.1 ± 0.3 15.3 ± 0.8

Neural network simulations are run on an on-board computer with an Intel Core2Duo T7500 processor running at

2.2Ghz and 2GB RAM communicating via wifi with a workstation powered by an Intel Core2Quad Q6600 running at 2.4Ghz and 4GB RAM.

This work was supported by the CRNS, DGA and ANR.