Simon L Gay
email: simon.gay@liris.cnrs.fr

Alain Mille
email: alain.mille@liris.cnrs.fr

Amlie Cordier
email: amelie.cordier@univ-lyon1.fr

Autonomous object modeling and exploiting: a new approach based on affordances from continual interaction with environment

We present an architecture for self-motivated agents to generate behaviors in an environment that is continuous, both in space and time, through a continual interaction process. The long-term goal is to design agents that construct their own knowledge of objects and space through experience of the environment, rather than exploiting pre-coded knowledge. The agent exploits this constructed knowledge to exhibit behaviors satisfying its self-motivated principles, based on valences attributed to interactions, that specify inborn behavioral preferences. Over time, the agent learns the relation between its perception of objects and the interactions that they afford, in the form of data structures, called signatures of interaction. The agent keeps track of enacted interactions in a spatial memory in which it can use signatures to recognize and localize distant possibilities of interactions, and exhibits behaviors that satisfy its motivation principles, accordingly to this approach. In this paper, we propose a continual decision cycle between an agent and its environment to cope with the constraints that an artificial agent would meet in a physical environment.

I. INTRODUCTION

In this paper, we address the problem of the construction, interpretation and exploitation of a short-term memory representation of an environment offering a continual interaction process between an artificial agent and this environment. Our work is based on a model described in Georgeon and Aha [START_REF] Georgeon | The radical interactionism conceptual commitment[END_REF], called Radical Interactionism (RI), in which action-perception couples are considered as atomic and kept embedded within data structures called interactions, modeling Piaget's notion of sensorimotor scheme [START_REF] Piaget | The construction of reality in the child[END_REF]. This approach assumes that new agents come up with a predefined set of uninterpreted interactions associated with predefined valences, and seek to enact interactions with positive valence and to avoid interactions with negative valences. This motivation principle is called interaction motivation [START_REF] Georgeon | The radical interactionism conceptual commitment[END_REF], and is related to the problem of intrinsic motivation [START_REF] Oudeyer | Intrinsic motivation systems for autonomous mental development[END_REF]. The agent perceives and interprets its environment by identifying affordances rather than recognizing objects on the basis of predefined features. This approach addresses the knowledge grounding problem [START_REF] Harnad | The symbol grounding problem[END_REF] by letting knowledge of objects arise from experience.

Our long term goal is to implement artificial agents in robots to make them able to interact with initially unknown envi-ronments. Such an agent must overcome severe constraints. Let's take an example of a planetary exploration rover that must autonomously face environmental situations that can be partially or completely unknown, implying several constraints:

1) The agent cannot use a predefined set of object models. Instead, the agent must learn to recognize interactional opportunities (or affordances) offered by the environment.

2) The agent cannot rely on a predefined internal model, as the rover may move on unknown surfaces. Thus, the rover must construct an internal representation of space based on its possibilities of interactions.

Instead of trying to solve a specific class of problems, we try to make an agent able to learn to integrate its environment in order to generate behaviors satisfying its motivational principles. Point 1) is well studied in literature. For example, Pfeifer and Scheier [START_REF] Pfeifer | Sensory -motor coordination: The metaphor and beyond[END_REF], Ugur et al. [START_REF] Ugur | The learning and use of traversability affordance using range images on a mobile robot[END_REF] and Baleia [START_REF] Baleia | On exploiting haptic cues for selfsupervised learning of depth-based robot navigation affordances[END_REF] proposed approaches where an agent learns to define and classify objects of its environment through possibilities of interaction (such as "walkthroughable" or not). However, these approaches do not let the possibility to generate spatial behaviors. Pisokas and Nehmzow [START_REF] Pisokas | Experiments in Subsymbolic Action Planning with Mobile Robots[END_REF] and Ugur et al. [START_REF] Ugur | Oztop Unsupervised Learning of Object Affordances for Planning in a Mobile Manipulation Platform[END_REF] proposed models that learn relations between actions and affordances to generate plans to reach affordances, but rely on predefined goals and cannot be used to generate exploitable models of surrounding space. Point 2) implies that the agent can construct a peripersonal [START_REF] Hoffmann | Body schema in robotics: A review[END_REF] and extrapersonal [START_REF] Previc | The neuropsychology of 3-D space[END_REF] representations of its environment.

II. DEVELOPMENTAL LEARNING AND RADICAL

INTERACTIONISM

Developmental learning [START_REF] Vernon | A survey of artificial cognitive systems: Implications for the autonomous development of mental capabilities in computational agents[END_REF] makes the assumption that the agent is agnostic of its environment [START_REF] Georgeon | Designing environment-agnostic agents[END_REF]. This assumption prohibits considering that the agent can identify states, both for its environment and for itself. The agent can only know its environment through its own interactional experience and uses its experience to construct cognitive structures that it can use to satisfy intrinsic motivational principles. This work proposes to consider interactions as atomic [START_REF] Georgeon | The radical interactionism conceptual commitment[END_REF], rather than separated actions and perceptions. The interactional cycle thus reviews the traditional cycle as considered by literature to model agentenvironment interactions. Experience is constructed from the emergence of regularities in interactional sequences. This framework is now questioned for studying how objects can emerge from interactions of an agnostic agent with its environment. The theoretical complexity of this question is potentially huge and classical approaches come up against this problem because of the combinational complexity of the environment.

The RI approach only considers the point of view of the agent, which theoretically simplifies this question as the limited ability to experience the environment reduces intrinsically the computational complexity. The issue of objects emergence within the agent's interactional space is critical for proposing a new kind of agent's adaptation process when its environment is largely unknown. This research work addresses the design of an agent architecture satisfying the RI principles, able to build spatial structures and dynamic structures taking into account emerging objects. Several RI models were developed with the long-term aim of implementing them in robots moving in a real environment [3][2]. However, these models used a discrete interaction cycle, which cannot be applied on a physical robot. In this paper, we present a main milestone of the RI model allowing a continual interaction cycle between an agent and its environment.

A. Formalization of Continual RI Model

An agent has a set of actions (A) and perceptions (O) characterized by its actuators and sensors. Theories of cognition suggest that perception cannot be separated from the action that generates it. Indeed, the same perception can have different meanings, depending on the action that produced it. Previous models of RI used interactions under the form of couple action-perceptions. Each interaction i is attributed an inborn valence ν i characterizing the intrinsic satisfaction of experiencing i. The decision cycle starts with a decision of the agent, that intends to enact an intended interaction i t and experiences, at the end of the decision cycle, the enacted interaction e t that was effectively experienced. An RI agent tries to capture regularities offered by its environment to generate behaviors that allow to enact interactions with positive valences and avoid interactions with negative valences.

In order to make a continual interaction cycle, we propose to define an interaction i as a set i={( k ,t k )} k∈ 0;n , where k ∈A∪O and t k ∈R -. The agent is equipped with a register Φ of length ∆t Φ consisting in a timeline that is continually updated through time and contains actions and perceptions experienced between t -∆t Φ and t. The length of this register must be defined according to interactional possibilities and must be long enough to help the agent characterize its movement. As an example, an agent with motion sensors (such as a vestibular system) can characterize its movement with a few elements, while an agent with no vestibular system would need a longer record of past actions and perceptions. This doesn't affect functioning of the model, but only affects efficiency and relevance of the generated internal model. At each t, it is thus possible to define e t , an enacted interaction as the set of couples ( k ,t k ) in the register (e t ≡ Φ t ). We note I the set of possible interactions.

We adapted the RI model for a continual decision cycle, using these continual interactions. We define, between two interactions i t1 and i t2 , with t2-t1<∆tΦ , a variation ∆i containing the set of couple ( k , t k ) that must be added to i t1 to obtain i t2 . We only consider elements that must be added, as couples that move out of Φ will have disappeared at t 2 . When the agent experiences consecutively i t1 and i t2 , we consider that i t1 is a predecessor of i t2 and note it 1 +∆i1,2→it 2 , where ∆i 1,2 is the variation between i t1 and i t2 . We can then consider that the agent is continually intending a variation of interaction di t and continually experiences enacted interaction e t through enacted variations de t .

The Continual RI (CRI) is formalized as follows: the agent begins with a set It 0 ⊆I of interactions and a set ∆I of possible variations di. At each interaction i of I is attributed an inborn valence ν i characterizing the satisfaction of experiencing i. The agent continually tries to intend a variation di t and experiences the enacted variation de t . The register Φ is updated to define the enacted interaction e t that the agent experienced at t. The intention of the agent is a success if di t = de t and a failure otherwise. If di t = de t then de t is considered as an alternative of di t . Figure 1 illustrates this formalism.

B. A Parallel CRI Model

Discovering spatial properties of the environment with a unique interaction at a time is unrealistic. The Parallel Radical Interactionism (PRI) model [START_REF] Gay | Autonomous construction and exploitation of a spatial memory by a self-motivated agent[END_REF] was proposed to overcome this limitation. In PRI model, the agent experiences additional stimuli, in addition to the enacted interaction. However, these stimuli have to be considered with the movement produced by the enacted interaction. As an example, the optic flow on a retina can only carry a spatial information if it is considered with the movement that generates it. The PRI model defines secondary interactions as an association between an interaction and an additional stimulus. We can apply this principle to CRI model to define the Continual PRI (CPRI), because we make the assumption that register Φ, and thus e t , can characterize the movement of the agent at time t. We call a primary interaction a set ip={( k ,t k )} k and a secondary interaction an association between an interaction and an additional stimulus is=(i {p,s} ,( s,0)), with i {p,s} the associated interaction of i s .

Environment Agent di ={(ε , 0)} k k∈N t t time e t i s t t-Δt Φ e = {(ε ,t )} m m m/ t ∈ [-Δt , 0] m t de ={(ε , 0)} k' k'∈N t i = (e , ε ) t s,l t s,l Φ Fig. 1.
Model of the Continuous Radical Interactionism. The agent continually tries to enact a variation of interaction di ∈ ∆I ⊆ P((A ∪ O) × R -), and experiences the variation de ∈ ∆I that is actually enacted. Experienced variations are stored on a timeline of length ∆t φ that is continually updated, defining the enacted interaction et. In the parallel CRI model, the agent also experiences several additional stimuli, that are associated with et to form secondary interactions.

Formally, the CPRI model differs from CRI in that, at time t, the agent experiences a set of enacted interactions {e k } t , containing a unique primary interaction and a set of enacted secondary interactions associated with this primary interaction.

A secondary interaction i s , like a primary interaction, succeeds when i s ∈ {e k } t , and fails when its associated interaction i s is in {e k } t ), but i s / ∈ {e k } t . When the associated interaction is not enacted, the result of i s is ignored.

III. A CONTINUAL SPACE MEMORY

Space memory is a set of mechanisms dedicated to the integration and exploitation of environmental properties observed through interaction with the environment. This section presents concepts and principles of the space memory in a continual interaction process.

A. Signatures of interactions

This mechanism is based on the assumption that the result of enacting an interaction depends on a limited context of elements in the environment. We expect such contexts to define objects with which the agent can interact. This definition of objects relates to the concept of affordances proposed by Gibson [START_REF] Gibson | The theory of affordances[END_REF]. A RI agent can only perceive its environment through enacted interactions. We define the interactional context E t , characterizing the environment experienced by the agent, as the continuous sets of enacted interactions that were enacted between [t-∆t E ,t], where ∆t E is the length of the interactional context. ∆t E is independent from ∆t Φ , and must be long enough to let the agent detect the causes of its interactions. CRI model allows an interesting simplification: two primary interactions i 1 and i 2 such that i1+∆i1,3→i3 and i2+∆i2,3→i3 only differ by their oldest elements, as ∆i 1,3 = ∆i 2,3 . If we assume that oldest elements have a lower influence on the movement, it is possible to consider that i 1 and i 2 characterize similar movements. Then, it is possible to consider secondary interactions i s,1,k = (i 1 , ( k , 0)) and i s,2,k = (i 2 , ( k , 0)) as the same input in context E.

We define a signature of an interaction i as a structure learned through experience that can characterize the presence or the absence of the object affording i in a context E, and thus, the enactability of i. We formalize a signature S i of an interaction i as a function S i : P(E) → [-1; 1], where P(E) denotes the set of possible interactional contexts, that gives a numerical value in [-1, 1] that reflects the possibility of successfully enacting i in an interactional context E. S i (E) = 1 means an absolute certainty of success and S i (E) = -1 an absolute certainty of failure. S i is learned and reinforced when i succeeds or fails to generate accurate predictions. A signature must be reversible: it must be possible to estimate the set of minimal interactional contexts (noted Ŝi) that should be present when an interaction is considered as enactable. We use S i to predict the enaction result of i and Ŝi to extract and exploit information about the object affording i.

Although defining objects by learning to recognize affordances they provide is abundant in literature (e.g. [START_REF] Ugur | The learning and use of traversability affordance using range images on a mobile robot[END_REF], [START_REF] Min | Affordance research in developmental robotics: A survey[END_REF]), the signatures approach differs by the use of interactions, which allows implicit relations between interactions, and thus, spatial properties of the environment, to be discovered.

We use the implementation of signatures proposed by Gay et al. [START_REF] Gay | Autonomous construction and exploitation of a spatial memory by a self-motivated agent[END_REF], based on formal neurons. This implementation characterizes a signature S i with a vector of weights Wi=[wi,1,...,wi,n] that is reinforced each time i is enacted as a success or a failure through delta rule. This implementation is interesting as the weight vector of a signature allows observing the average context affording an interaction, and thus, the objects as defined by the agent.

B. Object instances

A signature S i of an interaction i characterizes a context, at a certain position relative to the agent, in the form of sets of couples (j k ,t k )∈ Ŝi\tk∈[-∆tE ,0]. To detect such affordances in space, we exploit a property of signatures: each interaction j k may have its own signature S j k . It is thus possible to replace elements j k of S i that are related to the same primary interaction j with their own signatures S j k , resulting contexts Ŝj,∆t j,i i that, after enacting j then ∆j,i / j+∆j,i→i (with ∆t j,i duration of ∆j, i), are expected to afford i. The main difference with implementations on RI/PRI agents is that we need to consider delay between interactions. These contexts characterize objects affording i that the agent can reach through enaction of j and ∆j, i. When such a context is present in the interactional context E t (i.e. S j,∆t j,i i >0), we consider that an instance of the object affording i is present at position (j, ∆t j,i ), with a certainty of S j,∆t j,i i (Et). Note that these contexts do not characterize reachability of the object instance, but only characterize the presence of the instance and the position relative to the agent.

This process can be recursively applied, allowing to backmove an interaction through an increasingly long sequence σ = [(j l , ∆t l )] l∈N . An instance of the object affording i is then considered as present at position σ, at a distance d≡∆t1+...+∆tn ,(n=Card(σ)), with a certainty of S σ i (E t ). In an implementation based on formal neurons, predecessors are computed as follows: we note W i,(j,∆t j ) the subset of weights of signature S i limited to weights related to interaction j and its associated secondary interactions {j k } at time ∆t j . The set of weights W (j,∆t j ) i of S (j,∆t j ) i is computed using sets W j k of j and {j k }:

W (j,∆t j ) i = w k ∈W i,(j,∆t j ) w k max w k ∈W i,(j,∆t j ) (w k ) × Wj k (1)

C. Spatial structure

This structure stores and tracks detected object instances, even when they escape from the agent's sensory system. In a previous work [START_REF] Gay | Autonomous construction and exploitation of a spatial memory by a self-motivated agent[END_REF], we proposed a structure that can characterize the surrounding context of the agent without preconception and without using the notion of space. This work showed that only two types of information are required to localize an object instance affording an interaction i a : the interaction i m that allows to move closest to this instance, and an estimation of its distance as the number of interactions needed to reach this instance. The spatial structure stores and tracks object instances under the form of triplets (ia,im,d)∈I×I×N + . In CRI model, object instance positions are characterized with triplets (ia,(im,∆t),d)∈I×(I×R + )×R + , with (i m , ∆t) the first element of a sequence σ. However, these changes do not require modifications in the mechanisms of the spatial structure.

D. Decisional systems

We propose three decisional mechanisms to define the next variations to enact. The first mechanism is dedicated to sensorimotor learning: at time t, if there are variations di that were not tested before in the context of e t , the mechanism selects one of them, allowing to construct the set of possible interactions.

The exploration mechanism tests and improves signatures of interactions. This mechanism considers variations ∆i et,it+∆t considered as enactable, i.e. ∀∆t i ∈ [0, ∆t[, S it+∆t i (E t+∆ti ) > 0 where Et+∆t i ={(i l ,t l )}/(i l ,t l + ∆ti)∈Et. The maximum length of variations is thus limited by ∆t E . Using possible variations, the mechanism defines a learning utility value lu di to each known variations di t :

lu dit = max σ/di1,σ=di 1 |S it+∆t (E t+∆t )| × (nb et,di + 1) (2)
where nb et,di counts the number of enactions of variation di from e t ensuring that learning utilities values will decrease through time. Then, if max di (lu di ) > τ learn , the agent selects and tries to enact the variation di that has the maximum lu di . We note τ learn ∈]0, 1[ the learning threshold that defines the minimum reliability value to consider an interaction as reliable. Otherwise, the agent uses the exploitation mechanism. Thus, the more accurate interactions signatures become, the less used the exploration mechanism is. The exploitation mechanism helps generating behaviors that satisfy the agents motivational principles at short and medium terms. The principle is that this mechanism adds a positive utility value to interactions that enable moving closer to object instances affording interactions with high valence ν, and a negative value when the object instances afford interactions with negative valences. The utility value is weighted by the distance of object instances so that far object instances have a lower influence. The exploitation mechanism considers candidate variations. A variation di t is candidate for enaction if i/e t + di t → i is predicted as a success. A variation di can bring an object instance characterized by triplet (ia,(im,∆t) k ,d k ), stored in spatial structure M , closer by dt if there exists a variation ∆i et,im considered as enactable and that begins with di. The mechanism defines an exploitation utility value eu di to each candidate variations (3) and proposes a candidate with the greatest exploitation utility.

eu dit = ν i + (ia ,(im,∆t) k ,d k )∈M t / ∃∆i e t ,im enactable ∧ di t ∈∆i e t ,im ν ia × f (d k ) (3)
where f : R + →]0; 1] is a function that characterizes object influence according to their relative distance. After tests [START_REF] Gay | Autonomous construction and exploitation of a spatial memory by a self-motivated agent[END_REF], we chose the function f : x → β.e -γ.x (where β = 0.01 and γ = 0.05 are influence coefficients).

IV. IMPLEMENTATION

We tested our mechanisms on an artificial agent moving in a 2-dimensional continuous and static environment. The test environment is intentionally simple to make it easier to analyze. Indeed, with a 3D environment, or with more than three colors, it would not have been possible to display and analyze signatures as RGB images. However, the complexity of the environment does not affect spatial memory principles: an agent with 3D interactional possibilities would construct a structure reflecting a 3D environment. The sensorimotor possibilities of the agent define a list of five actions and perceptions, listed below:

forward impulsion, -bump in a solid object, -left rotation impulsion, -eat something edible. -right rotation impulsion, Bump and eat perceptions can be observed while the agent gives an impulsion. The agent can intend 12 different di: wait (di = ∅), forward impulsion, left impulsion, right impulsion, bump, eat, forward and bump, forward and eat, left and bump, left and eat, right and bump, right and eat. The simulation step of the environment defines the shortest period dt that the agent can experience. A forward impulsion sets the linear speed of the agent to 0.2 grid cell per step. Speed decreases at each simulation step to 0, simulating fluid friction. After a forward impulsion, the agent moves for nearly its length (1 grid cell) in 10 simulation steps. A rotation impulsion sets the angular speed of the agent to 18 • per step, then decreases at each step to 0. After giving an impulsion, the agent rotates nearly 90 • (left or right) in 10 simulation steps. Bump is enacted when the agent bumps in a solid element. Eat is enacted when the agent moves over an edible element.

The timeline Φ and the length of interactional context E t are limited to 10 simulation steps. To make easier the results analysis, we only defined interactions composed of at most two non-empty variations, and limited the number of non-empty variations on the timeline to three. When three non-empty variations are already present on the timeline, the agent can only wait. An interaction is thus under the form i=({ k ,t0} k ) or i=({ k ,t0} k ,{ k ,t1} k ),t0,t1∈[-10,0]. Experiments use valences defined as follows: if t 0 = 0, then valence of i is 50 when { k } contains eat, -10 when it contains bump, 2 when it contains forward and -2 when it contains turn. Otherwise, valence is 0. We thus arbitrarily decided that the newborn agent strongly likes eating, dislike bumping and slightly likes moving forward. We added a set of secondary stimuli provided by the agent visual system, that can detect colors among red, green, blue, and measures distances. Visual interactions consist in seeing a colored element while enacting a primary interaction, at a certain, but unknown, position of space. We discretize the visual field as a regular grid of 30×15 positions. These interactions have a predefined valence of 0. We thus define 30×15×3 = 1350 possible secondary interactions per primary interaction.

We also simplified the interactional context, using the simplification proposed in Section III-A to gather secondary interactions as the same inputs of the interactional context. Position of the agent, relative to the object Fig. 2. Sample of signatures obtained after 400,000 simulation steps. Weights of signatures are normalized according to the greatest weight, and organized topographically to match position of the corresponding secondary interaction in space. For each position, the three weights related to seeing one of the three colors are displayed using the three color channels to define a RGB image, allowing to observe the color of objects considered by the agent. The little disc indicates the neuron bias, green means a normalized value of 1, red a normalized value of -1. Interactions associated with signatures are displayed below signatures. This representation allows an external observer to observe an average context required to successfully enact the displayed interaction. We observe that interactions containing move forward are afforded by the absence of green and blue element in front of the agent (mid-red blobs), those containing bump (green disc at t = 0) are related to a green element, and those containing eat event (blue disc) are related to a blue element. While the size of the object nearly correspond to the size of the agent, the exact position of the object depends on elements that compose interactions. The third line shows signatures of some secondary interactions consisting of seeing green, blue or red element at position designated by as a red square. We observe that these interactions are related to the presence of an element of the same color at a position that characterizes the movement produced by primary interaction since the last event (orange arrow): some properties of the environment, such as inertia, are thus encoded in these signatures.

This simplified interactional context can contain at most 30×15+1 = 451 different elements. We implemented a hardcoded spatial structure similar to the structure proposed by Gay and Hassas [START_REF] Gay | Autonomous object modeling based on affordances in a dynamic environment[END_REF], that implements properties observed in agnostic spatial structure [START_REF] Gay | Autonomous construction and exploitation of a spatial memory by a self-motivated agent[END_REF].

The environment contains three types of elements characterized by a color that makes them recognizable with visual interactions: walls (green) affording bump, preys (blue) affording eat, and algae (red) that are walkthroughable. These three elements are opaque: the agent cannot detect an element that is behind another one.

V. EXPERIMENTS AND RESULTS

We report on two experiments. The first focuses on the sensorimotor learning and exploration decisional systems and the analysis of properties emerging from signatures of interactions. The second tests the exploitation decisional mechanism and analyses emergent behaviors of the agent.

A. Emergence of signatures

We first let the agent move in its environment, driven by the sensorimotor learning and exploration mechanisms. In the first 10,000 simulation steps, the agent discovers most possible interactions, although new interactions are regularly discovered. Then, the agent's behavior becomes more and more driven by the exploration mechanism. Figure 2 shows examples of signatures obtained after 400,000 simulation steps, displayed in an observer-friendly way. We observe that interactions containing move forward, bump and eat elements are related to elements that are in front of the agent, respectively the absence of wall and prey (dark red blobs), the presence of a wall (green blobs) and the presence of a prey (blue blobs). However, the position of the blobs depends on elements that compose the interaction: past elements change the movement of the agent when the last event is enacted, which changes the relative position of the object that affords the interaction. These changes in position indicate that properties of the environment, such as inertia of the agent, are encoded in signatures, and can help defining when to enact a variation.

We observe, on signatures of secondary interactions, that the result of seeing an element of color c at a position p while enacting a primary interaction i is related to seeing an element of the same color c at a position p that characterizes the movement produced by enaction of i. Thus, movement produced by a primary interaction, and thus spatial properties of space, can be characterized by signatures of secondary interactions. We used these signatures to detect object instances around the agent. Figure 3 shows an example of environmental configuration. We can observe positions relative to the agent where objects are detected.

B. Navigation behavior emergence

This experiment tests the exploitation mechanism, using the hard-coded memory described in Section IV. The agent moves in its environment, driven by the three decisional mechanisms. Once most usual interactions are discovered, the exploration mechanism begins to influence the behavior of the agent. Then, through time, signatures become accurate, and the exploitation mechanism is more often used: at simulation step 100,000, the exploitation mechanism provides nearly 30% decisions, and 56% at step 300,000. Once signatures of interactions containing bump and eat becomes accurate enough, the agent begins to move toward preys and to avoid walls.

We then deactivate sensorimotor learning and exploration mechanisms to study exploitation mechanism and how the agent interprets elements of its environment. We observe that the agent is strongly attracted by preys, as they afford interactions with a high valences: the agent moves from a prey to another one. We use this property to observe how the agent interprets elements of the environment, by hiding a prey with an other element and observing how the agent behaves in front of the new element. Figure 4.b) shows the path used to catch a prey. We first add algae, then walls. We observe that with algae, the agent behaves like if they were not present: indeed, algae have the same interactional properties as empty space, and became interactionaly invisible. With walls, the agent uses another path avoiding the wall. As walls afford interactions with negative valence, they are considered as repulsive. This new path illustrates how the space memory works: as the agent comes closer to the wall, any interaction containing move forward gets a strong negative utility value. When the utility value becomes strong enough to overcome turn interactions, the agent rotates. After bypassing the wall, the agent moves again towards the prey.

VI. CONCLUSION

We proposed new RI models and mechanisms to enable an artificial agent to continually interact with its environment, ) the agent is strongly attracted by the prey, as it affords interactions with high valences. c) we hide the prey with algae. The agent uses a similar path than without algae: the agent learned that algae have the same interactional properties as empty space. The path is not exactly the same because the agent uses the noisy position given by the spatial memory. d) we add a wall between the agent and the prey. The agent bypasses the wall while keeping its distance with the wall. Walls thus becomes repulsive as they afford interactions with negative valences.

integrating properties of its environment and its body such as inertia. We presented a formalization and an implementation of a continual RI model that considers continual agentenvironment interaction process rather than a discrete cycle action-result, enabling implementation of RI agents in robots. It appeared that these changes do not impact importantly the spatial structure as it was proposed in Gay et al. [START_REF] Gay | Autonomous construction and exploitation of a spatial memory by a self-motivated agent[END_REF], indicating that this structure can be directly used in a CRI implementation. This work sheds some lights on how a physical agent can interact with its environment, learn properties of its environment and behave according to intrinsic motivational principles, without any preconceptions about its environment or its sensorimotor possibilities. In future works, we intend to implement our mechanisms in more complex systems, and in particular agents using a continuous set I of interactions.

  neuron bias low weight (in absolute value) weights with high value (in absolute value) characterizing the object affording the interaction interaction associated to the signature. Here: ({forward;-6},{bump;0}) Movement encoded by signatures of secondary interactions

Fig. 3 .

 3 Fig. 3. Detection of object instances. Top left: the agent (gray shark) in its environment. Walls are represented as green blocks, preys as blue fishes and algae as red flowers. A black line shows the agent's path over the last 200 simulation steps. Here, the agent is turning left (left impulsion three simulation steps ago). The inset picture shows enacted visual interactions on a topographic and colored representation that makes them recognizable by an external observer. Top right: Detected instances of objects affording bump interactions and of objects affording eat. As external observers, we represent object instances with circles and lines (green for bump and blue for eat) indicating positions and orientations characterized by sequences σ of instances. Bottom: examples of sequences characterizing object instances (afforded interactions are indicated in red). Sequences of instances affording bump begin with move forward and turn right, while sequences related to eat begin with move forward and turn left, characterizing that there is a solid object in front-right side of the agent and an edible object in front-left side.

Fig. 4 .

 4 Fig.4. a) initial configuration for each run. b) the agent is strongly attracted by the prey, as it affords interactions with high valences. c) we hide the prey with algae. The agent uses a similar path than without algae: the agent learned that algae have the same interactional properties as empty space. The path is not exactly the same because the agent uses the noisy position given by the spatial memory. d) we add a wall between the agent and the prey. The agent bypasses the wall while keeping its distance with the wall. Walls thus becomes repulsive as they afford interactions with negative valences.