Réaliser une thèse ma toujours semblé l'objectif qui avait le plus de sens à réaliser lorsque je faisais mes études scientifiques. Je voyais l'occasion d'écouler un temps légitime à défricher, décortiquer, bricoler les composantes d'un problème passionnant. Lorsque je suis arrivé à l'ISIR, je portais cette idée de produire un travail abouti et concret; état de l'art, méthode, résultats. Au final, l'idée de la thèse n'est pas l'obtention d'un diplome, mais plutôt de toujours savoir proposer des réponses à des questions qui, finalement, ne se terminerons probablement jamais. Ainsi, à travailler indéfiniment sur un sujet original, on commence à l'incarner et le projet devient intime et personnel. Et, au fur et à mesure que le temps passe, à la thèse scientifique se mèle une thèse des sentiments. Ces années de doctorat n'ont pas été évidentes, mais au delà de ce pléonasme, elles seront sûrement parmi les plus riches de ma vie. Et cette richesse provient des leçons apprises par, et pour, les personnes qui ont croisé mon chemin. Pour commencer je voudrais remercier ma famille, qui malgré l'éloignement Parisien, ont toujours été un socle solide indispensable pour ces aventures. A mes amis, il n'y a de mots assez fort pour décrire l'importance que vous avez eu ces dernières années. Je pense que vous vous en rendez compte et je compte bien vous rendre au centuple tout ce que vous m'avez donné. Je voudrais remercier mes deux encadrants, pour leur bienveillance, leur confiance mais aussi leur courage de s'attaquer à ce sujet multiforme: philosophique et singulier, biologique et artificiel, topologique et discret, isolé mais à l'origine de tout. A Bruno, pour avoir trouvé les idées et les mots qui m'ont convaincu de m'engager à ses côtés. Aux heures incensées que Sylvain a passées à m'accompagner pendant la rédaction de toutes (!) les publications et la répétitions des conférences. Pour ses encouragements et sa bienveillance qui ont été un précieux balisage pour me remettre dans le droit chemin lorsque les choses ne se passaient pas si bien. Enfin je voudrais remercier mon jury de thèse. Ce n'est pas évident de juger un travail plutôt formel qui sort des sentiers classiques. Merci de votre curiosité et de votre ouverture d'esprit. ix Chapter 1. Introduction Espace géométrique extérieur 1.3. The framework: a formal approach to the emergence of spatial representations 13 2.2. Types of sensory invariance

v

English Abstract

In this thesis, we shall propose a formalism to develop the notion of sensorimotor spatial perception in a robotic context. Usually in classical approaches to robotics, the perception of space is given to the agent through predefined models of the world such as the agent's forward kinematics and the spatial positions of effectors and sensors. However, the awareness of space does not necessarly require to be a priori provided. As an example, in the sensorimotor contingency theory, developped by J. Kevin O'Regan (2001), it is supposed that knowledge of space can be obtained from the dependencies between sensory inputs and generated actions. In this work, we shall study how an embodied agent, situated in an unknown environment with very little a priori knowledge about its body or its sensors, can build a representation of its interaction with the physical space.

First, we shall provide the agent with the minimum a priori knowledge required for interpretation of its sensorimotor flow, such that the approach is general enough and is valid for the majority of robotic agents. Then, it shall be demonstrated that, by following a "refinement process", the agent can exploit basic sensory invariants during successive environments to obtain a representation of its sensors distinguishable spatial configurations in space. However, the state of the environment being unknown to the agent, the sensory invariants can be seen as random variables, so that the formalism shall be extended to stochastic processes. Furthermore, in the probability theory context of the refinement process, the agent obtains an internal representation with a metrical structure based on the sensory invariance probabilities. Hereafter, it shall be demonstrated that under some topological assumptions on the motor space, this metrical internal representation allows planning and representation of sensors' continuous trajectories in space. Finally, by computing similarities between the internal representations obtained from the agent's different sensory streams, it shall be shown that the agent is able to build a representation of its sensors topographical structure, e.g. arrangement of the camera pixels, as well as to know when it interacts with its own body which should lead to the discovery of the self. vi

French Abstract

Dans cette thèse, nous proposons un formalisme afin de développer la notion de perception sensorimotrice spatiale dans le contexte robotique. Généralement en robotique classique, la perception de l'espace est innée à l'agent grâce à la modélisation en amont d'un modèle cinématique du robot et de la configuration spatiale de ses capteurs. Cependant, la connaissance de l'espace ne doit pas nécessairement être une donnée a priori. Par example, l'approche des contingences sensorimotrices, développée par J. Kevin O'Regan (2001), suppose que cette connaissance peut être obtenue à partir des dépendances entre les entrées sensorielles et les commandes motrices. Dans ce travail, nous étudions comment un agent incarné et situé dans un environment inconnu avec très peu d'information a priori sur son corps ou ses capteurs, peut construire une representation de son interaction avec l'espace physique.

Pour commencer, nous devons donner à l'agent la quantité minimale de connaissances nécessaires pour l'interprétation des données du flux sensorimoteur, ainsi l'approche est suffisamment générale pour être valide pour une majorité d'agents robotiques. Puis, nous démontrons qu'en suivant un "processus de raffinement", l'agent peut exploiter ses invariants sensoriels basiques pour construire une représentation de l'espace des configurations spatiales distinguables de ses capteurs. Cependant, l'état de l'environment étant inconnu pour l'agent, ces invariants sensoriels peuvent être modélisés comme des variables aléatoires et le formalisme peut être étendu aux processus stochastiques. Ainsi, dans le contexte probabiliste, l'agent peut construire une représentation interne avec une structure métrique basée sur la probabilité d'obtenir des invariants sensoriels. Une fois obtenue, la structure métrique permet de définir des hypothèses topologiques de l'espace moteur afin d'obtenir une représentation interne qui permet la planification ainsi que la représentation de trajectoires continues des capteurs dans l'espace. Pour finir, en comparant les représentations obtenues pour les differents flux de données sensorielles, il est possible de montrer que l'agent obtient aussi une representation de la structure topographique de ses capteurs, par exemple l'arrangement des pixels d'une caméra, mais aussi de savoir quand l'agent interagit avec son propre corps ce qui lui permettrait de découvrir le soi.

Perception in robotics: towards autonomous agents

For years, robots were mainly used in industry, most of their use were to replace human operators in difficult, tedious or dangerous tasks. We have applied their physical abilities to improve manufacturing times while reducing costs and increasing precision. Because they worked in closed loop their domain of operation required a very simple environment which can be quasi perfectly modeled by engineers such as a fully automatic manufacturing chains. Furthermore, the last decade has given rise to a generation of robots that are able to adapt to their environments, recognize objects and even moving humans while acting accordingly. The major reason for these innovations is due to the improvement in sensors technologies, computing power and the rise of Artificial Intelligence. However, even if they are capable of complex behaviors, can we say that these robotic agents have the ability of perception ?

Classical approach to perception in robotics

While they are far from being characterized as "intelligent", these agents do interpret their sensory inputs and have representations of the world which are given as ad hoc model of the world provided by engineers. Classically in robotics, perception can be defined as the interpretation of sensory inputs using predefined models of the world and the sensor's structure, in order to perform specific tasks. This classical model of robots can be described by the sense-plan-act paradigm as described by S. Russel and P. Norvig (Russell and Norvig, 2016) and is shown in Figure 1.1. The agent perceptive abilities are therefore developed a priori and adapted to the agent's tasks and the steps of perception, planning and action are mostly decoupled. However, while very efficient in simple environments with specific tasks, classical approaches of robotic perception generally fail when dealing with the unstructured, unpredictable real world. An example can be found in rescue robotics where the environment is so complex and hazardous that, in the current state of the art, the best performances are generally obtained when the robots are controlled by a human operator [START_REF] Delmerico | The current state and future outlook of rescue robotics[END_REF]. The recent DARPA challenges are a great example of how the robotic community attempts to tackle these problems [START_REF] Atkeson | What happened at the darpa robotics challenge, and why[END_REF].

Learning to adapt

In order to improve the adaptability of robotic agents, the robotic community have considered approaches where the agent is able to "learn" from its interactions with the world. A learning agent has the ability to adapt the design of its perceptive structure according to its own experiences.

These learning methodologies can be classified by the agent's context, be it virtual or real, as well as the amount and type of prior information that is given as guidance. In this robotic context, learning is generally driven by goal directed actions. In the Reinforcement learning paradigm, the agent is given a feedback that characterizes the success or failures of its actions, it then tries to optimize its internal models to maximize rewards. In this approach, the nature of the reward shapes the perceptive design. As an example, if the agent receives a reward when reaching a particular area in space, then the 1.1. Perception in robotics: towards autonomous agents 5 models obtained for movement planning will be intrinsically spatial [START_REF] Jonschkowski | Learning state representations with robotic priors[END_REF]. Then, the question of which goals and which rewards should guide the learning in order to obtain truly autonomous agents, becomes critical.

Cognitive development

One paradigm that would allow the agent to naturally define its sets of goals is directly inspired by the observation of the cognitive development in infants such as described in the work of J.Piaget (Piaget, 1937 andPiaget, 1977). The robotic approaches that study the developmental mechanics and architectures of cognition can be regrouped under the term of Developmental robotics. These approaches have the double interest of being used to evaluate the validity of theories in cognitive development of humans or animals. In developmental robotics, the agent attempts to learn hierarchical sets of skills and knowledge, in progressing complexity, from its direct interaction with the environment. The contributions in this field of research come with a wide variety of prior knowledge that depends on the development phase in which the cognitive agent is placed. The priors ranges from uninterpreted sensorimotor inputs and intrinsically motivated goals [START_REF] Oudeyer | Intrinsically motivated learning of real-world sensorimotor skills with developmental constraints[END_REF] to known kinematics and known spatial representations of objects [START_REF] Koppula | Learning human activities and object affordances from rgb-d videos[END_REF]. The idea being that a full autonomous agent is able to develop by itself the progressive set of required robotic priors to go from development phase to another such is in the overlapping wave theory described by R. S. Siegler [START_REF] Siegler | Emerging minds: The process of change in children's thinking[END_REF].

The bottom-up approach

Modern approaches in developmental robotics emphasize on the importance of "bottom-up" processes for which information is gathered from the lower levels of the interaction, such as uninterpreted sensorimotor interaction, and processed "up" to an integration at the cognitive level which can then be transmitted back in the more classical "top-down" way as feedbacks for controlling lower level processes. These approaches require the robotic agent to be situated, i.e. able to perceive and act in its environment, but also embodied, Chapter 1. Introduction i.e. its interaction is performed through a physical body, embedded in the environment. Hence, the agent's knowledge about the world should start from the lowest interpretation possible as in sensorimotor approaches. Therefore, in this context of cognitive development, perception becomes an emergent property of the agent's body interaction with the world.

Emergence of perception

Before developing the concept of the emergence of perception in the robotic context, let's have a quick look at some philosophical approaches to perception.

A philosophical glimpse to perception

In philosophy, the question of the nature of perception is at the root of the notion of consciousness. Indeed, when we, as human beings, interact with the world, our cognitive system represents its physical processes into feelings and phantasms that shapes our decisions. While these perceived states are based on our sensations, the "raw" uninterpreted electrochemical signals coming to the brain, they may not be explained in terms of their physical origin. Indeed, our sensors are imperfect: they are discretized along our body, our eyes have blind spots but still, the internal images feel continuous and almost perfectly tuned for our necessary interactions with our environment [START_REF] O'regan | Why red doesn't sound like a bell: Understanding the feel of consciousness[END_REF]. This fascinating process of creation of meaning from physical signals is very difficult to grasp scientifically and philosophically such that it has even been stated as the "hard problem of consciousness". "Hard" meaning that the explanation of consciousness is beyond the usual methods of science, in opposition to the "easy" problems that results from the direct acquisition and exploitation of the information present in the sensory inputs that can be done by the computational and neural mechanics in the brain [START_REF] Chalmers | Facing Up to the Problem of Consciousness[END_REF]. If such a hard problem exists, then perception cannot emerge from scientific principles. However, the existence of the hard problem is controversial and has been disputed, in particular by some cognitive scientists such as S. Dehaene.

An information based theory of consciousness

By trying to explain the neurological origin of consciousness, biologists have searched for the neurobiological events that occurs when experiencing subjective consciousness which they described as neural correlates [START_REF] Koch | The quest for consciousness[END_REF]. However, as pointed out by S. Dehaene in his book 'Consciousness and the Brain' [START_REF] Dehaene | Consciousness and the brain: Deciphering how the brain codes our thoughts[END_REF], correlation does not imply causation so that the measurements of neural signatures of consciousness are insufficient to explain the origin of a subjective experience. However, in its version of the Global Workspace Theory, initially proposed by B. J. Baars [START_REF] Baars | Global workspace theory of consciousness: toward a cognitive neuroscience of human experience[END_REF], S. Dehaene links consciousness to a functional process that combines selection of information, which is then broadcasted across the brain, and self-monitoring of this information, which is the capacity of referring to itself. Hence, he postulates that consciousness can be explained as a mathematical theory that could be applied to artificial agents: "As consciousness theory improves, it should become possible to create artificial architectures of electronic chips that mimic the operation of consciousness in real neurons and circuits. Will the next step be a machine that is aware of its own knowledge? Can we grant it a sense of self and even the experience of free will?."

In this context, at least, the quest for building conscious artificial agents is not vain. We can notice that this approach regards cognitive processes as information processing mechanisms and is in line with the Bayesian theories of cognitive processes [START_REF] Friston | The free-energy principle: a unified brain theory?[END_REF], Friston, 2012[START_REF] Bessière | Modèles probabilistes formels pour problèmes cognitifs usuels[END_REF] which have seen recent applications with the so-called active inference in developmental robotics [START_REF] Pio-Lopez | Active inference and robot control: a case study[END_REF].

Differently to the view of a brain centered approach to consciousness, an other paradigm places the body and its interactions at the center of emergence of cognitive process.

An embodied theory of perception

The crucial importance of experience in cognitive processes can be found in the phenomenological approaches of perception developed by M. [START_REF] Merleau-Ponty | Phenomenology of perception[END_REF] which has, in turn, inspired the theories of embodied cognition or enactivism (Varela, Thompson, and Rosch, 2017). The enactive approaches address the hard problem of consciousness by assuming that cognitive structures emerge, from the dynamical interactions of an embodied agent with its environment, in the sensorimotor patterns of neural activities. They are the natural consequence of an autonomous agent maintaining its structure while in constant dynamical interaction with an environment.

At the intersection of the sensorimotor interaction and the enactive approach is the sensorimotor theory of perception introduced and developed by A. Noë and K. J. O'Regan (J K O' Regan and[START_REF] O'regan | A sensorimotor account of vision and visual consciousness[END_REF][START_REF] O'regan | Why red doesn't sound like a bell: Understanding the feel of consciousness[END_REF]. In the sensorimotor approach to perception, the perceptual experience is seen as the mastering the sensorimotor dependencies, called sensorimotor contingencies. By characterizing the subjective experience of "feeling" with the intrinsics properties of the interaction of the body with its environment, then the problem of finding an physical origin to perception is solved.

The robotic approach to sensorimotor perception 1.2.1 The schematic view of the sensorimotor perception in robotics

The enactive approach and the sensorimotor theory are initially planned to explain cognitive processes in living being but are relevant in the context of robotic agents. Indeed, robotics agent share the embodiment and situatedness properties of living beings, moreover, they have the convenient properties of possessing measurable streams of data and known computation resources. In term of pure schematic representation of a robot, it can be modeled as a black box, receiving inputs from sensors placed on a body, embedded in a environment, and generating commands to actuators, that controls the configurations of its body, as its outputs. Following the sensorimotor approach, perception emerges from the laws between commands and sensory inputs which are can be fixed and memorized in the form of internal representations of the interaction of the body with the world. Following the classical sense-plan-act paradigm of perception in robotics, intrinsic tasks and goals can then be shaped into these internal representations and applied in accordance with a learned internal dynamical model of motor control as can be seen in Figure 1.2. One can notice that, in opposition to the classical perception loop in Figure 1.1, the representations of the world and the agent's body are no more given as prior information but rather constructed internally from the laws governing inputs and outputs.

Towards the perception of space

While the sensorimotor approach to perception proposes some answers to a very vast field of cognitive processes, our interest is to develop one of the most crucial type perceptions required in active agents, namely the perception of space.

The structure of space as an empirical construction of the mind has largely influenced Henri Poincaré in its developments of models of non-Euclidean geometry [START_REF] Poincaré | L'espace Et la Géométrie[END_REF]. It is certainly not a coincidence that Husserl, in its book 'Ideen I' [START_REF] Husserl | Idées directrices pour une phénoménologie pure et une philosophie phénoménologique[END_REF](Husserl, published in 1913) ) at approximately the same period, poses parallels between geometry and the phenomenological approach to consciousness. More generally, the developments in philosophy and geometry have always been intertwined since Thales and the first Greek writings [START_REF] Torretti | Philosophy of geometry from Riemann to Poincaré[END_REF]. In its article "L'espace et le géométrie" published in 1895, H. Poincaré formulated an intuition about how the human mind can construct the geometrical properties of space. He stated that the compensation of the variations of sensory inputs after the rigid motion of a rigid object in space, as shown in Figure 1.3, provides an internal, kinesthetic representation of the rigid transformations in space. The group of rigid motions being the only required element to develop a geometrical model of a constant curvature space. By interpreting the strict sensory invariance as a particular form sensorimotor dependencies, recent approaches in robotics has applied the sensorimotor perception for the discovery of the geometrical properties of space [START_REF] Philipona | Is there something out there?: Inferring space from sensorimotor dependencies[END_REF][START_REF] Laflaquière | A non-linear approach to space dimension perception by a naive agent[END_REF][START_REF] Terekhov | Space as an invention of active agents[END_REF]. These approaches had the merit of working with any sensor apparatus, quasi-uninterpreted sensory inputs and any geometrical spaces.

Before space: the physical structure of moving sensors

The problem with the previous approaches on the discovery of geometrical properties of space from sensorimotor inputs, is that the motor exploration required to obtain perfect compensation is very tedious. Indeed, imperfect sensory comparisons can lead to disaster in the internal representations. Moreover, in the same vein, A. Laflaquière et al. have approached the representation of space from the internal representation of the agent's spatial configuration of its sensors [START_REF] Laflaquière | Learning agents spatial configuration from sensorimotor invariants[END_REF]. This approach has been simulated for the case of a serial agent with revolute joints, endowed with a retinal like sensor which is sensitive to illumination in the environment. After a step of motor exploration, they have projected, inside the space of explored joint angles, the obtained sensory inputs. Every set of joint angles representing a 'single sensory input' are grouped together and structured with and internally computed metric. Then, in the case of a "rich enough" environment, each sensory input corresponds to unique position of the sensor in space. Hence, the obtained mapping from multiple joint angles to a single sensory input reproduces the forward model from joint angles to sensor position. While being a very interesting conceptual approach to the question of representation of the sensors physical space, the proposed approach suffers from a lack of formalization. Indeed, it does not allow a more general interpretation of what is being represented in the case of the interaction with a more complex environment. The current research work shall attempt to fill that gap.

The bootstrapping scenario in robotics

More generally, a robotic agent that starts its life from scratch, receiving quasi-uninterpreted inputs from unknown sensors, generating commands without knowing their impact in the world, while trying to perform useful tasks, is also known in robotics as the bootstrapping scenario of learning (Pierce and B. J. [START_REF] Pierce | Map learning with uninterpreted sensors and effectors[END_REF]. The bootstrapping problem ask for the Chapter 1. Introduction creation of a "universal learning agent" such that, given an unknown body with unknown sensors and actuators, the agent is able to learn to use them. However, is it possible to obtain a universal learning algorithm that would allow an agent to works with range finder data or from the RGB stream from camera ? Then what are the minimal a priori required for such agent to work ?

In a series of papers about Spatial Semantic Hierarchy (B. Kuipers andByun, 1991, B. Kuipers, 2000, B. J. [START_REF] Kuipers | Bootstrap learning of foundational representations[END_REF], B. Kuipers, 2007, B. Kuipers, 2008) D. Pierce and B. Kuipers describe a way to obtain successive hierarchically organized set of abstract representations on the sensors and actuators that can be used to perform various navigation tasks. More recently, a well formalized approach to the bootstrapping problem in the context of robotics vehicles has been proposed in the PhD Thesis of A. [START_REF] Censi | Bootstrapping vehicles: a formal approach to unsupervised sensorimotor learning based on invariance[END_REF] In his work, A. Censi describes the influences in the choice of semantics required to interpret sensory inputs in terms of invariance in the representation. Indeed, sensory inputs cannot be totally uninterpreted. In order to be perceptually integrated, it is important that the computations in the "brain" of the agent and the interpretation of inputs are defined with the same semantic rules. Therefore semantics can be seen as axioms of perception. In the current approach to keep at a minimum the a priori information, we will emphasize on the quasi-uninterpreted nature of the sensory inputs with minimal assumptions for the interpretation of sensory inputs.

The current work can be seen as a continuation from the developments of A.

Laflaquière in its PhD thesis (A. [START_REF] Laflaquière | Approche sensorimotrice de la perception de l'espace pour la robotique autonome[END_REF] while incorporating some concepts from the bootstrapping scenario in robotics.

The framework: a formal approach to the emergence of spatial representations

In order to develop a formalism in the context of the emergence of spatial representation, we shall place ourself in a well-defined framework.

The agent

The agent is assumed to be embodied in a physical environment by the mean of a fully controllable body. The agency of the body, i.e. its spatial state in the physical world, is assumed to be completely described by the state of its motor system. This naturally excludes the agents capable of locomotion in the world. Ideally, the case of agents with ability of full body displacement should come as a natural extension of the current work.

The motor state of the agent is assumed to be transmitted, as an efferent copy of the motor commands, through a set of kinesthetic sensors and is called the motor configuration. These sensors are only sensitive to the agent's internal state and their inputs, called proprioceptive inputs, are directly connected to the agent's "brain".

Additionally, the agent is assumed to have sensors on its body that are sensitive to the physical properties in the environment. This includes possible self-interactions as the agent's body is itself a part of the environment. These sensors generate what is called the sensory inputs.

The sensory stream can thus be separated into a proprioceptive (motor configurations) and an exteroceptive (sensory inputs) part which form the sensorimotor inputs.

In the previous approach, the separation of proprioception and exteroception streams of information was one of the main controversy as there are no biological justification for such a distinction [START_REF] Gapenne | The co-constitution of the self and the world: action and proprioceptive coupling[END_REF]. Indeed, its not clear where proprioception starts and exteroception stops in the flow of sensory information because sensors can be sensitive to both body changes and environment changes. One direct example is when one's hand passes in the field of view. Another example is the information that comes from torques in the agent's actuators. Indeed, when the agent only interacts with the physical force of gravity, torques are directly representative of the body's spatial agency as gravity shall be a constant physical property. However, if the agent interacts with a physical object in the environment, torques can be seen as an exteroceptive information because they react to a variation in the environment physical properties. However, this multi-modality of certain sensors with both proprioception and exteroception can contain useful information Chapter 1. Introduction in the context of the discovery of the self [START_REF] Yoshikawa | Does the invariance in multi-modalities represent the body scheme?-a case study with vision and proprioception[END_REF] and shall even be exploited later in this work, for the discovery of selfinteraction. Nevertheless, there have have been recent works in developmental robotics that justify the separation between pure proprioception and pure exteroception [START_REF] Schmidt | Bootstrapping Perception using Information Theory: Case Studies in a quadruped Robot Running on Different grounds[END_REF]. Indeed, by using information theory, proprioception can be separated from the exteroception from its causal properties between sensory changes and generated commands.

The formalism: MPhiES

A formal way to see the previously introduced agent is by the MPhiES relation.

• A motor configuration is written m and the set of all motor configurations is noted M and is called the motor configuration set.

• A sensory input is written s and the set of all sensory inputs is noted as S and is called the set of sensory inputs.

• The state of the environment is written ε and the set of all possible environmental states is noted E and called the environment set.

A unique sensory s input is obtained for each pair of motor configuration and environmental state (m, ε). The uniqueness of the sensory input allows for the definition of the sensorimotor function noted Ψ. Therefore, the relation between s, m and ε can be formalized such that,

s = Ψ(ε, m) = Ψ ε (m). (1.1)
The relation between the sets can be summarized in the following diagram

M S Ψ ε . (1.2)
Note that in the current formalism, the sensorimotor function shall not depend on time. Therefore, at fixed motor configuration, a sensory variation is only provoked by a change in the environmental state.

1.3. The framework: a formal approach to the emergence of spatial representations 15

General assumptions

On the commands and proprioceptive inputs

The agent can generate commands and, in the current approach, these commands are first generated in a naive way. Indeed, the motor control part of the agent can be seen as basic on/off switches linked to its actuators. When all commands are off, then the obtained motor configuration is assumed perfectly fixed and stable.

The agent is then assumed to be able to detect any change in its motor configurations. However, as a naive agent at the beginning of its life, it is not allowed to know the structure of its motor configuration space. Therefore, if the agent's actuators are revolute joints, then it does not have the concept of angles. So, if a proprioceptive input is π/2 radian it does not mean anything for the agent, except that it is a fixed motor configuration. Moreover, all inputs from all actuators are regrouped into a single variable m which is either fixed or changing. In this context, motor control is very difficult but it provides a very general approach, valid for any actuators and any architectures.

On the sensory inputs

Similarly to the proprioceptive inputs known as motor configurations, the agent must be given an a priori notion of changes in the set of sensory inputs. These changes must provide the agent with a computational interpretation of sensory variations. These a priori rules are necessary for the integration of information and can be called semantics B. [START_REF] Kuipers | The spatial semantic hierarchy[END_REF] In the current approach, they are given as an operator δ between sensory inputs.

δ is called the comparison operator and its value is the only input used for computations by the agent. In this context we can perfectly master the amount of prior information given to the system, as computational rules. Indeed, the semantics obtained from δ can be sorted by the size of their group of symmetric transformations on the set of sensory inputs as described in Censi, 2013p. 33. These symmetries of δ are all the transformations in the set of sensory inputs S that does not change the results of the operator δ. As an example, if the set of sensory inputs is given to be in the interval [0, 1] and the comparison operator between two elements s, s ∈ [0, 1] is the relation 'bigger than' such

Chapter 1. Introduction that δ(s, s ) =    1 if s < s , 0 otherwise. (1.3)
Then, the boolean values of the sensory comparison are independent of certain transformations in the set of sensory inputs. As an example, the values of the comparison 'bigger than' are the same when all the sensory inputs are increased by a constant, because this transformation preserves the ordering between pairs of sensory inputs. Therefore, this transformation on S is said to be part of the group of symmetries of δ, i.e. transformations that does not change the results of δ. Then, the bigger is the set of symmetric transformations, the more robust is the design of the agent to any change in the sensory apparatus.

The sensory comparison is a sensory interpretation, it forms a perception, as an operation on raw and uninterpreted sensory inputs. The manifestations of this sensory comparison are well studied in the field of psychophysics. Indeed it is well known that humans might not always be able to perfectly distinguish stimuli with close intensities. This fact has been modeled with psychophysical laws that give the probability for two stimuli of different intensity to be distinguished (Fechner, D. H. Howes, and Boring, 1966). In this case, the comparison operator is a fuzzy relation.

The sensory inputs always come with a predefined "format", as an example pixels values are generally encoded with a set of bytes that are interpreted as a quantity which can be compared. Other typical choices of format are the set of real numbers or the set of natural numbers. This also comes as a priori, a choice of format implies a choice of comparison operator. Indeed values of real numbers can be compared, distances can be measured, etc. However, in the context of robotics bootstrapping, the agent should be able to work with any type of sensor, so that the amount of a priori in the sensory interpretation must be reduced to a minimum. Therefore, the most basic interpretation possible can be obtained when the sensory inputs are just considered as symbols.

Then, the agent can only interpret if two sensory inputs as equal or different 1.4. Contributions so that the comparison operator δ can be defined such that,

δ(s, s ) =    1 if s = s , 0 otherwise. (1.4)
This kind of agent requires the less amount of a priori for the sensory interpretation which makes it a serious attempt to define a universal agent.

The main challenge of the current research work is to evaluate how much can be interpreted as information about the world with a minimum amount of a priori in sensory interpretation.

Contributions

The contribution of the proposed research work can be summarized into the following contribution.

• A general mathematical formalization of the sensorimotor interaction of a naive agent with an unknown environment is proposed. This formalization naturally extends the previous work from A. Laflaquière as it takes into considerations the variations in the environment.

• An active process of refinement is introduced in the context of a changing environment that allows the naive agent to build an internal representation of the notion of "points" in the physical space. This allows for an intuitive interpretation of the theoretical perceptive limits of such naive agent.

• In order for the agent to be able to reach the previously defined perceptive limits it will be shown that the agent's interactions with its environment must satisfy certain statistical hypotheses. Then, we will propose a method to exploit these statistical properties to allow the agent to bring out a structured representation.

• A theoretical study is then performed and necessary hypotheses are proposed so that the agent's structured representation is a topologically accurate representation of the displacement of its sensors in the physical space. Thus, from its interaction with the environment, it will be Chapter 1. Introduction

shown that the agent is able to represent the physical continuity of its sensors displacements.

• An experimental framework is then proposed and the refinement process is adapted to a realistic situation. Thanks to the previous considerations, the obtained internal representation can then be easily interpreted and two criteria will be proposed to evaluate its accurateness towards application such as path planning.

• An application for the discovery of the sensors' spatial structures is presented, which is fully based on the properties of the previously obtained internal representations. An other application is then proposed, which allows the agent to discover the notion of self-interaction.

Part I

Theoretical developments

Chapter 2

The Refinement Process

In this chapter, we shall describe what are the theoretical possibilities of the agent's internal constructions based on the proposed framework. The discussion is brought from both external and internal point of views. The internal point of view deals with the agent's internal experience and information integration from generating commands and receiving sensorimotor inputs while the external point of view is used for interpretation, evaluation and statement of necessary assumptions on the physical world.

In the current approach, the agent is able to explore its set of motor configurations by generating naive commands, the sensors are then moved in the physical space and send sensory inputs depending on the environmental physical state. It is then able to interpret the variations of sensory inputs through the comparison operator. No other interpretation in the sensory inputs is assumed possible. Therefore, the main question arises: what can the agent represent just from the interpretation of sensory inputs equality ?

In this chapter, the answer is given in a theoretical way that will serve later as a reference. Through a process called the refinement process, it is demonstrated that the agent is only able to obtain a specific set of perceptive elements, or perceptive atoms, which can be formally defined from its limited sensory and motor possibilities.

In this chapter, we shall provide an external point of view about the notion of sensory invariance that can be obtained in this context. Then, from an internal point of view, the refinement process is defined and formalized. This obtained perceptive atoms are then interpreted and placed in correspondence

Chapter 2. The Refinement Process with the external notion of sensors' poses in the physical space.

The sensorimotor types of invariance

The agent being in interaction with a physical environment, its sensory invariance, obtained with the inequality operator δ, depends on its sensory and motor architecture as well as the physical properties in the environment.

We shall define them in this section.

Because the theoretical developments might be quite technical, we shall represent the different notions with the same illustrative examples that will used during all the presented work. However, one has to keep in mind that the validity of the introduced notions largely exceeds the context of application of the presented simple examples.

A simple illustrative example

In order to make the concepts used in this chapter more intuitive, we will use a running example with a simple agent in interaction with a basic environment.

The illustrative agent

The majority of the results will be explained through a simple planar agent as shown in Figure 2.1. This illustrative agent is composed of two serial arms placed in the xy-plane, both are of length l and are controlled by two revolute joints. The motor configuration is parametrized by the tuple (m 1 , m 2 ) where m 1 , m 2 ∈]π, π] are the angles, in radian, of the joints with respect to the the x-axis. The environmental states are simply any gray scale background of the plane. Therefore, the sensor apparatus is made of a single-pixel camera which sends a value corresponding to the illumination of the background.

For illustration purposes the single-pixel camera is assumed to measure illumination at a single point in space so that the pixel does not have any real spatial spread. The sensor's pose can be parametrized by a tuple (x, y, θ) corresponding to its Cartesian coordinates and orientation in the original frame (e x , e y ) attached at the pivot point of the first revolute joint. Here the sensory inputs are obviously invariant to a change in orientation θ. This will be used later to illustrate a category of sensory invariance based on sensors symmetries.

The illustrative environment

In the illustrative examples, we will mainly consider environments with binary colors white and black (usually shown in gray in the figures) in the Euclidean space. The first environmental state is called ε 1 .

In Figure 2.2(a) is shown an environmental state ε 1 with a black blob which intersects with the lower left area of the agent pose space and two black blobs on the outside of the pose space that can't be sensed by the agent. Hence, from the sensorimotor point of view of the agent, the motor configurations leading to a sensor pose directly on the black blob will generate a 'black' sensory input while the ones in the white area will generate a 'white' input. Here, one can represent the current sensorimotor state in the motor configuration space, e.g. a square of side ]π, π], by representing the sensory inputs for each motor configuration.

The theoretical sensorimotor state

In Figure 2.2(b) is represented all the sensorimotor information the agent can theoretically access after interacting in an environment state ε 1 . The motor configurations leading to a 'black' sensory input are therefore in a sensory invariance, as the application of the comparison operator δ for any pairs of these sensory inputs give zero. For the agent, these sensory invariant configurations are said to be sensory equivalent. Then, the current sensorimotor state can be represented by two clusters in the set of motor configurations corresponding to the motor configurations giving a 'white' and a 'black' sensory inputs.

FIGURE 2.3: Perfect environmental state in terms of separation of the sensors spatial configurations. LEFT, for each position of the sensors there is a different color. RIGHT, the sensorimotor state of the agent after exploring all its motor configurations. Assuming the sensors has perfect resolution, each sensor's position in the agent's working space generates a single sensory input.

A perfect environment ?

If the environmental state is 'rich enough', it is possible that the environment already distinguishes the highest possible number of motor configurations by their generated sensory inputs. This is the case of the color wheel represented in Figure 2.3, assuming the camera is perfectly sensitive to the colors.

In this example, each point in space reachable by the sensors give a different sensory input.

However, this case if not very likely to occur. Moreover, real sensors generally have a limited resolution and their inputs may be quantified so that the set of all sensory inputs is generally not infinite. In such a case, the agent would not be able to distinguish, in one environment, all the sensor positions in space.

The fact of regrouping sets of sensory invariant motor configurations, at a fixed environmental state, is called a motor configuration sensory equivalence and shall be formalized using the mathematical tools of equivalence relations in the next section.

Types of sensory invariance

For a fixed environmental state, multiple motor configurations can generate the same sensory inputs. Such sets of equivalent motor configurations are call the motor configuration sensory equivalence sets or motor equivalence sets for short. All the motor configurations inside such sets are equivalent from the internal point of view of the agent as it cannot distinguish them from their generated sensory inputs. These motor equivalence sets can be categorized into three different types: spatial sensory invariance, kinematics redundancy and sensors symmetries as represented in Figure 2.4.

Spatial sensory invariance

The first sensory invariance is due to the physical properties of the environmental state in which the agent is in. In a fixed environmental state, the sensors, while in different configurations in the physical space, can generate the same sensory inputs. The physical properties of the environment are redundant in space. This sensory invariance is called the spatial sensory invariance and is presented in SubFigure 2.4(a) with the illustrative agent. This sensory invariance is the only one that has an external origin, i.e. it varies with the environmental state. The other types of sensory invariance can be deduced from the intrinsic architecture of the agent.

Kinematics redundancy

The second sensory invariance comes from the kinematics of the agent. The mechanic architecture of the agent can have redundancies. Different motor configurations are be mechanically redundant when the obtained sensor's spatial configurations are exactly identical. This is shown in SubFigure 2.4(b) for a similar agent with a 3rd degree of freedom: the position and the orientation of the sensors for the presented motor configurations are the same. The kinematics redundancy is an intrinsic property of the agent's mechanical architecture, therefore it is not dependent on the environmental state. Another sensory invariance which is also independent of the environmental state is due to the symmetries of the sensors.

Sensors symmetries

The sensors symmetries correspond to the sensory invariance obtained when the sensors undergo a spatial transformation which does not impact the generated sensory input. One can take the example of a temperature sensor, which when rotated, always sends the same sensory information. In SubFigure 2.4(c) is shown the same kind of symmetry for the single pixel camera.

The sensor symmetries are separated from the kinematics redundancies for reasons of clarity in the interpretation of the motor equivalences. Indeed, it is also possible to regroup the sensory invariance into the two classes: intrinsic (that doesn't depend on the environmental states) and extrinsic (that does depend on the environmental states).

Chapter 2. The Refinement Process

Towards a representation of sensitive sensor's spatial configurations

Because kinematics redundancy and sensors symmetries are intrinsic properties of the agent's mechanical structure, they cannot be "refined", i.e. the agent will never be able to distinguish the motor configurations that are inside these classes of sensory invariance. However, this is not the case for the motor configurations sensory invariance caused by spatial redundancy. Indeed, when interacting with the environment, the spatial sensory invariance will naturally evolve with the environmental states.

The idea of refinement

Therefore, it is theoretically possible for the agent to progressively distinguish the equivalent motor configurations by remembering the equivalences successively obtained with different environmental states. If the agent is able to do so, then it should be left with the intrinsics invariants from the kinematics redundancy and sensor symmetries. Furthermore, the final sensory invariants should correspond to the sensors' spatial configurations that generates different sensory inputs, or said differently, the sensitive spatial configurations of the sensors. We can notice that these sensitive spatial configurations will be represented as clusters inside the set of motor configurations. Hence, they are theoretically fully a accessible from an internal point of view if the agent us able to "sufficiently" interacts with the environment.

Illustration of the refinement idea

Let's place the agent in two different environmental states ε 1 and ε 2 as shown in Figure 2.5(Left). Theoretically, for each environmental states the agent can access the sensorimotor states presented in Figure 2.5(Right). Hence, if the agent is able to remember the sensory equivalent motor configurations from both environmental states, then the agent can obtain new sensory invariance based on the sequence of sensory inputs obtained at each motor configuration. In Figure 2.6, is represented with colored areas, the sensory invariants obtained with the composition of the two environmental state {ε 1 , ε 2 }. We can notice that even if both environmental states presents the same 'black' colors, each spatial invariant must be specific to a single environment. The approach of sensory invariance is, there, not very intuitive because the pertinent information is sensory invariance between the motor configurations but not the actual values of the sensory input. This way, changing 'black' to 'gray', or even inverting 'white' and 'black' in either environment doesn't change anything from the agent point of view as such transformations are in the group of symmetries of the comparison operator δ. Thus, pertinent information can only be obtained from sensory invariance between motor configurations in a fixed environmental state. Moreover, in Figure 2.6 on the right, the color code corresponds to the following sensory sequences: in white is represented the sensorimotor states 'white' at ε 1 followed by 'white' at ε 2 . In light gray is represented 'black' at ε 1 and 'white' at ε 2 . In dark gray is represented 'white' at ε 1 and 'black' at ε 2 and finally in black is represented 'black' for both environmental states.

2.3.

Throughout the interaction with different environmental states, the motor configurations are refined into multiple clusters each corresponding to unique motor configuration invariant set. These sensory equivalent clusters of motor configurations are represented in Figure 2.7. Moreover, each cluster is representative of a region of the sensors spatial configurations. As more and more environmental states are explored, all the sensory invariants clusters will inevitably shrink as well as their corresponding represented regions of sensor spatial configurations. If there are sufficient environmental states, the obtained sensory invariants clusters can correspond to the one obtained from the single "perfect" environment case shown in Figure 2.3 where each sensor position gives distinct sensory inputs.

In the general case, it is not obvious what is represented by the sensitive sensor's spatial configurations, hence, a theoretical framework shall help us to understand the general case with unknown kinematics and unknown sensors.

Motor configuration invariants Environmental states FIGURE 2.7: Representation of the refinement of the motor configuration set through the successive environmental states ε 1 and ε 2 . At the beginning, all motor configurations are equivalent, then they are partitioned into an increasing number of sensory invariance clusters when the number of experienced environmental states increases.

The refinement process

The process of refinement of the sensory classes should allow the agent to obtain an internal representation of the space of its "sensitive" sensors spatial configurations. This is due to the progressive disappearance of the extrinsic type of sensory invariance along with the successive interactions. Th refinement is obtained by progressively integrating the sensory invariants obtained at different environmental states. As the agent interacts with more and more environmental states, the motor configuration invariants sets inevitably shrink until they are eventually reduced to a not-refinable fundamental part. This process of refinement can be formalized by using the mathematical tools of equivalence relations and quotient sets.

Formalization of the sensory invariance

Sensory invariance as equivalence relations

For a fixed environmental state ε ∈ E , the sensory invariance forms an equivalence relation = ε on the set of motor configurations. Pairs of motor configurations are considered to be equivalent when they generate the same sensory

[m] ε = {r ∈ M; r = ε m}. (2.2)
The set of all the equivalence classes for the environmental state ε is called the quotient set M/ ε and is defined as

M/ ε = {[m] ε ; m ∈ M} (2.3)
Thus, elements of the set M/ ε are subsets of motor configurations. An element m ∈ [m] ε is called a representative of the equivalence class [m] ε . Any element of the class can be chosen as a representative of the class.

The multi-environment case

When the agent explores other environments, we can extend the sensory equivalence for multiple environments. Let's call E ⊆ E a subset of environmental states, then a sensory invariance obtained for all the environmental states in E can also be represented via an equivalence relation = E between motor configuration. Formally, for any pair of motor configurations

m, m ∈ M, m = E m if and only if ∀ε ∈ E, Ψ ε (m) = Ψ ε (m ).
(2.4)

Similarly to the previous notation, let's write as [m] E the equivalence class of m defined as [m] E = {r ∈ M; r = E m}.

(2.5)

These equivalence classes are the intersection of all the equivalence classes obtained for each environmental states. Hence, for a configuration m ∈ M,

we have that [m] E = ε∈E [m] ε . (2.6)
Then, let M/ E be the quotient set for this multiple environmental states equivalence relation, it is defined as

M/ E = {[m] E ; m ∈ M} (2.7)
The concept of partition and their ordering

The quotient set M/ ε is a family of subsets of M that forms a partition of M. Definition 1 (Partition of a set). A family of subsets of a set is said to form a partition if it is composed of non-empty pairwise disjoint subsets whose union forms the set1 .

As an example, partitions of the set {0, 1, 2, 3, 4, 5} can be the family of subsets {0, {1, 2}, {3, 4, 5}} or also {{0, 1, 2}, {3, 4, 5}}. However, the family {{0, 1, 2}, {2, 3, 4, 5}} is not a partition because 2 is found in two subsets and the subsets are not pairwise disjoint. The partitions are the heart of the mathematical concept of refinement because they naturally represent the separation of a set into smaller subsets.

An equivalence relation naturally creates a partition on the set on which it is defined. As an example, for the equivalence relation = ε , any element of the motor configuration space M belongs to one and only one equivalence class. So that M/ ε forms a partition of M. More generally, for any subset of environmental states E ⊆ E , the quotient set M/ E is also a partition of M. However, the more environmental states are experienced, the more "finer" should be the obtained partition.

Definition 2 (Partition refinement). Let α and β be two partitions of the set M. α is said to be a refinement of β if every element of α is in a subset of some element of β.

Alternatively, α is said to be finer than β, respectively β is coarser than α. The finer-than relation is a partial order binary relation so it can be written α ≤ β.

Chapter 2. The Refinement Process

As an example, the families of subsets

α = {0,
(m ) = Ψ ε (m ). In particular, because E ⊆ E we have that ∀ε ∈ E, Ψ ε (m ) = Ψ ε (m ) so that m = E m . Therefore m , m ∈ [m] E , thus [m] E ⊆ [m] E . So, any equivalence classes in M/ E is inside of an equivalence class in M/ E .
The limit case is obtained when the sensory invariance correspond to the whole possible set of environmental states E = E . The resulting equivalence relation is noted = E and the quotient set M/ E is called the fundamental motor configuration quotient set or when there is no ambiguity the fundamental quotient set. This is also a partition of M and in fact it is a lower bound of the "finer than" relation between the partition, i.e. this partition is always a refinement of any partition obtained for any subset of environmental states. Additionally, the trivial partition of motor configuration {M} is obviously an upper bound.

The set of all the motor quotient sets together with the finer-than relation ≤ form a partially ordered set. This means that certain points can be ordered with a ≤ relation. The word "partial" refers to the fact that some points cannot be compared, this is different from a to "totally" ordered set where each pair of points can be compared, such as in the real line with ≤: (R, ≤). When a partially ordered set possesses an upper and a lower bound, it can also be called a lattice. Here the lattice of quotient sets with the "finer than" relation is noted L, with its greatest element being the trivial partition {M}, i.e. a single block of all the motor configurations, and its lowest element is the fundamental quotient set M/ E . Hence, for any subset of environmental states E ⊆ E we have M/ E ≤ M/ E ≤ {M}.

(2.8)

We can also notice that because of the possible intrinsic sensory invariance (kinematic redundancy and sensors symmetries), that agent may not be able to refine certain subsets of motor configurations. This can be formalized such that the fundamental quotient set M/ E is also refined by the set of all the singleton elements of the motor configuration set M and

M ≤ M/ E ≤ M/ E ≤ {M}, (2.9)
for any subset E ⊆ E .

The equivalence classes in the fundamental motor configuration quotient M/ E sets can naturally be considered as the atoms of the agent's perception. They regroup the motor configurations into "perceptive points". We

Chapter 2. The Refinement Process shall now formalize what these not-refinable points represent from an external point of view.

An outside point of view of the refinement process

We have shown in the subsection §2.3.2 that illustrates the refinement, that each sensory invariant represents a region of the physical space. To be more precise, these regions represents the positions reached by the sensors in the physical space. By adding the forward kinematics in the formalization we are able to interpret the equivalence classes in terms of sensors' poses.

The sensors' pose set and the formalism MXS

In the current framework, the sensors are attached to the controllable body of the agent. Their states in the physical space are assumed to be fully captured by the parameter x ∈ X called the pose of the sensors and X is the pose set.

Usually in robotics, a sensor pose corresponds to its position and orientation in a robot reference frame.

Then, the pose of the sensors is fully controlled by the motor configurations of the agent. The function that links the motor configurations to the sensor poses is called the forward model and is noted f such that x = f (m).

(2.10)

In the formalization, the pose set X represents all the reachable poses by the agent so that X = f (M) with f being surjective (equivalently onto). Hence, each pose x ∈ X always has at least one preimage in M by f . Moreover, the sensory input s is linked to the sensors' pose x and the environmental state ε through the forward sensory function: φ ε such that s = φ ε (x).

(2.11)

Then, the sensorimotor function Ψ ε that directly links the motor configurations to the generated sensory input, in a specific environmental state ε, can be written as the composition

Ψ ε = φ ε • f . (2.12)
Therefore, the relation between the sets can be summarized in the following diagram

M X S Ψ ε f φ ε . (2.13)
The different intrinsic sensory invariance can be formalized as properties of non-injectivity of the forward model f and the forward sensory function φ. Indeed, the kinematics of the agent possibly contains redundancies so that a unique sensors pose can be obtained from different motor configurations. Moreover, two different poses can generate the same sensory input due the sensor smymetries. We can notice that these intrinsic sensory invariants do not depend on the environmental state.

Similarly to the refinement in the motor configuration, we can formalize the concept of refinement in the externally defined set of poses S.

The refined pose set

Similarly to previous subsection, the sensory equivalent sensors' poses can be subject to spatial invariance in the environmental states. This extrinsic type of sensory invariance can be refined through the refinement process.

Let's define the sensory equivalence for two poses as the equivalence relation = ε , the ambiguity with the equivalence relation defined in (2.1) is voluntary, such that, for two poses x, x ∈ X ,

x = ε x ⇔ φ ε (x) = φ ε (x ).
(2.14)

Let's now explain why the equivalence notations are the same for both poses and motor configuration.

Chapter 2. The Refinement Process Proposition 2. For an environmental states ε ∈ E and any pair of motor configurations m, m ∈ M, we have the following state of equivalence:

m = ε m ⇔ f (m) = ε f (m ). ( 2
= ε m ⇒ f (m) = ε f (m ).
If a pose x generates a sensory input s = φ ε (x). Then, because the forward model f in surjective, the set of all motor configurations leading to this pose

x is the set f -1 (x). Then naturally, the motor configuration m ∈ f -1 (x) also generates the sensory input s. Hence, for any two poses x, x ∈ X , if they generate the same sensory input, meaning that x = ε x , then all the motor configurations in the preimage of x and x by f also generate the same sensory inputs. So that f (m) = ε f (m ) also implies m = ε m.

Then, for any subset E ⊆ E , the equivalence relation = E defined in relation (2.4) can be extended to the sensors poses such that

x = E x , if and only if ∀ε ∈ E, x = ε x . (2.16)
Then similarly to Proposition 2 there is an equivalence between motor and pose equivalence classes.

Proposition 3. For a subset E ⊆ E of environmental states and any pair of motor configurations m, m ∈ M, we have the following equivalence:

m = E m ⇔ f (m) = E f (m ).
(2.17)

Proof. The proof is similar to the one in Proposition 2.

Hence, by taking the same notations as previously, let's denote by X / E the quotient set of X with equivalence relation = E as the set

X / E = {[x] E ; x ∈ X }.
(2.18)

For the case where E = E , the pose quotient X / E is called the fundamental pose set or the set of sensitive sensors' poses. Each element of this set corresponds to the sensors' poses that cannot be distinguished from a sensory input, however, there always exists at least one environmental states that sensory distinguishes two poses from two different finest equivalence classes of sensors' poses. Hence, the term "sensitive" poses. We have defined externally, the set of sensors' poses than can be theoretically be distinguished from an internal point of view, but we shall explain the relation between this set and the set of motor sensory invariants obtained for a after the full refinement.

Equivalence of the fundamental quotient sets

Let's define the map f E between the quotient sets m/ E and X / E such that

f E : M/ E → X / E [m] E → [ f (m)] E (
= [m] E . Then f E ([m] E ) = [ f (m)] E exists
as f is a surjective map and every motor configuration has an image by f . For any other representative m ∈ k we have that m = E m and from Propo-

sition 3, f (m) = E f (m ) such that [ f (m)] E = [ f (m )
] E and thus f E (k) exists and is unique.

For any r ∈ X , let's take a Representative x ∈ X such that x ∈ r. Then, by surjection of f , the preimage of x by f exists. This preimage is called M such that M = f -1 (x). Then for any m, m ∈ M we have that f (m) = E f (m )) and from the relation 3 this implies that m = E m and so f -1 E ([ f (x)] E ) exists. Then, lets x ∈ X be another representative of r in X and M = f -1 (x ).

Chapter 2. The Refinement Process

For any m ∈ M and m ∈ M , because f (m) = E f (m ) we also have that m = E m . Thus f -1 E (r) exists and is unique and f E is a bijection.

Then, there exists a bijective (one-to-one) relation between the motor and pose quotient sets M/ E and X / E , for any subset E ⊆ E . As sets, they are perfectly equivalent. This applies also for the full refinement obtained when E = E . Then, we can say that the fundamental motor configurations invariant set M/ E represents the sensitive poses of X / E as they share the property of having the same number (possibly infinite) of points.

We have shown that a point in the fundamental motor configurations quotient set represents a region in the sensors' pose set which is not-refinable from a sensorimotor point of view. This result is quite intuitive and one can wonder about the utility of defining the fundamental pose set. The main idea behind the introduction of this set is simply to evaluate the internal interpretation using a external, knowledgeable, view. Indeed, further in this work, we will be faced with the question of how to interpret and evaluate the internal representation using properties naturally present in the physical space. Generally, the question of the evaluation of the representation is very important when dealing with learning techniques, as pointed out in [START_REF] Bengio | Representation learning: A review and new perspectives[END_REF], indeed data representation obtained by using techniques such as unsupervised deep learning are very difficult to interpret, which leads to problems to evaluate their generality. In the present work, the external considerations are introduced to insure that some fundamental properties of the physical are preserved in the representation so that the agent's future behavior can be understood.

So far, the approach has been quite formal and technical. The next subsection illustrates the refinement process from a more intuitive and concrete way in the context of a simple agent.

Illustrative Example of the refinement process.

Let us illustrate the refinement process via the simple agent of Figure 2.1. In this example, the agent is a serial arm with two segments of length 0.5 controlled by 2 revolute joints. It is endowed with a single pixel camera at the end of the arm which is sensitive to the illumination of the background. The set of all reachable sensor's positions on the Euclidean plane forms disk of radius 1, noted D. The agent is then placed in a simple 2 dimensional environment whose environmental states are straight lines separating the plane into a 'black' and a 'white' areas as in presented in Figure 2.8. The agent's sensor sends a 'black' input if the background at its current pose is black and 'white' input if the background is white. When the sensors' position lies directly on the straight line it sends either 'white' or 'black' arbitrarily.

The refinement lattice

At the top of Figure 2.8 is shown three environmental states, parametrized as ε 1 , ε 2 and ε 3 . Then, at the bottom of the same figure are shown the sensorimotor states that could be obtained for each environmental state. The motor configurations generating the same sensory inputs can then be regrouped together, in sensory equivalence classes, for each environmental state. When the agent integrates multiple environmental states, the equivalence classes naturally shrink to form finer partitions of the set of motor configurations

$\pi$ $0$
FIGURE 2.9: Refinement lattice for straight lines environmental states ε 1 , ε 2 and ε 3 . At the beginning, all the motor configurations are equivalent. For each image, the colors depict the equivalence classes, e.g. sets of motor configurations that have generated the same sensory inputs for each environmental states. The ≥ symbol represents the 'coarser than' relation between the quotient sets, see Definition 2.

which are arranged in the form of a lattice, as formalized in the subsection §2.4.1. This construction is shown in Figure 2.9.

The set of sensitive sensors' poses

Now, let's tackle the case where the agent has seen all the possible environmental states, i.e. all the possible straight lines. From an internal point of view, it has reached the lowest element of the lattice, the so called fundamental motor configuration set M/ E . Then, how can we interpret M/ E in the physical space ?

First, lets consider any two configurations m = (m 1 , m 2 ) and m = (m 1 , m 2 ) where m 1 , m 1 represent the angles of the first revolute joint of the agent and m 2 , m 2 the angles of the second joint. These two configurations lead to two sensor poses x = (x, y, θ) and x = (x , y , θ ), each being parametrized by a position in R 2 and orientation in ]π, π]. The link between the sensor pose and the motor configuration is given by the forward model f such that

f ((m 1 , m 2 )) = (cos(m 1 ) + cos(m 1 + m 2 ), sin(m 1 ) + sin(m 1 + m 2 ), m 1 + m 2 ).
(2.20) We are now looking for the conditions under which m and m always generate the same sensory inputs. The sensor is sensitive to the illumination at the end-effector position in the set of all reachable poses D which can also be called the working space. However, the sensor's orientation does not change the sensory input so that if the position of the sensor is fixed while its orientation θ vary then the generated sensory inputs would not change. This type of invariance is known as sensory symmetry. We can notice that if we had not consider the orientation of the sensor in its pose, this sensory invariance would have been considered as a kinematic redundancy. The only requirement for a choice of the pose space is that at a fixed pose and a fixed environment they can only be one and only one sensory input. Furthermore, if the sensor is in two distinct positions on the plane, then, one can always find a straight line in the environment that separates them, as an example the one going through their middle point which exists because they are distinct in the plane. Hence, the only way for two sensor's poses to always generate the same sensory inputs is obtained when they are at the same position in the plane, possibly with different orientations. Then, for the two considered motor configurations m, m and (x, y, θ), (x , y , θ ) the sensor's poses, the equivalence relation = E can be formalized this way: m = E m , if and only if (x, y) = (x , y ).

(2.21)

The equivalence class [m] E represents a unique position (x, y) ∈ D. Therefore, the set of all equivalence classes, the fundamental set M/ E , is directly the working space D of the agent. In this particular example, the agent, which only receives sensory inputs from a single pixel camera and proprioceptive information from its joints angles, is theoretically able to internally represent each end-effector positions. It has removed all the spatial environment invariance (the fact that at each environmental state, a full region of space has the same illumination) and is left with the intrinsic sensory invariants corresponding to kinematics redundancy of the arm (in this simple case they are no such redundancy) and sensors symmetries (rotation of the camera).

Conclusion

In this chapter, we have proposed a formal approach that describes the theoretical perceptive limits of a naive agent in interaction with an environment.

We have shown that, by following a refinement process, the agent can distinguish the intrinsic and extrinsic origin of its sensory invariants. Hence, such agent has the theoretical ability, by progressively computing the sensory equivalence classes in its motor configurations, of obtaining an internal representation of its sensitive sensors' poses.

However, the refined sets of motor configurations might not to be directly linked to what we classically consider as a 'point in space'. Indeed, the agent does not know any details about its sensor apparatus, so that if it has a camera at the end of an arm, there might be an other sensor on an other part of its body sending audio or tactile input. In the current approach, all sensors inputs are regrouped into a single sensory input viewed a single symbol. Hence, the sensitive sensors' poses include the set of all sensors at the same time. Therefore the sensor's physical space has a possibly very high dimension. This makes the interpretation of the fundamental set to be quite unintuitive and complex to visualize.

Moreover, as each internal representation point represents a "sensitive" poses, it is not yet structured, i.e. there are no relations between these points. Such internal representation is not directly exploitable because the agent cannot represent paths or trajectories inside an unstructured set of points.

Additionally, the theoretical refinement process assumes that the agent is not limited in its explorations. Indeed, the physical environment is possibly continuous so that the number of environmental states is possibly infinite and uncountable2 and it is not reasonable to assume that the agent can interact with all of them. In reality, the experience must be constrained to be at most a countable number of experiences, i.e. we should be able to index each experience with an element from the set of natural numbers N = {1, 2, 3, • • • }. However, in this case, the agent may not be able to interact with all the environmental states. The countable nature of experience naturally introduce a possible variability in the agent possible history of experienced and this impacts the nature of the obtained internal representations. When the number of experiences is at most countable, then the agent can't interact with all the different environmental states in one unique life. Therefore, the refinement process can possibly lead to different perceptive structures that depends on the history of the experienced environmental states in the agent's life. This variability can be dealt by introducing convergence properties in the countable refinement case, so that the internal representation must converge towards a unique consistent set of equivalence classes which does not depends on the history of experienced environmental states.

We shall discuss this in the next chapter. Indeed, the refinement may be seen as a stochastic process where the environmental states, and thus the sensory comparisons too, are random variables. Then, some statistical assumptions on this interaction can be introduced so that the internal representation consistently converge towards a unique set of equivalence classes that can be also be used as perceptive atoms.

Chapter 3

Stochasticity on the refinement process

To deal with limited number of the possible agent's experience, we shall introduce a framework that guarantees the convergence of the refinement process into a single consistent fundamental motor configurations set. In this context, distinguishing different agent's possible lives can be seen as choosing a set of environmental states that the agent will experience. To go further, we shall assume that the choice of environmental states follows the laws of a stochastic process. Indeed, from an internal point of view, the current environmental state is not known and its changes can be represented as random changes in sensory invariants.

In this chapter, we shall adapt the previously defined refinement process into a probability theory framework by introducing some probabilistic structures in the environmental states the agent interacts with. The deterministic concept of finest sensory equivalence classes is adapted to account for probabilities in the environment which allows for the definition of a probabilist internal representation. Finally, we shall discuss the statistical properties that the interaction must have to allow the agent to obtain a consistent internal representation, i.e. a unique internal representation obtained irrespectively of the history of environmental states. Therefore, the properties of convergence of the refinement shall be introduced in a stochastic framework.

A probability theory framework for the interaction between agent and environment

In this section, we will use probability theory to formalize the possible interactions of the agent with its environment. Most of the technical developments are from the book "Probability and Measure" by P. [START_REF] Billingsley | Probability and Measure[END_REF].

The probability space of the environment

In the probabilistic framework, choosing a specific environmental states in the whole environment set E can be seen as a random experiment. A random experiment can be modeled by a probability space which consists of three parts:

• A sample space, which is the set of all possible outcomes of the experiment.

• A set of events, which contains grouping of outcomes.

• A probability measure, the assigns probabilities to the events.

Therefore, the sample space is the set of all environmental states E . The fact of choosing one environmental state ε ∈ E is the outcome. Then, we can define the events as the subsets of E . However, the set of outcomes of E possibly being infinite and even uncountable, the set of events shall take the form of a sigma-algebra, i.e. a collection of subsets of E with the following properties:

• It contains the set E itself.

• It is closed under complementation: if it contains a subset of E , it also contains its complement in E .

• It is closed under countable unions: if it contains a countable number of subsets of E it also contains their unions.

Here the chosen sigma-algebra is P (E ) the power set of E , i.e. the sets of all subsets of E , so that an event is simply any subset of E , it is known that such set is a sigma-algebra. An event can take two possible values after the outcome of the experiment: a success or a failure. As an example, the event 3.1. A probability theory framework for the interaction between agent and environment 49

E = {ε 1 , ε 2 , ε 3 } is a success if the outcome of the experiment 'choosing an environmental state in E ' belongs to E, otherwise it is a failure.
So far, everything was already introduced in the framework. We now assume that there exists a probability in the success of the events in P (E ). These probabilities are characterized by the probability measure P on the set of events.

A measure on a sigma-algebra is a way to assign a number to each of its subsets, this can be interpreted as a generalization of the notion of the relative "size" of these subsets. It must satisfy the following conditions:

• Non-negativity: for every subset E of the sigma-algebra P (E ), we have that P(E) ≥ 0.

• Null empty set: P(∅) = 0.

• Countable additivity: for every countable collection {E i } ∞ i=1 of pairwise disjoint sets in the sigma-algebra, we have that

P( ∞ k=1 E k ) = ∞ ∑ k=1 P(E k ).
Here, the measure is normalized so as to be a probabilistic measure, e.g. it satisfies the additional property that P(E ) = 1. Hence, for a subset E of environmental states, the value P(E) represents the probability for a sampled environmental state to be inside E.

We have now defined the probability space for the random experiment of choosing an environmental state as the triplet (E , P (E ), P).

Some illustrative probability spaces

To make the reader more familiar with the concept of probability space, let's use our familiar illustrative planar agent with straight line environments. For the examples, we shall propose three different environments with different probability spaces.

Parametrization of straight lines environmental states

First, we need to model the process of choosing a straight line in the environment set. Almost all the straight lines, except the ones going through the center O of D can be parametrized by a 2 dimensional parameter called the podal point Q, see Figure 3.1. The podal point is the point of the straight line which is the closest to the center O of the disk, there can be only one straight line by podal point. The straight lines going through the center that are removed, with out loss of generality, from the set of environmental states. We also assume that the direction in which the half-plane is 'white' or 'black' is not important as it does not influence the sensory invariant sets. Hence, there is a duality between the set of straight lines in the environment and the sets of its podal points.

The three sets of outcomes

The first two environments contain all the possible straight lines intersecting with the disk D. So, the sets of environmental states E 1 , E 2 are equivalent to the set of podal points in the disk D of radius 1 without the center point: The third environment only contains vertical straight lines intersecting with D. So, the set E 3 is equivalent to the set of podal points on the horizontal line.

E 1 = E 2 = {(x, y) ∈ R 2 ; 0 < x 2 + y 2 ≤ 1} = {(x, y) ∈ D \ (0, 0)}. ( 3
E 3 = {x ∈ R; -1 < x < 1}. (3.2)
The three straight lines environments are shown in Figure 3.2(top). We have now defined the set of outcomes of the random variable E for the three environments. These three environments have different straight lines distributions which are captured by three different probability measures.

Probability measures

The probability measures P 1 , P 2 and P 3 of the three environments describes the distribution of the straight lines on the plane.

For the first two environments, the straight lines have been chosen to cover the disk D entirely. However, there is an infinite number of possible distributions of straight lines. The two chosen distributions for environments 1 and 2 are easy to simulate. They have been first described in Bertrand, 1889 and some of their statistical properties have been studied in [START_REF] Marinoff | A Resolution of Bertrand's Paradox[END_REF].

The first method obtains a heterogeneous covering of the disk by the straight lines while the second one is homogeneous. The third distribution is introduced to show that the full pose space can't always be represented in the internal representation obtained after full refinement. It is built such that the straight lines are all vertical and uniformly distributed along the horizontal axis. In this last case, vertically aligned sensor poses can't be distinguished. All these distributions are shown in Figure 3.2.

Environment 1: method 1

The first distribution is obtained by simply choosing the straight lines' podal points with a uniform distribution on the disk of radius 1 as shown in Sub-Figure 3.2(a)(bottom). The obtained covering for 1000 environmental states is shown in SubFigure 3.2(a)(top). It is clear that the density of the straight lines increases with the radius so that pairs of sensor's poses far from the center are more likely to be separated by an environmental state.

Let's describe the probability space of environment 1. We can characterize the distribution of the straight lines thanks to its duality with the set of podal points. Let's call E the random variable representing the outcome of the random experiment of choosing an environmental state, and Q the random variable representing the podal point of E . Following method 1, the podal points are uniformly distributed on the disk D as shown in SubFigure 3.2(a)(bottom), the probability to obtain a podal point in a certain a region A of points inside the disk is simply the ratio between the area of A over the total area of the disk.

P(Q ∈ A) = Area(A) Area(D) . (3.3)
3.1. A probability theory framework for the interaction between agent and environment 53

So, the probability for the value of random variable E to be inside a subset E ∈ P (E 1 ) is equal to the probability measure P 1 of E such that

P 1 (E) = P(Q ∈ A) = Area(A) Area(D) , (3.4)
where A is the set of podal points corresponding to the straight lines in E. Hence the first environment is characterized by the probability space (E 1 , P (E 1 ), P 1 ).

Environment 2: method 2

The second distribution is obtained by uniformly choosing the polar angle of the podal point in ]0, 2π] and uniformly in ]0, 1] the distance of the podal point from the center. The distribution of podal points is shown in Sub-Figure 3.2(b)(bottom) and the covering of D by the straight lines in SubFigure 3.2(b)(top). In this second case, the computation of the probability measure is less obvious as the set of podal points is not homogeneous, but the computations are still straightforward in polar coordinate. Let's call (R, Θ) the random variables corresponding to the polar coordinates of the podal random variable Q, R correspond to the radius and Θ to the orientation. In this method R and Θ are chosen independently so the

joint distribution f R,Θ is f R,Θ (r, θ) = f R (r) f Θ (θ), (3.5)
where f R and f Θ are the marginal distributions. The variables R and Θ having uniform distributions we have that

f R (r) = 1 when r ∈ [0, 1] 0 otherwise (3.6) f Θ (θ) = 1 2π when θ ∈ [0, 2π] 0 otherwise (3.7)
Then the probability to obtain a podal in a certain set A of points inside the disk is the integration over A of the joint density:

P(Q ∈ A) = 1 2π (r,θ)∈A drdθ, (3.8)
where dr and dθ are Lebesgue measures. Hence, one can compute the probability measure P 2 for any subsets of environmental states E ∈ P (E 2 ) as

P 2 (E) = 1 2π (r,θ)∈A drdθ, (3.9)
where A is the set of podal points corresponding to the straight lines in E and (r, θ) are the polar coordinates of the podal points. Note that this value corresponds to the area of A in the polar coordinates plane normalized by 2π.

As an example, if A is a disk centered at the origin with radius 0 < a < 1, then the probability to sample a straight line of podal point inside A is:

P(Q ∈ A) = 1 2π (r,θ)∈A drdθ (3.10) = 1 2π 0<r<a dr 0<θ<2π dθ (3.11) = a. (3.12)
Using method 2 to generate the environmental states, the environment probability space is (E 2 , P (E 2 ), P 2 ).

Environment 3: Method 3

In the third environment, the straight lines are also chosen to cover homogeneously the disk D. This happens by simply sampling the podal points on the horizontal diameter of D following a uniform distribution on [-1, 1]. The resulting covering is shown in SubFigure 3.2(c)(top) and the uniformly sampled podal points are shown in SubFigure 3.2(c)(bottom). Remember that the only environmental states possible are the ones in E 3 , e.g. the set of podal points in the interval [-1, 1]. Hence the probability to sample a podal point from any set A of points inside in the interval [-1, 1] is given by the ratio of the total length of A over the length of the diameter:

P(Q ∈ A) = Length(A) 2 .
(3.13)

Observability of the fundamental sets

55 Thus, the probability measure P 3 for any subset E ∈ P (E 3 ) is given by

P 3 (E) = Length(A) 2 , (3.14)
where A is the set of podal points corresponding to the vertical straight lines of E.

Observability of the fundamental sets

We have characterized the statistic properties of agent/environment interaction with a probability space (E , P (E ), P). Thanks to the introduction of the probability measure P, we shall be able to generalize the deterministic notion of fundamental configuration invariant sets from a point of view of the probability theory. Indeed, instead of having fundamental sensory equivalence between two motor configurations defined as the sensory invariance obtained for all environmental states in E which requires the interaction with all the environmental states, we shall extend the definition to: two motor configurations are sensory equivalent if there is a probability equal to 0 of observing an environmental state that distinguishes them. This should allow to obtain equivalence classes that are probably not-refinable which are accessible with only a countable number of interactions.

Observable equivalence between motor configurations

Let's define, for two motor configurations m and m , their environmental invariant set E (m,m ) which are the subsets of E for which the motor configurations generate the same sensory inputs, as

For all ε ∈ E (m,m ) , m = ε m and (3.15) for all ε ∈ E c (m,m ) , m = ε m . (3.16) Where E c (m,m ) is the complement of E (m,m ) in E , E c (m,m ) = E \ E (m,m )
. Then two motor configurations are said to be observably equivalent if the probability to observe an environmental state that distinguish the two motor configurations is 0, i.e. the complement of their environmental invariant set is a set of P(E c (m,m ) ) = 0.

(3.17)

The set of equivalence classes [m] P based on this equivalence relation is the quotient M/ P such that

M/ P = {[m] P ; m ∈ M}. (3.18)
The quotient set M/ P is called the observable fundamental motor configuration quotient set or observable fundamental set. This set is automatically coarser than the fundamental invariant set M/ E . Because when m = E m , the environmental invariant set E c (m,m ) = ∅ and by definition P(∅) = 0.

M/ E ≤ M/ P .

(3.19)

Observable equivalence between poses

Similarly, for interpretation purposes, we can define observable equivalence between poses. If two motor configurations lead to the same pose, then naturally, the sensory inputs generated at these two motor configurations are always the same. Hence, the set of environment states that makes two poses

x and x generates different sensory inputs is exactly the same than the set of environment states that makes their corresponding motor configurations generates different sensory inputs. More formally, suppose that m ∈ f -1 (x) and m ∈ f -1 (x ) then we have that

x = P x if and only if m = P m . (3.20)
So that the pose quotient set built from this equivalence relation is called the observable fundamental pose quotient set or observable sensitive pose set X / P such that

X / P = {[x] P ; x ∈ X }. (3.21)
Proposition 5. The sets M/ P and X / P are equinumerous. Hence, the observable fundamental set M/ P can be used as an internal representation of the observable sensitive poses in the observable fundamental pose quotient set M/ P .

Observability of the fundamental sets

57

Illustration of the observable fundamental sets

Let us illustrate the probabilistic internal representation obtained for the three environments defined previously. The sensory equivalence classes are obtained when the probability to obtain an environmental state in E c (m,m ) is equal to 0. Where E c (m,m ) is the set of all environmental states that sensory distinguish both motor configurations.

Parametrization of the environmental invariant sets.

In the straight line environment with the 2 DoF planar agent with a singlepixel camera, two motor configurations m and m generate different sensory inputs when their sensor's position f (m) and f (m ) are separated by a straight line in the environment. The set of such straight lines are represented by their podal points in blue in Figure 3.3. This region is given by the union of the disk of diameter the line segment O f (m) and the disk of Chapter 3. Stochasticity on the refinement process diameter O f (m ) minus their intersection. Let's call A (m,m ) this region of D. Then let's compute the observable fundamental sets for the three introduced environments.

Observable fundamental set of environments 1 FIGURE 3.4: Environment 1

For environment 1, the probability measure P 1 of A (m,m ) has been shown in equation 3.4 to be the ratio of its area in the Cartesian plane over the total area of D. The only way this region has a 0 area is when the intersection of both disks covers their union. In such case we have that P 1 (E c (m,m ) ) = 0. However, this happen only if f (m) = f (m ). Hence, for any motor configuration m, its observable fundamental equivalence class [m] P is directly a point in D. And the observable fundamental set is

M/ P 1 = {(x, y) ∈ D}. (3.22)
Observable fundamental set of environments 2 For environment 3 however, the probability measure P 3 of A (m,m ) is more complex. Indeed, it has been shown in equation 3.14 that P 3 corresponds to the ratio of the length of the intersection of A (m,m ) over the length of the horizontal line. However, when the poses f (m) and f (m ) are vertically aligned then the two disks of separating podal points intersect together exactly on the horizontal line as shown in Figure 3.7. In this case, the intersection of A (m,m ) with the horizontal line results in isolated points and isolated points have length 0. In any other configurations, except when the positions are equal, the intersection has a non zero length. Then the only motor configurations that have a probability measure P 3 of A (m,m ) equal to 0 are

• when f (m) = f (m ),
• when f (m) and f (m ) vertically aligned.

Therefore, in environment 3, the observable fundamental equivalence classes [m] P 3 of a motor configuration m ∈ M is the set of all motor configurations leading to sensor poses in D vertically aligned with f (m). Then the observable fundamental set M/ P 3 is in bijection with the points in the horizontal line segment [(-1, 0), (1, 0)] such that

M/ P 3 = {x ∈ R; -1 < x < 1]}. (3.24)
3.3. The refinement process as a measure-preserving dynamical system 61

The refinement process as a measure-preserving dynamical system

So far, we have provided a probabilistic framework to describe the choice of a single environmental state. However, the refinement process is obtained progressively with a successive interaction of environmental states. Moreover, the succession of the environmental states may be governed by certain laws.

The succession of interactions as a measure preserving transformation

The succession of experienced environmental states is modeled by a transformation T on the set of all environmental states E . The transformation T gives, for each environmental state ε, its next environmental state T(ε) with which the agent will interact. Hence, if the initial environmental state is denoted ε 0 then the next experienced environmental state would be denoted as

ε 1 = T(ε 0 ), more generally ε n+1 = T(ε n ).
The successive experiences are then indexed by the integer n which indicates a countable number of experiences. The transformation T depends both on the natural variations in the environment and the moment that the agent chooses to interact.

In order to be coherent with the probabilistic framework introduced earlier, we will require that the probabilistic properties of the environment does not change through the successive experiments, such that the transformation T is a measure-preserving transformation of E . T being measure-preserving means that for any subset E ∈ P (E ) we have that P(T -1 (E)) = P(E). With these properties, the interaction of the agent with the environment is called a dynamical system and is represented by the tuple (E , P (E ), P, T).

Then, let E be the random variable that determines the initial environmental state of the agent with a probability distribution given by the probability measure P. Then the process of sampling successive environmental states from the dynamical system is governed by the transformation T, it can be Chapter 3. Stochasticity on the refinement process written:

{E , T(E ), T • T(E ), • • • , n-1 T • • • • • T(E )} = {E , T(E ), T 2 (E ), • • • , T n-1 (E )}
. Then the refinement process can be represented by the map:

M/ {E ,••• ,T n-1 (E )} : E × N → L. (ε 0 , n) → M/ {ε 0 ,••• ,T n-1 (ε 0 )} (3.25)
Where L represents the set of motor configuration quotient sets defined in the previous section

We can notice that, when the initial environmental states is chosen, the resulting interaction is deterministically given by the map T. The idea is to characterize the properties of the refinement through the possible agent "lives". Indeed, an agent's life, i.e. all its possible interactions with the environment, is determined by a T and an initial environmental state 0 . Then, under correct assumptions on T, it is possible to obtain that, whatever the agent life is, the resulting internal representations are consistent with each other when the number of experiences tends to infinity.

Hypotheses for a convergence of the refinement process

In this section, we shall study what happens when the number of interactions tends to infinity. We shall presents three different hypotheses that guarantee the acquisition of a correct internal representation, for any life the agent have a probability to live.

The sensory comparison as a random variable

During the refinement process, the agent is assumed to be able to make statements about the physical world through the interpretation of its sensory measurements. However, the refinement has now been formalized as a stochastic process such that these comparisons can be defined as random variables. For every pair of motor configurations m and m ∈ M, the comparison variable 3.3. The refinement process as a measure-preserving dynamical system 63 is a function of the outcome of the random variable E such as

∆ (m,m ) (ε) = 0 if m = ε m ; 1 otherwise. (3.26)
The map ∆ (m,m ) (E ) is a random variable defined on the set E and takes its values in the discrete set {0, 1}. Moreover, from the definition of environmental invariant sets in (3.15) we have that

∆ -1 (m,m ) (0) = E (m,m ) and ∆ -1 (m,m ) (1) = E c (m,m ) .
(3.27)

It is then easy to compute the discrete probability density function, also called the probability mass function because the random variable is discrete, of ∆ (m,m ) (E ) as

P(∆ (m,m ) (E ) = 0) = P(E ∈ ∆ -1 (m,m ) (0)) = P(E (m,m ) ) and P(∆ (m,m ) (E ) = 1) = P(E ∈ ∆ -1 (m,m ) (1)) = P(E c (m,m ) ).
(3.28)

Where P(A) denotes the probability for the event A to be a 'success'. Moreover, one can verify that the probabilities sum up to 1:

P(∆ (m,m ) (E ) = 1) + P(∆ (m,m ) (E ) = 0) = P(E c (m,m ) ) + P(E (m,m ) ) = P(E c (m,m ) ∪ E (m,m ) ) = P(E ) = 1. (3.29)
The refinement process can thus be characterized by the sequence of successive random variables {∆ (m,m ) (T n (E ))} for all pairs (m, m ) of motor configurations.

Proposition 6 (Identically distributed). The random variables in the sequence {∆ (m,m ) (T i (E ))} for i = 1 : n, are identically distributed.

This means that for any pair ∆ (m,m ) (T j (E )), ∆ (m,m ) (T k (E )) in the sequence we have

P(∆ (m,m ) (T j (E )) = ω) = P(∆ (m,m ) (T k (E )) = ω), for every ω ∈ {0, 1}.
Proof. The proof comes from the fact that T is a measure-preserving transformation. Hence, for any j, k ∈ N we have that

P(T -k (E)) = P(T -j (E)) = P(E)
, for all measurable sets E.

(3.30) Moreover, by definition,

P(T k (E ) ∈ E) = P(E ∈ T -k (E)) = P(T -k (E)). (3.31)
And thus

P(T k (E ) ∈ E) = P(T j (E ) ∈ E). (3.32)
So the random variables in the sequence {T n (E )} are identically distributed. Now taking E (m,m ) and E c (m,m ) as the measurable sets, we have that

P(E ∈ E (m,m ) ) = P(T k (E ) ∈ E (m,m ) ) = P(T j (E ) ∈ E (m,m ) )
and

P(E ∈ E c (m,m ) ) = P(T k (E ) ∈ E c (m,m ) ) = P(T j (E ) ∈ E c (m,m ) ).
(3.33)

Now, remembering relation (3.28) it transforms into

P(∆ (m,m ) (T j (E )) = ω) = P(∆ (m,m ) (T k (E )) = ω), for every ω ∈ {0, 1}.
Thanks to the measure-preserving property of the transformation T representing the successive interactions of the agent with its environments, we have shown that the distribution of the sensory comparison random variables is not affected by the successive explorations.

3.

3. The refinement process as a measure-preserving dynamical system 65

The Hypothesis of independence of the sensory comparisons

For the first hypothesis, we will assume that the outcome of the sensory comparisons for each experienced environmental state is independent of the previous outcomes. This hypothesis is inspired from the fact that the environmental states may vary sufficiently rapidly so that, when the agent interacts with a new environmental state, it has already changed 'sufficiently' and the result of a new sensory comparison are independent to the previous outcomes. Let's formally define independence of a sequence of random variables.

Definition 3 (Independence of a sequence of random variables). A sequence of random variables {X n } is said to be independent if

P( k j=1 {X i j ∈ A j }) = k ∏ j=1 P({X i j ∈ A j }).
for any sub-collection of k random variables X i 1 , ..., X i k , where k ≤ n, and for any collection of events

{X i 1 ∈ A 1 }, • • • , {X i k ∈ A k } where the A k are measurable sets.
Then, we can state the independence hypothesis.

Hypothesis 1 (Independence). For all pairs m, m ∈ M of motor configurations, the sequence of random variables {∆ (m,m ) (T n (E ))} is independent.

If this hypothesis is verified, we can obtain a first convergence result.

Proposition 7. If hypothesis (H1) is satisfied then, for any pair of motor configurations m, m ∈ M, the stochastic refinement process converges to the observable fundamental set M/ P . More formally,

lim n→∞ M/ {E ,••• ,T n-1 (E )} = M/ P , almost surely. (3.34)
Proof. Let's consider two motor configurations m and m in M. We need to prove now that the equivalence classes of the refinement for these two motor configurations converge almost surely towards the equivalence classes built from the probability measure P. The "almost surely " part means that the convergence should happen for almost all the sampled environmental states but may not happen for a negligible set of environmental states, e.g. a set of which P-measure is 0.

First, it is obvious that for any

ε ∈ E if ∑ n-1 k=0 ∆ (m,m ) (T k (ε)) = 0 then it means that for any integer k with 0 ≤ k < n we have ∆ (m,m ) (T k (ε)) = 0 and more particularly, m = {ε,••• ,T n-1 (ε)} m .
Thus, the probability for m and m to be in the same equivalence class after n refinement steps is

P(m = {E ,••• ,T n-1 E )} m ) = P( n-1 ∑ k=0 ∆ (m,m ) (T k (E )) = 0). (3.35)
We know from equation (3.28) that for any k ∈ N,

P(∆ (m,m ) (T k (E )) = 0) = P(T k (E ) ∈ E (m,m ) ) = P(E(m, m )). (3.36)
Thanks to Hypothesis (H1) we have that {∆ (m,m ) (T k (E ))} is an independent sequence of random variables and

P( n-1 ∑ k=0 ∆ (m,m ) (T k (E )) = 0) = n-1 ∏ k=0 P(∆ (m,m ) (T k (E )) = 0) = n-1 ∏ k=0 P(E (m,m ) ) = P(E (m,m ) ) n .
(3.37)

Let's call A n ∈ P (E ) for n ∈ N the subsets of E corresponding to the success of the statement:

"m = {E ,••• ,T n-1 (E )} m " or equivalently the success of "∑ n-1 k=0 ∆ (m,m ) (T k (E )) = 0".
Because they are in the event-space P (E ) these subsets are called "events". It is easy to see that the events in the sequence are nested, e.g.

A 0 ⊇ A 1 ⊇ • • • ⊇ A n-1 ⊇ A n ,
as the successive events are more and more restrictive requiring sensory invariance for more and more refinement steps.

Lemma 7.1 (continuity of probability measures). Let {A n } n∈N be a sequence of nested events, then

P( lim n→∞ A n ) = lim n→∞ P(A n ).
(3.38)

3.3. The refinement process as a measure-preserving dynamical system 67 Hence,

P( lim n→∞ m = {E ,••• ,T n-1 (E )} m ) = P( lim n→∞ n-1 ∑ k=0 ∆ (m,m ) (T k (E )) = 0) from lemma 7.1 = lim n→∞ P( n-1 ∑ k=0 ∆ (m,m ) (T k (E )) = 0) from equation (3.37) = lim n→∞ P(E (m,m ) ) n .
(3.39)

Note that an event is said to occur almost surely when its probability to happen is 1. Here,

P( lim n→∞ m = {E ,••• ,T n-1 (E )} m ) = 1 ⇔ P(E (m,m ) ) = 1. (3.40) Which means that lim n→∞ m = {E ,••• ,T n-1 (E )} m ⇔ m = P m , almost surely. (3.41)
This being valid for all pairs of motor configurations the result follows.

Hence, under hypothesis (H1) the stochastic refinement converges almost surely towards the observable fundamental set M/ P . An other important results on the convergence can be obtained under a "weaker" hypothesis where we allow the random variables in the sequence {∆ (m,m ) (T i (E ))} to be dependent of each other.

The hypothesis of ergodicity in the stochastic refinement process

The second weaker hypothesis is also based on a property of the transformation T governing the successive experiences of the agent. The main principle is to characterize in T the fact that, if the agent experiences a 'sufficiently long' sequence of environmental states, the statistics obtained from the sequence of sensory invariants will be the same than the statistics over the whole set E . The property that elegantly guarantees such results is ergodicity. Contrarily to the previous hypothesis, ergodicity allows for successive random variables to be dependent of one another, hence it is more realistic and nevertheless sufficient to obtain good convergence properties. Then, let's define ergodicity in the measure theoretic framework.

Definition 4 (Ergodicity of a transformation T). Let (Ω, Σ, µ) be a probability space and T : Ω → Ω a measure preserving transformation. Then T is said to be ergodic with respect to µ if the following condition holds:

for every A ∈ Σ such that T -1 (A) = A either µ(A) = 0 or µ(A) = 1. (3.42)
Now, we can state the ergodicity hypothesis.

Hypothesis 2 (Ergodicity). The transformation T is ergodic.

While the definition is quite technical, the ergodicity of the transformation T brings nice computational shortcuts as it enables the agent to deduce the statistical characteristics of the refinement process from a "sufficiently long" sample. More formally, we have the following proposition.

Proposition 8. If hypothesis (H2) is satisfied then, for any pairs m, m of motor configurations in M, we have that, for almost every initial environmental states, the sample mean of all the comparisons converges towards the probability of both motor configuration to generates different sensory inputs. More formally:

lim n→∞ 1 n n-1 ∑ k=0 ∆ (m,m ) (T k (E )) = P(E c (m,m ) ), almost surely. (3.43)
The almost sure convergence means that, if the relation (3.43) does not work, then it is for a negligible set (of P-measure 0) of initial environmental states. These environmental states having a probability zero of being the outcome of the experiment of choosing an environmental state.

Proof. In order to prove the proposition, we need to introduce a well-known theorem about ergodicity.

Lemma 8.1 (Birkhoff's Pointwise Ergodic Theorem). Lets (Ω, Σ, µ, T) be a dynamic process where Ω is the space of outcomes, Σ the event space, µ a probability measure on Σ and T a measure-preserving, ergodic transformation with respect to the measure µ. Lets ν : Ω → R be a real-valued µ-integrable 3.3. The refinement process as a measure-preserving dynamical system 69 function. Then for almost every ω ∈ Ω, we have that

lim n→∞ 1 n n-1 ∑ k=0 ν(T k (ω)) = Ω νdµ. (3.44)
Proof. The proof is well known and can be found in [START_REF] Birkhoff | Proof of the ergodic theorem[END_REF] Let's show that the map

∆ (m,m ) (E ) is P-integrable. In fact, because ∆ (m,m ) (E )
takes the discrete values {0, 1}, we have that

E ∆ (m,m ) (E )dP = E ∆ (m,m ) (ε)dP(ε) = E c (m,m ) 1 • dP(ε) = P(E c (m,m ) ) < ∞. (3.45) Hence ∆ (m,m ) (E ) is P-integrable.
In fact the result of this integral is called the expected value of the random variable.

E[∆ (m,m ) (E )] = P(E c (m,m ) ). (3.46)
Therefore, applying lemma 8.1 for the map ∆ (m,m ) (E ), for any motor configurations m and m and for almost every ε ∈ E , we have that

lim n→∞ 1 n n-1 ∑ k=0 ∆ (m,m ) (T k (ε)) = P(E c (m,m ) ). (3.47)
Finally, the considered space being a probability space, there is an equivalence between the statements "almost surely" and "almost everywhere" since the set of ε for which the last equation is not satisfied has a P-measure 0 which corresponds to a zero probability to occur.

Therefore, the expected value of the comparator operator ∆ is representative of the probability for two motor configurations to be equivalent. This is an important result that will be exploited later to build nice internal representations.

In order to show that hypothesis (H1) is more restrictive on the refinement process than hypothesis (H2) we have the following corollary.

Corollary 8.1. If hypothesis (H2) is satisfied then we have that

lim n→∞ M/ {E ,••• ,T n-1 (E )} ≤ M/ P , almost surely. (3.48) Proof. Let's show that, if the limit event lim n→∞ m = {ε,••• ,T n-1 (ε)} m is true for almost every ε ∈ E , then necessarily m = P m .
For any ε ∈ E and any pair of motor configurations m, m ∈ M and for all n ∈ N, if the limit event lim

n→∞ m = {ε,••• ,T n-1 (ε)} m is true then necessarily ∆ (m,m ) (T n (ε)) = 0 for all n ∈ N. Hence, it implies that lim n→∞ 1 n n-1 ∑ k=0 ∆ (m,m ) (T k (ε)) = 0.
Moreover, according to Proposition 8, for almost every ε ∈ E , we have that

lim n→∞ 1 n n-1 ∑ k=0 ∆ (m,m ) (T k (ε)) = P(E c (m,m ) ).
Thus, for almost every ε ∈ E and for any m ∈ M, if m is in the equivalence class of m at the limit when n → ∞, then we have that P(E c (m,m ) ) = 0 and so m also belongs to the equivalence class of m in M/ P which is sufficient to prove that each equivalence class in of lim n→∞ M/ {E ,••• ,T n-1 (E )} is inside an equivalence class of M/ P . However, the Hypothesis (H2) does not imply convergence of the refinement towards the fundamental set M/ E as shown in Corollary 8.1. Indeed, the observable sensory equivalence = P can tolerate rare occurrences of sensory inequalities (the sequence of such events must behave in o(n)) so that full sensory equivalence over all elements E is not satisfied. However, thanks to Property 8, the agent can exploit the sample mean of the comparison operator ∆ as it converges almost surely towards the value of the probability P. Indeed, by computing the 0 of the average value 1 n ∑ n-1 k=0 ∆ and use it as 3.3. The refinement process as a measure-preserving dynamical system 71 an estimator of the values of P equal to 0. Then, the agent obtains equivalence classes that asymptotically tends towards the observable fundamental set M/ P with probability 1. Proposition 9. If either Hypothesis (H1) or (H2) is satisfied then, for any pair m, m of motor configurations in M, the sample mean of the sensory comparisons ∆ (m,m ) (T n (E )) converges almost surely towards its expected value, such that

1 n n-1 ∑ k=0 ∆ (m,m ) (T k (E )) ---→ n→∞ P(E c (m,m ) ), (3.49)
almost surely.

Proof.

• Under (H1), the proof is just the application of the Law of Large Numbers for the Bernoulli process composed of the i.i.d random variables {∆ (m,m ) (E )}.

• Under (H2), the proof is given by Proposition 8.

Both previous hypotheses give an insight on the theoretical conditions required for the agent/environment interaction, viewed as a dynamical process, in order to obtain nice convergence properties. The first Hypothesis (H1) assumed independence between the sensory comparisons, if this hypothesis is satisfied then the equivalence classes of the refinement process converge almost surely towards the equivalence classes in the observable fundamental set M/ P . However, independence between measurements may not be verified in a real environment as the succession environmental state may vary smoothly, so that the sensory comparisons may be dependent of the previous observations. The second Hypothesis (H2) assumes that the transformation T is ergodic. This constrains the statistical properties of the agent/environment interaction so that, for almost all initial environmental states, the sample mean of the sensory comparisons converges towards their expected values. Hence, the agent can estimate the observable equivalence classes for which the expected value of the sensory comparisons is 0 and build an asymptotically correct internal representation.

Chapter 3. Stochasticity on the refinement process

The refinement process can also be totally defined from the agent's internal point of view without any reference to the transformation T of the probability measure P.

An internal point of view of the random refinement process

From its internal point of view, the refinement process is defined by actively chosen moments of interaction with the environment. Each interaction is indexed an integer n. Then, at each interaction n, the agent observes the value of its sensory comparison viewed as binary random variables ∆ (m,m ) [n] for all pairs of motor configurations. Then, if the discrete random process satisfy the following hypothesis, we shall see that it obtains a correct asymptotic internal representation.

Hypothesis 3. The discrete time random process ∆ (m,m ) [n] is wide-sense stationary and ergodic in mean.

The terms are defined as follows.

Definition 5. Wide-Sense Stationarity (WSS). A random process X[n] is said to be wide-sense stationary (WSS) if its mean and autocorrelation functions are invariant in time, i.e.,

• The first moment is finite and does not depend on time.

E[X[n]] = E[X] = m, where m < ∞ is a constant. • R X (n, n + k) is a function of k. R X being the auto-covariance of X. • The second moment is finite. E[X[n] 2 ] < ∞. Definition 6. Ergodic in mean. A WSS random process X[n] is said to be ergodic in mean if its sample mean converges in squared-mean to its expected value E[X]. 1 n n-1 ∑ k=0 X[k] ---→ n→∞ E[X] (3.50) in squared-mean. Convergence in squared-mean means that lim n→∞   E   1 n n-1 ∑ k=0 X[k] -E[X] 2     = 0. (3.51) 3.
3. The refinement process as a measure-preserving dynamical system 73 Hence if Hypothesis (H3) is satisfied, we have that for any pair of motor configurations (m, m ), the sample mean of ∆ (m,m ) converges in squared mean towards its expected value, more formally

1 n n-1 ∑ k=0 ∆ (m,m ) [k] ---→ n→∞ E[∆ (m,m ) ], (3.52)
in squared-mean. The value E[∆ (m,m ) ] corresponds to the probability to experience an environmental state which sensory distinguishes m and m and by definition,

E[∆ (m,m ) ] = P(E c (m,m ) ). (3.53)
So, the agent obtains an estimation of the probability for two motor configurations to be sensory distinguished.

Conclusion on the convergence of the refinement process

Assuming that at least one of the previously introduced Hypotheses holds, the agent can estimate the probability measure P of the environment invariant set E c (m,m ) . The estimator is defined as the sample mean ∆ of the sensory comparisons for all pairs motor configurations m and m :

∆ (m,m ) = 1 n n-1 ∑ k=0 ∆ (m,m ) [k]. (3.54)
Furthermore, the convergence of the sampled mean towards the expected value has been shown to be valid almost surely (for Hypotheses (H1) and (H2) and in squared-mean for Hypothesis (H3), furthermore all these types of convergence imply convergence in probability [START_REF] Vaart | Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics[END_REF], i.e the probability for the estimator to be arbitrarily close to the parameter estimated is 1. Moreover, an estimator that converges in probability is called consistent, it is also unbiased as it converges towards its expected value.

To summarize, under either Hypothesis (H1), (H2) or (H3) then, for any pair of motor configuration m, m ∈ M, we have that

∆ (m,m ) ---→ n→∞ P(E c (m,m ) ), in probability. (3.55)
Therefore, from internal measurements of ∆ (m,m ) , the agent can asymptomatically reach the observable equivalence classes, i.e. when P(E c (m,m ) ) = 0. Thus, for any initial environmental state, the obtained internal representations are consistent and converges towards M/ P the observable fundamental quotient set.

For the case where the hypotheses are not satisfied and the sample mean ∆ does not converge towards its expected value, then the agent can obtain different internal representations that depends on the history of experience environmental states. Thus, the agent's interaction with its environment is variable and inconsistent and no general properties can be extracted.

Conclusion

Before this chapter, the refinement process was introduced as a deterministic and possibility uncountably infinite process. In this chapter, the refinement process has been extended in a probabilistic framework that allow for more realistic considerations. So, by assuming the existence of a probability measure P on the set of environmental states, we have replaced the theoretical fundamental equivalence classes (= E ) with observable fundamental equivalence classes (= P ) which can be obtained by computing statistics in the interaction. These new "atoms" of perception represent the motor configurations that have a probability equal to 0 to be distinguished by a sensory input. The set of all these equivalence classes M/ P , is called the observable fundamental set and has been shown to be in direct bijection with the observable fundamental pose quotient set X / P representing the "sensitive" poses that have a probability equal to 0 to be distinguished. Hence, in this framework, it is not necessary for the agent to interact with all the possible environmental in order to obtain a correct internal representation. Furthermore, we have also introduced some dynamics in the refinement process that was lacking in the previous formalization. From these dynamics, the agent obtains sequences of sensory inputs that can then be compared and used to build a progressive refinement of the motor configurations set. In the probabilistic framework, the sensory comparisons inherit some probability distributions from the laws of succession of the experienced environmental states. Under some constraints, presented in 3 different hypotheses, statistical independence of the sensory comparisons, ergodicity of the dynamical system or ergodicity of the sequence of sensory comparisons, it has been shown that, by computing statistic on its sensory comparisons, the agent can asymptotically approach the observable fundamental set M/ P . Under these hypotheses, the sample average of the sensory comparisons ∆ is a consistent and unbiased estimator of the probability for two motor configuration to be sensory distinguished. The observable equivalence classes being reached when this probability is equal to 0.

However, the internal representation still only represents isolated points in sets. Nonetheless, the sample mean of the sensory comparisons provides more information about the pairs of motor configuration than just their sensory equivalence. Indeed, it has be demonstrated that it converges towards the probability for the pair of motor configuration to be sensory distinct. Hence, based on these considerations, one can construct a notion of 'closeness' in the set of motor configurations. As an example, two motor configurations that have a probability 0.5 of being distinguished must be farther away than two motor configurations with a probability 0.01. These measurement between points have the structural property of a distance in the internal representation that shall be exploited in the next chapters.

Chapter 4

From set to space: Emergence of structures from the refinement process So far, the agent has been able to discover the "points" of its sensorimotor interaction. These points correspond to the motor configurations that generates sensory inputs that have a probability equal to 0 of being distinguished, which form the so-called observable fundamental set M/ P . However, these points are not related together, their existence are independent from each other. If a trajectory is defined on this set of points, it would have no other interpretation than a discrete sequence of points inside a set. In order to obtain a higher level of interpretation, it is required to add a structure to the set of points. Indeed, the concept of structure of a set in mathematics is of major importance as its gives a meaning to the points in this set which are obtained with the introduction of additional mathematical objects, such as an other set, relations, or operations in the set. Some standard structures are algebra, topology, measure, metric or order. As an example, an algebra in the set of real numbers allows the computation of operations such as multiplication or addition. Topology allows the definition of open sets and thus continuous transformations, etc. Hence, the emergence of a structure in the internal representation is crucial for its exploitation. Chapter 4. From set to space: Emergence of structures from the refinement process

Emergence of structure from statistics on sensory invariants

In the previous chapter we have introduced a statistic on the sensory comparisons obtained by the agent after a sequence of interactions with the environment. This statistic represents the frequency for two motor configurations to generate different sensory inputs and the frequency can be intuitively linked to a notion of distance between motor configurations.

The sensory dissimilarity

The average value of the sensory differences, expressed for a pair of motor configurations after a finite number of interactions n, is called the sensory dissimilarity σ n . Let's call s[n -1], respectively s [n -1] (the indexation starts at 0), the sensory inputs observed for the motor configuration m, respectively m at interaction n. Then, the sensory dissimilarity is computed as

σ n (m, m ) = 1 n n-1 ∑ k=0 δ(s[k], s [k]), (4.1) = 1 n n-1 ∑ k=0 1 s[k] =s [k] , (4.2)
where δ is the sensory comparison operator available to the agent which correspond here to

1 s[k] =s [k] , which is an indicator function that equals 1 when condition (s[k] = s [k]
) is satisfied and 0 otherwise. When σ n (m, m ) = 0, it means that the sensory inputs s[k] and s [k] have always been equal for all k < n. On the opposite, if the dissimilarity is equal to 1, then all sensory inputs have been different. The closer σ is to 0, the more similar are the sequence of sensory inputs generated by both motor configurations, hence the name 'dissimilarity'.

Then, a structure emerges from the values of σ n for pairs of points in the motor set M. This structure is built internally from the agent's interactions with the environment, hence the obtained structure will be denoted as the empirical structure.

Properties of the empirical structure

In mathematics, a set with an added structure is called a space and is usually written as the tuple (set, structure). Therefore we will refer to a structured set as a space. When there is no ambiguity, the structure will not always be specified. Hence the current motor space is the tuple (M, σ n ).

Some properties of this space are easily obtained from the rules of construction of the empirical structure.

Proposition 10. The map σ n is a pseudometric on M.

Proof. First, let's remember what a pseudometric is.

Definition 7. (pseudometric) A map d : X × X → R ≥0 is said to be a pseudometric if it satisfies the following conditions: for all x, y, z ∈ X:

(a) d(x, y) ≥ 0 (non-negativity). (b) d(x, y) = d(y, x) (symmetry). (c) d(x, z) ≤ d(x, y) + d(y, z) (triangle inequality).
Now, let's prove that σ n is a pseudometric.

Non-negativity:

For a finite number n of repetitions, the sensory dissimilarity σ n , computed with (4.1), can only take the n

+ 1 followings values [0, 1 n , 2 n , • • • , n-1 n , 1].
Therefore, the values of dissimilarity are always nonnegative.

Symmetry:

The sensory comparison computed with (4.1) is obviously symmetric as {s

[k] = s [k]} ⇔ {s [k] = s[k]}, so is the operator 1 s[k] =s [k] ⇔ 1 s [k] =s[k] thus σ n (m, m ) = σ n (m , m).
Triangle inequality: Let m, m and m be three motor configurations. For any interaction indexed by k, let s[k], s [k] and s [k] be the respective generated sensory inputs. Then, let's write the table of all comparisons. Chapter 4. From set to space: Emergence of structures from the refinement process

s[k] = s [k] s[k] = s [k] s [k] = s [k] Valid 1 s[k] =s [k] 1 s[k] =s [k] + 1 s [k] =s [k]
[k] = s [k] must be False too.
In fact, all the combinations with two False and one True cannot happen. Therefore, checking all possible comparison cases and computing the comparisons in columns 5 and 6, we have that

1 s[k] =s [k] ≤ 1 s[k] =s [k] + 1 s [k] =s [k] . (4.3)
This being valid for all finite k, it follows that

n-1 ∑ k=0 1 s[k] =s [k] ≤ n-1 ∑ k=0 1 s[k] =s [k] + 1 s [k] =s [k] . (4.4)
Finally, by multiplying both sides by 1 n , we have that

σ n (m, m ) ≤ σ n (m, m ) + σ n (m , m ). (4.5)
The sensory dissimilarity σ n being a pseudometric on M, the space (M, σ n ) is called a pseudometric space. The sensory dissimilarity obtained from the process of repeated explorations has now the meaning of a distance between two motor configurations.

As a dissimilarity, when σ n (m, m ) = 0, then both motor configurations m and m have generated identical sensory inputs at each repetition, they can be considered sensory equivalent and thus regrouped in an equivalence class

[m] σ n , [m] σ n = {r ∈ M; σ n (m, r) = 0}. (4.6)
The equivalence classes are thus the kernels or null-space of the sensory dissimilarity σ n (m, •), for all motor configurations m ∈ M. The set of all such equivalence classes forms the current motor quotient set and is noted M/ σ n . It is possible to extend the sensory dissimilarity σ n , originally defined between motor configurations, to a sensory dissimilarity σ * n between equivalence classes, such that This space also carries interesting properties.

σ * n ([m] σ n , [m ] σ n ) = σ n (m, m ), ( 4

Proposition 11. The map σ *

n is a metric on M/ σ n .

Proof. First let's remember the definition of a metric.

Definition 8. (metric) A map d : X × X → R ≥0 is said to be a metric if it satisfies the following conditions: for all x, y, z ∈ X: A metric, is a pseudometric that satisfies the identity of indiscernible such that when two points are distinct from each other, they must be at a strictly Chapter 4. From set to space: Emergence of structures from the refinement process positive distance. Indeed the pairs of points in M that are at a sensory dissimilarity 0 has been regrouped together into a single point in the quotient set M/ σ n . The fact of constructing a metric space from the quotient of a pseudometric space is called a metric identification (N. R. [START_REF] Howes | Modern Analysis and Topology[END_REF].

We have now guaranteed that, at each step of the refinement process, via the computation of the sensory dissimilarity, the agent is able to obtain a structured set of points, e.g. the space (M/ σ n , σ * n ) which is a metric space. Therefore, the sensory dissimilarity can be interpreted as a perceptive distance, with metric properties, between the sensory equivalent sets of motor configurations.

However, because the number of explorations is finite, the current motor quotient space (M/ σ n , σ * n ) is a discrete metric space. Indeed, every pair of points are separated from each other by a distance of at least 1 n , where n is the current exploration repetition, so that the points in this space form a discontinuous sequence. Now, one can wonder what happens if the number of repetition tends towards the infinity. Does the structure converges, does it converges towards a continuum ? While in practice, a finite experience must already be exploitable by the agent, and this will be discussed in a later section, it is very interesting to study under which conditions such structure tends to represent continuous properties of the agent's pose space.

Convergence of the empirical structure

So far, the introduced structure is built from finite experience, hence the denomination empirical structure. Furthermore, the empirical structure depends on the history of experienced environmental states and without specific constraints on the environment, it is not guaranteed that the structure is meaningful, i.e. related to physical properties in the environment. A direct way to get rid of this variability in the built structure is to exploit the ergodic property of the environment. Indeed, according to the previously introduced ergodic hypothesis in Chapter 3, when the refinement process has converged, the properties of the sensory dissimilarity are consistent, in the sense that the resulting structure after convergence is unique irrespective of the history of the agent's past experiences. Indeed, it will be shown that the converged structure properly characterizes the agent/environment interaction.

Limit of the sensory dissimilarity

In the previous chapter we have introduced some statistical properties on the refinement process. Indeed, it has been assumed that, given a fully characterized environment, the sensory comparisons, represented by a random variable ∆, have a distribution parametrized by a probability measure P on the subsets of the environmental set E . So, at interaction n, two motor configurations m and m have the probability P(E c (m,m ) ) to generate different sensory inputs, such that

P(∆ (m,m ) [n] = 1) = P(E c (m,m ) ). (4.8)
Where E c (m,m ) is the set of all environmental states that makes m and m generates distinct sensory inputs and P can be interpreted as the probability to observe an environmental state inside it. Then under the Hypothesis of ergodicity in mean (H3), the sample mean ∆ (m,m ) (3.54) has been shown to be a consistent and unbiased estimator of the value P(E c (m,m ) ). Then, the sensory dissimilarity σ n (m, m ) is an estimate of P(E c (m,m ) ) and we have that,

σ n (m, m ) ---→ n→∞ P(E c (m,m ) ),
with probability 1. (4.9)

To clarify the notations, the limit will be written

σ(m, m ) = P(E c (m,m ) ). (4.10)
The asymptotic empirical structure Therefore, the discrete pseudometric space (M, σ n ) asymptotically approaches the space (M, σ). Similarly to the previous considerations, this space has a interesting properties.

Proposition 12. The map σ is a pseudometric on M.

Proof. Let's m, m , m ∈ M be three motor configurations. Chapter 4. From set to space: Emergence of structures from the refinement process Non-negativity. P being a probability measure, it is non-negative, e.g. for any E ∈ P (E ), P(E) ≥ 0. Hence σ(m, m ) = P(E c (m,m ) ) ≥ 0 so σ is nonnegative.

Symmetry.

It is obvious that because the sensory inequality is symmetric,

E c (m,m ) = E c (m ,m) and P(E c (m,m ) ) = P(E c (m ,m) ).
Triangle inequality. First, we have that

σ(m, m ) + σ(m , m ) = P(E c (m,m ) ) + P(E c (m ,m ) ).
P being a probability measure, it is sub-additive so that

P(E c (m,m ) ∪ E c (m ,m ) ) ≤ P(E c (m,m ) ) + P(E c (m ,m ) ). Let us show that E c (m,m ) ⊆ E c (m,m ) ∪ E c (m ,m ) by contradiction. Assume that E c (m,m ) ⊆ E c (m,m ) ∪ E c (m ,m ) , then there exists at least one ε ∈ E c (m,m ) which is also outside of E c (m,m ) ∪ E c (m ,m ) . Then m = ε m by definition of E c (m,m ) but because ε is neither in E c (m,m ) nor in E c (m ,m )
, necessarily m = ε m and m = ε m , so m = ε m which gives a contradiction. Therefore by monotonicity of the probability measure,

P(E c (m,m ) ) ≤ P(E c (m,m ) ∪ E c (m ,m ) ) ≤ P(E c (m,m ) ) + P(E c (m ,m ) ), hence σ(m, m ) ≤ σ(m, m ) + σ(m , m ).
So, it has been proved that the limit space (M, σ) is a pseudometric space. Moreover, we have seen in the previous subsection that it is possible to perform a metric identification by gluing together the points at σ-distance 0. The resulting space is of particular interest as it correspond to the theoretical limit of the agent's constructions under the refinement process.

The empirical structure in the fundamental observable motor configuration space.

Gluing together the pairs of motor configurations that are at σ-distance 0 is equivalent to clustering together pairs of points for which the environment invariant sets are of probability measure 0, these points corresponding to the observable equivalence classes. More formally, σ(m, m ) = 0 is equivalent to P(E c (m,m ) ) = 0. Then, using the definition of the observable equivalence classes in (3.17), we have that

σ(m, m ) = 0 ⇔ [m] P = [m ] P .
(4.11)

Hence, the set M/ P is actually the quotient of M for which the pairs of points with dissimilarity σ equal to zero have been glued together. Then, extending the asymptotic sensory dissimilarity to pairs of points in the observable fundamental set as the map σ * :

M/ P × M/ P → [0, 1] such that σ * ([m] P , [m ] P ) = σ(m, m ) = P(E c (m,m ) ), (4.12)
where m and m are any representative of the respective equivalence classes [m] P and [m ] P .

Then we have the following result.

Proposition 13. The map σ * is a metric on M/ P .

Proof. The proof follows directly from the metric identification.

Thus, the observable fundamental space (M/ P , σ * ), is a metric space.

Conclusion

When interacting with multiple environmental states, the agent captures sensory invariants between its explored motor configurations. These invariants can be used to regroup the motor configurations into sensory equivalence classes that represent the set of "points", i.e. the agent cannot distinguish these configurations from a sensory point of view. Then, by choosing any pair of these "not sensory distinguishable" points, the respective history of sensory inputs can be more or less similar. Hence, by comparing the frequency of the sensory differences, the agent obtains a sensory dissimilarity σ * n between these points which has been shown to be a discrete metric. This metric is subject to evolution as the number of interactions increases. In fact, under the ergodicity hypothesis (H3) on the sensory comparisons, the sensory dissimilarity converges in probability towards the probability value to obtain an environment that makes these points generate different sensory inputs. This allows us to characterize the asymptotic structure obtained after an infinite number of explorations. Indeed, the final structure is shown to be also a metric space.

Therefore, the observable fundamental space (M/ P , σ * ) acts as a theoretical limit of the agent capabilities to distinguish sensor poses in this environment. As such, it would be interesting to see under which conditions this space carries the properties of continuity of the physical space. But before going into the realm of topological considerations, let's demonstrate the properties of structure emergence with the previously proposed illustrative environments.

Illustrations on the empirical structure.

In this section, some simple examples will be used to illustrate the concept of structure emergence as well as the convergence properties.

Emergence of the empirical structure

The principles for the emergence of an empirical structure are studied here using the simple planar agent introduced in 2.1 and the straight lines environment as described in 2.8. The first 3 steps of the agent interaction with the environment are shown in Figure 4.1 as well as the empirical structure emerging from that interaction.

At the beginning, the agent has not interacted with the environment so that all motor configurations are assumed equivalent. Therefore, the internal structure is a single point {M} gathering all motor configurations. Then, the agent performs the exploration of its motor configuration set (which is assumed instantaneous for demonstration purposes) in an environmental state ε 1 chosen randomly in the set of all straight lines intersecting with the agent pose space. All the motor configurations are then regrouped into the two sensory equivalence classes possible, black or white according to the side of the straight line the sensor is in. The sensory dissimilarity σ * 1 is then computed for these two equivalence classes. Because the agent has interacted with only one environment, the mean number of time the sensory inputs have been different is 1 over 1. Hence, after the first exploration, the current motor quotient space (M/ σ 1 , σ * 1 ) is a set of two clusters of points at a distance of 1. This can be represented as a weighted graph as shown in Figure 4.1. Chapter 4. From set to space: Emergence of structures from the refinement process

For the second exploration (n = 2), the environmental state has changed to ε 2 , which is a different straight line. The resulting sensory inputs are combined with the previous sensory inputs to form a vector of sensory inputs for each motor configurations. When the sensory "history" for different motor configurations is equivalent, which means that at both motor configurations the agent has always generated equal sensory inputs, they are regrouped into a sensory equivalence class. In this case, it gives 4 equivalence classes, and so 4 points in the current motor quotient set, corresponding to the different possible successions of white and black sensory inputs. The sensory dissimilarity is then computed to give the distances between all 6 possible pairs of points. The resultant structure is a motor quotient space (M/ σ 2 , σ * 2 ) with 4 points and 6 values of sensory dissimilarity for all pairs of points. This process is then repeated until convergence of the empirical structure. It has been shown in a previous section, that under some hypotheses on the stochastic properties of the agent/environment interaction, and particularly on the statistics of the experienced environmental states, the sensory dissimilarity converges to a constant value which depends on a probability measure P in the environment states and the chosen pairs of motor configurations.

Even if the type of environment is the same (here the straight lines), different environment statistics implies different empirical structures. Moreover, these structures can be characterized by the asymptotic value of the sensory dissimilarity for all pairs of explored motor configurations.

To illustrate these variations, let's characterize the empirical structure for the three environments described in subsection 3.1.2. Chapter 4. From set to space: Emergence of structures from the refinement process probability measure for this environment is given in equation (3.4) such that the probability that m and m generate different sensory inputs is The analytical computation of the intersecting area between two circles is textbook and can be found in Weisstein, n.d. Hence, the asymptotic sensory dissimilarity for environment 1 is:

P 1 (E c (m,m ) ) = Area(A(m, m )) π , ( 4
1 σ(m, m ) = r 2 - 2 π (r 2 cos -1 ( d 1 r ) + d 1 r 2 -d 2 1 )+ r 2 - 2 π (r 2 cos -1 ( d 2 r ) + d 2 r 2 -d 2 2 ), (4.14)
where

d 1 = r 2 -r 2 + (d/2) 2 d , and
d 2 = r 2 -r 2 + (d/2) 2 d , (4.15)
and r and r are half the distance of f (m) and f (m ) from the center and d is the Euclidean distance between them.

Hence the sensory dissimilarity depends on both the Euclidean distance between the sensor positions f (m) and f (m ) and their distance from the center of the disk of reachable positions. Notice that it must be a pseudometric as stated by Proposition 12. Hereafter, it is not a metric since the same sensor position can be reached from multiple motor configuration and condition (b) of Definition 8 required for being a metric is not satisfied, i.e. two motor configurations with sensory dissimilarity 0 can be distinct. Moreover, we have seen in Subsection §3.2.3 that the fundamental set for environment 1 is the set of positions in the disk D:

M/ P 1 ≡ {(x, y) ∈ D}. (4.16)
Therefore, the sensory dissimilarity 1 σ * between the points in the observable fundamental set M/ P 1 is also

1 σ * ([m] P 1 , [m ] P 1 ) = 1 σ(m, m ). (4.17)
This time, it is a metric on the set M/ P 1 because all the redundant motor configurations have been 'glued' together in the equivalence class [ . ] P 1 which represent a single position in space. Therefore the resultant observable fundamental space (M/ P 1 , 1 σ * ) is a metric space.

Spatial interpretation of the observable fundamental space in environment 1

It is possible to write the asymptotic sensory dissimilarity as a function of the Euclidean distance d( f (m), f m )) and r and r the radii of the circles of diameter O f (m) and O f (m ) such that However, it is also obvious that the sensory dissimilarity is not directly equal to the Euclidean distance between the sensors poses, otherwise the asymptotic empirical structure would be directly the Euclidean space. In fact, here, Chapter 4. From set to space: Emergence of structures from the refinement process In order to evaluate the differences between both structures, we can compare neighboring relations in the Euclidean space with the ones in the asymptotic empirical structure. First, let's fix a specific motor configuration m ∈ M.

1 σ * ([m] P 1 , [m ] P 1 ) = 1 σ(m, m ) = h(d( f (m), f (m )), r, r
The corresponding sensor's position in the Euclidean space is given by the forward kinematics f with f (m) ∈ D. Then, the set of -neighbors in the Euclidean space is the set of poses that are within an Euclidean distance > 0 of f (m). This set of neighbors is call an open ball of radius centered at f (m) and will be denoted B d ( f (m)) such that:

B d ( f (m)) = {x ∈ D; d(x, f (m)) < }. (4.21) The neighbor of f (m) in B d ( f (m))
is the set of points in the Euclidean plane that are strictly inside a circle centered at f (m) of radius . Some of these neighborhoods are shown on the right of Figure 4.4 for the radius = 0.24. These neighbors have radial symmetry so that no direction in the plane is preferred and their shape is invariant to any translation of the sensor's pose. These properties are indeed specific to a Euclidean geometry. Now, we can take a look at the open balls in the asymptotic empirical structure. Lets > 0 be the radius and denote by B 1 σ * ( f (m)) the set:

B 1 σ * ( f (m)) = {x ∈ D; 1 σ * (x, f (m)) < }. (4.22)
The notation f (m) has been used instead of [m] P 1 as they are equivalent, see the discussion of equation (4.16) for details. These balls in the empirical structure are different from the Euclidean distance balls as can been seen in Figure 4.4 which illustrates the neighbors of some poses f (m) in the form of the balls B 1 σ * ( f (m)) with the radius = 0.06. This value has been chosen so that the neighborhoods of both spaces corresponds exactly around the pose at the center. Furthermore, one can see that the shape of these neighborhoods are not as regular as the the d-balls in the Euclidean space. Indeed, there is a deformation of the neighborhoods due to the dependency of 1 σ * to the radius r and r of the poses.

Therefore, in the best of cases (asymptotic convergence), the internal space built by the agent is still a distorted version of the Euclidean space. A simple interpretation to this property is that the agent "perceptual distance" between points that are far from the center would be greater than points at the same Euclidean distance but close to the center of the pose space D. This is because they are more often sensory distinguished, which is intuitive when we look at the distribution of the straight lines in Figure 4.2. When the metric is distorted, then some difficulties in the exploitation of the empirical structure can arise. As an example, supposing one can define continuous paths and compute their length in such a space, a straight line between two points may be different than the straight line in the Euclidean space. Hence, a path planning application should bring different results in the empirical structure than in the usual "objective" Euclidean space, due to the intrinsic differences in both metric structures. The amount of distortion being a critical evaluation parameter for the possible exploitations. Chapter 4. From set to space: Emergence of structures from the refinement process

Empirical structure for environment 2

Asymptotic sensory dissimilarity for environment 2 FIGURE 4.5: Environment 2 with method 2. For each exploration, the straight line is chosen independently from probability P 2 so that it uniformly covers the disk D.

The asymptotic sensory dissimilarity in this environment can be written analytically:

2 σ(m, m ) = d ( f (m), f (m )) π , (4.23)
where d denotes the Euclidean distance between poses f (m) and f (m ) in the 2D-plane. The proof of this relation is given in Appendix A. From Proposition 12, 2 σ is a pseudometric in M. Furthermore, the observable fundamental set for environment 2 is also the set of sensor's positions in D: M/ P 2 = {(x, y) ∈ D}, so that any equivalence class [m] P 2 corresponds to the unique pose f (m). And thus the asymptotic sensory dissimilarity is a metric in the observable fundamental set such that

2 σ * ([m] P 2 , [m ] P 2 ) = d ( f (m), f (m )) π . (4.24)

Spatial interpretation of the observable fundamental space in environment 2

This environment is very special, because it has the convenient property that the sensory dissimilarity only depends on the Euclidean distance between the poses f (m) and f (m ). Poses at the same Euclidean distance have the same asymptotic sensory dissimilarity. Additionally, the asymptotic sensory dissimilarity is equal to a scaled Euclidean distance. Therefore, points, lines, angles, rigid objects, rigid transformations can be defined in the same ways in the internal space (M/ P 2 , 2 σ * ) than in a Euclidean space so that the internal space can be directly exploited without any differences with the Euclidean space.

Asymptotic empirical structure for environment 3

Asymptotic dissimilarity of environment 3 FIGURE 4.6: Environment 3 with method 3. For each exploration, a vertical straight line is chosen independently from probability P 3 so that it uniformly covers the horizontal line in D.

In this environment, the sensory dissimilarity 3 σ between two motor configurations corresponds to the horizontal distance between their poses, such that 3 σ(m, m ) = |xx |, (4.25) Chapter 4. From set to space: Emergence of structures from the refinement process where x is the abscissa in the plane of pose f (m), see Figure 2.1. It is clearly a pseudometric as it satisfies the condition of non-negativity, symmetry and triangle inequality but not the identity of indiscernible because two different, vertically aligned poses have a dissimilarity equal to 0. In this case, the observable fundamental set has been shown in subsection 3.2.3 to be in bijection with the horizontal segment line: M/ P 3 ≡ {x ∈ R; -1 ≤ x ≤ 1} such that any equivalence class [m] P 3 corresponds to the point in D of Cartesian coordinates (x, 0), where x is the abscissa of the sensor pose f (m). Hence the asymptotic structure added to the observable fundamental set M/ P 3 is the metric:

3 σ * ([m] P 3 , [m ] P 3 ) = |x -x |. (4.26)

Spatial interpretation of the observable fundamental space in environment 3

The observable fundamental space obtained with environment 3, (M/ P 3 , 3 σ * ) is isometrically equivalent to the line segment ([-1, 1], d) with the standard Euclidean distance on the line. This time, the observable fundamental space does not represent directly the Euclidean pose space of the agent. Indeed, vertically aligned poses are sensory equivalent so that the represented space is the horizontal Cartesian coordinates of the poses. This example shows that the represented space, may not only be a distorted version of the Euclidean pose space. Indeed, the represented space is a quotient space of the pose space obtained after regrouping poses that are in the same observable pose sets, e.g X / P . However, the quotientization of the Euclidean pose space may be very troublesome to visualize and even interpret in general because quotient mapping looses a lot of structural information.

Conclusion

In this chapter, we have introduced the empirical structure as an emergent structure computed from the statistics of sensory invariants in the refinement process. Under the ergodicity Hypothesis (H3), this structure has been

Conclusion

97

shown to converge asymptotically towards a metric on the observable fundamental set, interpreted as the theoretical "limit" of the agent's structuring abilities in the considered environment.

Then, the emergent structures have been studied for three different straight lines environments. For the different straight lines distributions, different metric structures emerge from the successive interactions of the agent with the environment. During a finite interaction, the agent's motor quotient space can be represented as a fully connected weighted graph as shown in Figure 4.1. As the refinement process proceeds, more and more points are distinguished and the weighted graph get more and more vertices and the edge weights, that represents distances, converge towards the probability for the two linked vertices to be distinguished by a sensory input.

Furthermore, the environmental statistics being fully described, it has been possible to analytically study the asymptotic space, the space obtained after convergence. In this space, the set of points form the observable fundamental set M/ P , e.g. the finest probable set of equivalence classes, and it has a metric structure from the asymptotic values of the sensory dissimilarity. When the environment is "nice enough" , e.g. environment 1 and 2, the statistics of the sensory comparisons lead to a "low" distorted representation of the Euclidean pose space. However, the assumed hypotheses does not guarantee the preservation of the properties of Euclidean pose space in the general case, as shown with environment 3.

In this chapter, we have described how the agent can structure its internal representation with a metric computed on the statistics of the sensory comparisons. However, in order for the space to be representative, we need to compare this obtained structure with structure of the actual physical space. Indeed, we would like to represent the space of the sensors' spatial configurations. There are two approaches here, the first is to endow the space of sensors' spatial configurations with an arbitrary metric that is empirically known to be useful for the tasks in space, the other is to derive a structure based on natural assumptions on laws of the physical space and its relation with the probabilities in the environment, and thus the computed statistics. We shall present both approaches in the two following chapters.

Chapter 5

The empirical structure as a representation of physical continuity.

During the refinement process, the agent is called to explore repeatively its motor configurations. By doing so, it captures the corresponding sensory invariants for the successive environmental states, which allows it to refine its motor configuration set into a set of sensory equivalence classes. If the refinement process is allowed to converge, then these equivalence classes represent the theoretically finest points that the agent can distinguish from its sensorimotor experience in the environment. Moreover, from an external point of view, it has been assumed that the sensory invariants follow a probability distribution given a priori on the environment. Hereafter, this probability distribution can be estimated by computing statistics in the sensory invariants, the so-called sensory dissimilarity. Hence, the sensory dissimilarity has been proved to possess metric properties, so that it can be interpreted as a distance between the sensory equivalence classes. In the asymptotic case, the set of finest equivalence classes, i.e. the observable fundamental set M/ P , together with the asymptotic sensory dissimilarity σ * , constitute a metric space which cannot be refined from experience.

A question arises, does this space actually represent anything which has a physical reality ? We have shown, in some convenient illustrative examples, continuity.

that is can be the case for nice environments, and that the sensory dissimilarity metric was linked to spatial properties such as distorted Euclidean distance. However, the statistics invariant have not yet be linked to any physical properties of the world. We will attempt to answer this question with the objective to show that, under some "natural" assumptions, this space actually represents topological properties in a space, constructed from the physical laws of continuity. This is in line with the fact that, perceptive constructions are always distorted versions of the natural properties of the physical space and the statistics of perception does not allow for an metrical representation of the world.

Introduction to the concept of continuity

All these considerations bring a very interesting questions on physical reality in the context of perception, namely: how to define continuity ? Is it an emergent property from the agent/environment interaction or a property imposed by external physical laws found in the environment ?

The question of continuity even nowadays, is very controversial amongst mathematicians and physicists. Does it emerge from construction or does it comes as an axiom ? In physics, the principal idea between continuity comes from the fact that energy cannot be transmitted through void and therefore it must be a continuous quantity over space. In mathematics, the continuity of a map is generally given by the existence of a limit that can be approached infinitesimally. In the current approach, we are constantly juggling between internal and external principle. We can define both points of view by saying that: what is internal, is empirical, and can be constructed, and what is external, is natural. Then, the evaluation of an empirical representation consists in finding the common properties between both views. Here, the mathematical definition of continuity can be interpreted as the empirical part, and the physical definition as the natural part, assuming the physical laws always hold. When the agent undergo a refinement process, which converges at its limit, we shall see that it actually obtains an empirically constructed version of continuity. Then, the agent being a physical object in a physical world, its movement should respect the natural property of physical continuity. Both version shall be compared.

The agent generates commands to its actuators, these commands are powered with a form of energy, generally electrical, that is transduced into mechanical energy. Following the theory of continuity of energy in the physical space, the generated actions must be continuous transformations. This statement will form our hypothesis of physical continuity. We can notice that, this statement can be described purely internally, in term of commands and their effects on motor configurations, even if its root lies inside our common knowledge about physical laws. Hence, we shall compare it to the internal version of continuity based on empirical constructions.

Then, we are led into two important concepts. If the empirical structure preserves the continuity of the actions then it is said to be topologically coherent.

Topological coherence is a necessary condition for the agent to exploit the empirical space as it is necessary for motion planning. Moreover, because here the continuity of actions is rooted in a physical property, the empirical structure can also be evaluated from external point of view. Hence it can be evaluated as a topological representation of some space derived from the sensor pose space with physical continuity.

In this chapter, after a short and intense introduction to topology, the hypothesis of continuity of actions will be developed into what is called the natural topological structures that represents the preservation of physical continuity inside the quotient spaces. Then it will be shown that the asymptotic empirical structure also carries a topological structure derived from the asymptotic sensory dissimilarity. Finally, some assumptions on the statistics of the sensory invariants have to be made for the converged empirical structure to topologically represent the natural structure induced by physical continuity.

A little reminder of topology

First let's begin with a definition of a topological structure. First, a set with a topological structure is called a topological space. A topological space is a set Chapter 5. The empirical structure as a representation of physical continuity.

of points together with the collection of subsets that forms the open sets. It must satisfy the following properties:

Definition 9 (Topological space). A topological space is a set X together with a family τ of subsets of X called a topology such that:

• the empty set and the set itself belongs to τ,

• any union of elements of τ is also in τ,

• any finite intersection of elements of τ is also in τ.

Then we denote by (X, τ) the topological space and the elements of τ are called the open sets.

Notice that the notation for topological spaces is similar to the metric space with the tuple (., .). To avoid any confusion, the topological structure will always be denoted by the Greek letter τ. (5.1)

Definition 11 (Homeomorphism). Two topological spaces (X, τ X ) and (Y, τ Y ) are said to be homeomorphic, or topologically equivalent, if there exists an map between the two spaces such that it is a continuous bijection with a continuous inverse. Such a map is called a homeomorphism.

A common view of Homeomorphism relations is that spaces that are in a homeomorphic together can continuously deformed into each other without being torned punctured or glued. An example of homeomorphic topological spaces is a ball and a cube. An other famous example is a doughnut being homeomorphic to a coffee cup. However, the doughnut can not be continuously deformed into a ball without tearing or gluing because it contains a hole and the sphere does not.

Multiple topologies can be defined on the same set of points. Because topologies are families of subsets of points, they are possibly comparable so that one is finer than the other which is coarser in the sense of inclusion between subsets of a family.

Definition 12 (Finer (resp. coarser) topology). Let's (X, τ 1 ) and (X, τ 2 ) be two topological spaces, the topology τ 1 is said to be finer (resp. coarser) than

τ 2 if τ 2 is contained in τ 1 (resp. if τ 1 contained in τ 2 ) : τ 2 ⊆ τ 1 (resp. τ 1 ⊆ τ 2 ). (5.2)
From the previous definition, we have at our disposal the notion of finest topology. This notion can be used for the construction of new topologies.

Definition 13 (Co-induced topology). Let X and Y be sets and f a function from X to Y. If τ X is a topology on X then the topology τ Y defined as

τ Y = {O ⊆ Y; f -1 (O) ∈ τ X }, (5.3)
is the co-induced topology on Y. The co-induced topology is the finest topology on Y such that f continuous.

The fact of gluing together points in a space with equivalence relations creates a quotient space. If the initial space has a topological structure then the quotient space also carries a topological structure called the quotient topology.

Definition 14 (Quotient topology). Let (X, τ) be a topological space and ∼ an equivalence relation. X/ ∼ is the quotient set containing all the equivalence classes of elements in X. Then, π : X → X/ ∼ is the map that sends a point in X to the equivalence class containing it. Finally, the topology:

τ ∼ = {0 ⊆ X/ ∼ ; π -1 (O) ∈ τ} (5.4)
is the quotient topology of the set X/ ∼ . It is the co-induced topology on X/ ∼ by π, i.e. the finest topology that makes the projection mapping π continuous and (X/ ∼ , τ ∼ ) is called the quotient space.

The natural topological structure

Now that we are more familiar with the concept of topological structure, we can introduce the hypothesis of continuity inside the formalism.

Chapter 5. The empirical structure as a representation of physical continuity.

Hypothesis of continuity

Hypothesis 4 (Hypothesis of continuity of actions). The commands sent by the agent to its actuators generates continuous transformations in the motor configurations.

This hypothesis is natural for any embodied agents in a real physical space. Indeed, it is assumed that commands are on/off switches that allows energy to flow in the actuator. Then, it reacts by transducing the incoming energy into mechanical work. However, mechanics have a non-negligible mass and the inertia imposes that the energetic transformation is a smooth function of time and the movement must therefore be continuous in space. Here, we transcend the physicality of space by defining continuity as a property of the map between command and time to motor configurations, and thus this definition stays totally intrinsic to the agent.

Topological formulation of the hypothesis of continuity

Natural topological structure in the motor configuration space

The hypothesis of continuity of actions implies the existence of a topological structure in the motor configuration space M, cf. 10, because otherwise, it would not be possible to define continuity. This topology, noted τ M is called the natural topology of the motor configuration space. And the space (M, τ M ), (5.5) is called the motor configuration space with its natural topological structure.

Moreover, because every motor configurations in M must be reached from a continuous path generated by the agent's commands, the space (M, τ M ) is also assumed to be path-connected. A path from motor configuration m to m can be seen as a continuous transformation, call it a as in 'action', from the unit interval [0, 1] with the standard topology of open intervals, to M with a(0) = m and a(1) = m . Definition 15 (Path connectedness). A space X is said to be path-connected, if there is a path joining any pair of points in X.

More generally, the notion of natural topology will be applied to other sets but will always be defined as the topology co-induced by a continuous map from (M, τ M ) to the considered set (see Definition 13).

Example of natural topological structure

M = {(m 1 , m 2 ) ∈ R 2 ; m 1 , m 2 ∈ [-π, π]}.
The natural identification that allows the agent to continuously travel is performed by simply gluing together the configuration at extreme angles. This "gluing", or more formally the topological identification, is represented with colors in middle and right parts of Figure 5.1. In BLUE: when m 2 = π or m 2 = -π the motor configurations are identified to an oriented circle by keeping the orientation when taking the values of m 1 from -π to π. In RED: similarly, when m 1 = -π or m 1 = π the motor configurations are identified while keeping the direction of m 2 from -π to π.

Chapter 5. The empirical structure as a representation of physical continuity.

Towards a topological representation: natural structure in the sensors' poses.

From its experiences, the agent can theoretically access the observable fundamental space (M/ P , σ * ) with its empirical structure. We would like to characterize the conditions under which this space actually represent continuity in the physical space. Therefore, we need to switch from an internal point of view, quotient spaces of the motor configuration space M with empirical topological structure, to an external point of view, quotient space of the pose space X with a natural structure of continuity.

Natural topological structure in the pose space.

Now that we have defined continuity in the motor configuration space, it is possible to define the natural topological structure in the pose space, i.e. a topology τ X on the set of poses X . The term "natural" represents here the fact that it is co-induced by a natural topological structure. Equivalently, it is the finest topological structure in X that make the forward model, the map f : M → X , continuous. The open sets are constructed along with Definition 13 such that

τ X = {O ⊆ X ; f -1 (O) ∈ τ M }.
(5.6)

The space (X , τ X ) (5.7)

is the pose space with its natural topological structure. We can notice that in robotics, the topological structure of the sensor poses is generally a priori constructed from the Euclidean space. This is formalized by the mathematical object of smooth manifolds which are locally equivalent to Euclidean spaces.

However, we have seen earlier that the agent cannot directly represent the points in the pose space because some sensors' poses may be sensory invariant. Therefore, in order to evaluate the proper represented space, we need to define the natural topological structure in the set of observable fundamental poses X / P , also known as the observable sensitive pose space ( § 3.2.2). Following the previous principles of the definition of natural topological structure, we can endow the set X / P with a natural topological structure co-induced by the quotient map X π X -→ X / P .

The natural structure on the observable fundamental pose set

The quotient set X / P can be co-induced with a topology from the quotient mapping π X : π X : X → X / P .

(5.8)

x → [x] P (5.9)

This topology is the called the quotient topology τ X / P and is built as follows

τ X / P = {O ∈ X / P ; π -1 X (O) ∈ τ X }.
(5.10)

The map π X is called a quotient map and by construction it is continuous. Hence, the topology τ X / P is the finest topology that makes the map π X continuous. Then, because it is co-induced from the natural topological structure τ X of X , the topological structure τ X / P is called the natural structure of the quotient set X / P . We obtain the space (X / P , τ X / P ) (5.11) called the observable fundamental pose space with its natural structure.

Chapter 5. The empirical structure as a representation of physical continuity.

A small example of the difference between the pose space and its quotient \(( FIGURE 5.2: An example of topological difference between the pose space X and the observable fundamental pose space X / P with natural topological structures. LEFT: the pose space (X , τ X ) is a surface. MIDDLE: if a part of the pose space is always black, it will be identified to a single sensory equivalence class, i.e. a point in the topological space. Let's assume that every other points are separated asymptotically during the refinement process. RIGHT: the blue part of space (X / P , τ X / P ) with the co-induced topology become a topological sphere because all the points of its boundary are glued to single point. Then, the pink region is a surface which is attached to the sphere via the black point. We can notice that a continuous trajectory in MIDDLE is kept continuous in the RIGHT. The dashed lines represent the boundary of the pose space.

In Figure 5.2 is represented an example for which the pose space and the observable fundamental pose space both with natural structure are different.

Utility of the observable fundamental pose space

Then the observable fundamental pose space (X / P , τ X / P ) is the space to be represented as it captures the natural structure of continuity of the physical world under the constraint that its points can be distinguished from the sensory inputs. But does its topological properties can be useful for the agent ? In this work, the utility of an internal representation can be defined as its ability to represent continuous paths of the sensors in the physical world. Indeed, this would allow the agent to perform trajectory planning of its sensors in its internal representation. And the link between the motor configuration space (M, τ M ) to the sensitive pose space (X / P , τ X / P ) can be interpreted as a continuous "forward sensorimotor model".

The empirical topological structure

109 Therefore, we can show the following useful property in the observable fundamental pose space:

Proposition 14. The space (X / P , τ X / P ) is path-connected.

Proof. From the Hypothesis of continuity 4, we have that (M, τ M ) is a pathconnected space. It is well known that path-connectedness of a space is preserved with continuous transformations. Therefore (X , τ X ) is path-connected because f is a continuous transformation and (X / P , τ X / P ) is also path-connected because the quotient map π X is also continuous.

Therefore, for any two observable fundamental equivalence classes [x] P and [x ] P in X / P , there is a path between them γ P : [0, 1] → (X / P , τ X / P ) (5.12) such that γ P (0) = [x] P and γ P (1) = [x ] P and there exists a path

γ : [0, 1] → (X , τ X ) (5.13)
such that γ(0) = x ∈ [x] P and γ(1) = x ∈ [x ] P and there exists a path

a : [0, 1] → (M, τ M ) (5.14) such that a(0) = m ∈ f -1 (x) and a(1) = m ∈ f -1 (x ).
Here the topological structures has to be specified because each path must be continuous. Therefore the agent can plan executable paths if they are obtained from the observable fundamental pose space with its natural topological structure.

We shall switch to the internal point of view of the agent and define the topological structure obtained from the empirical observations made by the agent.

The empirical topological structure

To be able to compare the topology between the natural and empirical structure, we first need to introduce the topological structure induced from the sensory dissimilarity distance.

Chapter 5. The empirical structure as a representation of physical continuity.

The spaces (M, σ) and (M/ P , σ * ) are respectively pseudometric and metric space. However a pseudometric or a metric induces a topology on the set on which they are defined. This topology is called a metric topology, respectively a pseudometric topology, and is generated by the family of open balls.

Definition 16 (Metric/Pseudometric topology). Let (X, d) be metric (resp. pseudometric) space. One can build the family of the open balls as the family of the sets

B d (x 0 , ) = {x ∈ X; d(x 0 , x) < } (5.15)
for all x 0 ∈ X and all > 0. Then the metric topology (resp. pseudometric topology) is the topology where the open sets are all the subsets that can be realized as the unions of open balls.

Therefore, the empirical topology on the set M/ P , let's denote it τ σ * , is the one generated by the family of open balls from the metric σ * , so that its open sets are unions of the open balls:

B σ * (k 0 , ) = {k ∈ M/ P ; σ * (k, k 0 ) < } (5.16)
where k 0 ∈ M/ P and > 0. Similarly one can induce the set M with a pseudometric topology τ σ from the pseudometric σ. From an external point of the view, we have described the natural topological structures of the motor configuration space (M, τ M ), the pose space (X , τ X ) and the observable fundamental pose space (X / P , τ X / P ) from the agent's internal point of view, it can theoretically access to the asymptotic empirical structure of the motor configuration space (M, τ σ ) and the observable fundamental motor configuration space (M/ P , τ M/ P ) as can be seen in Table 5.1.

The representation of physical continuity in the empirical structure.

In order for the space (M/ P , τ σ * ) to possess the properties of physical continuity, it must be topologically equivalent to (X / P , τ X / P ). This is obtained when both spaces are homeomorphic (see Definition 11). We shall describe some "natural" assumptions that allows for this to be valid.

(M, τ σ ) (M/ P , τ σ * ) π M (M, τ M ) (X , τ X ) (X / P , τ X / P ) f π X TABLE 5
.1: The internal and external theoretical topological structures.

The hypothesis of topological coherence between topological structures

The empirical topological structure is obtained from the statistics of the refinement process which in turn is derived from stochastic properties in the environment.

Hypothesis of topological coherence

Let's take the point of view of the sensor's poses in the physical space. If two poses are not distinguished in the physical space then they shall not be distinguish from a sensory point of view. Therefore, such "spatially equivalent" poses must have a zero probability to be distinguished. This implies that the empirical structure can't be strictly finer than the natural structure, it is conceptually impossible.

It is natural to consider that physical properties of space are very similar inside "small" region of space. Hence, if we take two sensor's poses, they have a certain sensory invariants statistic based on the probability of the environment to distinguish them. So, if one sensor is moved continuously and the other stay fixed, then the sensory invariant statistics should also change continuously. This property is called the topological coherence of the sensory invariants statistics and it is formalized in the following way.

Hypothesis 5 (Topological coherence). The natural topological structure is finer than the empirical topological structure (on the same set). continuity.

It follows that the identity mapping from the space (M, τ M ) to the space (M, τ σ ) is a continuous map. Indeed, we have that τ M is finer than τ σ : τ σ ⊆ τ M , so τ M contains all the open sets in τ X , therefore, taking any open set in τ σ then it is also in τ M so that the identity (M, τ M )

id.

-→ (M, τ σ ) is continuous by Definition 10.

We can remark that, if the agent has access to its natural topological structure in its motor configuration space (such as the torus configuration space as in the example shown in Figure 5.1) and can compute the limits of its sensory dissimilarity, then it is theoretically possible to verify if Hypothesis (H5) is verified by checking for discontinuities in the identity mapping.

Finally, we obtain a useful property on the observable fundamental motor configurations space, used as a representation: Proposition 15. If Hypothesis (H5) is satisfied, the observable fundamental motor configuration space with its empirical topological structure (M/ P , τ σ * ) is path-connected.

Proof. Under Hypothesis (H5) we have that the empirical space (M, τ σ ) is a continuous image of a path-connected space, so it is also a path-connected space. The map π M : (M, τ σ ) → (M/ P , σ * ) being a quotient map, it is continuous and thus (M/ P , σ * ) is a continuous image of a path-connected space.

We shall now derive all the properties of internal representation obtained if the topological coherence hypothesis is valid.

Implications of the hypothesis of topological coherence on the pose space

The hypothesis of topological coherence allows us to have one side of the topological equivalence between the internal representation (M/ P , τ σ * ) and the represented space (X / P , τ X / P ). This subsection describes the technical constructions needed to obtain this side of the relation.

Let's transfer the sensory dissimilarity σ between pairs of motor configuration to a sensory dissimilarity p between pairs of poses. Hence, given two 5.5. The representation of physical continuity in the empirical structure. 113 poses x, x ∈ X , the map p is defined as: p(x, x ) = σ(m, m ), for any m ∈ f -1 (x) and m ∈ f -1 (x ).

(5.17)

We can notice that, p is well defined as the sensory inputs does not change when the pose is fixed, so it is the same for all the motor configurations leading to the same sensors' pose as implied by the condition m ∈ f -1 (x). Therefore, the map f from (M, σ) to (X , p) is an isometry. Proposition 16. The map p is a pseudometric on X .

Proof. The proof is easy from the fact that σ is a pseudometric and f is a isometry.

The space (X , p) is a pseudometric space. Furthermore, similarly to the metric identification of σ to the quotient space at (4.12), let's define the dissimilarity p * between the equivalence classes in X / P . For equivalence classes Proof. Similarly to Proposition 13, the proof follows from metric identification.

Hence, the space (X / P , p * ) is a metric space and it can be linked to the motor configuration space with sensory dissimilarity σ * with the following proposition.

Proposition 18. The two spaces (M/ P , σ * ) and (X / P , p * ) are isometric and the map f P :

(M/ P , σ * ) → (X / P , p * ) such f P ([m] P ) = [x] P where x = f (m), is a bijective isometry.
Proof. Taking any pair [m] P , [m ] P ∈ M/ P of motor configurations equivalence classes, by definition of p * we have that

σ * ([m] P , [m ] P ) = σ(m, m ) = p( f (m), f (m )) = p * ( f P ([m] P ), f P ([m ] P ))
Chapter 5. The empirical structure as a representation of physical continuity.

so that f P is an isometry. From Proposition 5, f P is a bijection and the result follows.

Therefore, let's call τ p and τ p * the pseudometric and metric topology generated by the open balls in the pseudometric p and the metric p * on X and X / P . Then, the hypothesis of topological coherence gives us the following proposition on the sensory dissimilarity in the pose space.

Proposition 19. If Hypothesis (H5) is verified, then natural topology τ X on the pose space is finer than the topology τ p generated by the pseudometric p.

More formally, for any pose x ∈ X and taking any scalar ρ > 0 which defines an open ball centered at x as B p (x, ρ) = {x ∈ X ; p(x, x ) < ρ}, then there always exists an open set O ∈ τ X in the natural topology that is included in this open ball, O ⊂ B(x, ρ).

Proof. The forward kinematics function f : (M, τ σ ) → (X , τ p ) is continuous (it is an isometry). Then, from Hypothesis (H5), the identity mapping id. :

(M, τ M ) → (M, τ σ ) is also continuous, so that f : (M, τ M ) → (X , τ p )
is continuous too as a composition of continuous functions. Because τ X is the finest topology on X that makes f : (M, τ M ) → X continuous, necessarily it is finer than τ P .

Then we can compare the natural and empirical topologies in the observable fundamental pose set X / P . Proposition 20. If Hypothesis (H5) is verified then, the topology τ σ * is finer than the topology τ X / P , that is:

τ X / P ⊆ τ σ * . (5.19)
Equivalently, the identity map (X / P , τ

X / P ) → (X , τ p * ) is continuous.
Proof. For some clarity, let's add the following diagram

(X , τ X ) (X , τ p ) (X / P , τ X / P ) (X / P , τ p * ) π X id. X π X id. X / P
(5.20) 5.5. The representation of physical continuity in the empirical structure. 115 where id. stands for the identity map. We need to show that the identity map id. X / P from (X / P , τ X / P ) to (X / P , τ p * ) is continuous. From Proposition 19 the identity map id. from (X , τ X ) to (X , τ p ) is continuous. Moreover, π X being a quotient map, it is continuous from (X , τ X ) to (X / P , τ X / P ). The map π X , this time considered from the pseudometric space (X , τ p ) to the metric space (X / P , τ p * ) is called a metric identification, it preserves the induced topologies, i.e. the open sets in (X / P , τ p * ) and the open sets in (X , τ p ) are generated by the same families of open balls, and thus it is continuous. Hence, the composed map π X • id. X from (X , τ X ) to (X , τ p * ) is continuous and for any open set O ∈ τ p * , we have that π -1 X (O) ∈ τ X . Thus, the subset O ⊆ X / P has an open preimage by the quotient map π x in (X , τ X ), therefore it must belong to the quotient topology τ X / P . Hence, the map id. X / P from (X , τ X P ) to (X , τ p * ) is continuous. Then, any open set that belongs to τ p * must also belongs to τ X / P . Proposition 21. If Hypothesis (H5) is verified then the map f -1 P from (X / P , τ X / P ) to (M/ P , τ σ * ) is continuous.

Proof. As f P is a bijective isometry from (M/ P , τ σ * ) to (X / P , τ p * ), so is f -1 P from (X / P , τ p * ) to (M/ P , τ σ * ), in particular it is continuous. From Proposition 20, the identity map from (X / P , τ X / P ) to (X , τ p * ) is continuous and by composition of continuous map, we have that f -1 P from (X / P , τ X / P ) to (M/ P , τ σ * ) is continuous.

Under Hypothesis (H5), f -1 P is continuous and so, if there is an open set in the empirical topology τ σ * then its image by f P in X / P belongs to the open sets in the natural topology. Moreover, it also implies that the natural topology τ X / P is finer than the empirical topology τ p * and so the natural topology may have open sets that are not mapped to open sets in the empirical topology. However the empirical topology is induced by a metric on the space which implies strong separation properties on the set of points. The natural topology being finer, this should implies also strong separation properties in the natural topology.

Chapter 5. The empirical structure as a representation of physical continuity.

Separation properties

Separation properties are restrictions on the structure of a topological space that describe the concept of separation between points or sets. As an example, the metric spaces hold a strong separation property which is that they are Perfectly Normal Hausdorff spaces. Here we will be interested to the Hausdorff separation property as it will be sufficient to prove the result on topological equivalence of natural and empirical structure in the observable fundamental space.

Definition 17 (Hausdorff space). A topological space is said to be Hausdorff if, for each pair of distinct points, there exists a disjoint pair of open sets each containing one of the points.

Proposition 22. Metric spaces are Hausdorff.

Proof. Let (X, d) be a metric space where d is a metric on X. Then for any pair of points x, y ∈ X where x = y, then d(x, y) > 0 so if we define two open balls centered on x and y with radius

d(x,y) 2 as B(x, d(x,y)
2 ) and B(y, d(x,y) 2 ) they must be disjoint, i.e.

B(x, d(x, y)

2 ) ∩ B(y, d(x, y) 2 ) = {∅}.

(5.21)

Moreover, these balls belong to the topology induced by the metric so they are open sets.

As an example, let's take the real line R and the topology induced by standard Euclidean distance; it is a metric space so it is Hausdorff. Then for any pair of distinct points a, d ∈ R such that a < d, then there always exists two scalar b, c ∈ R such that a < b < c < d and the open intervals ] -∞, b[ and ]c, ∞[ respectively contain a and b and they are disjoint. We can sense here that this property allows to "separate" every points of the space using disjoint open sets in the topology.

Then naturally, thanks to the inclusion of the topologies τ σ * and τ X / P on the quotient pose space X / P , we have the following proposition. Proposition 23. Under Hypothesis (H5), the space (X / P , τ X / P ) is Hausdorff.

5.5. The representation of physical continuity in the empirical structure. 117

Proof. The space (X / P , τ p * ) is a metric space, so it is Hausdorff. For any two distinct points there exists disjoint open sets in τ p * that contains each point.

Then according to Proposition 20 these open sets also belongs to τ X / P and thus (X / P , τ p * ) is Hausdorff.

We have now that both the natural topology τ X / P and the empirical structure τ p * have the Hausdorff separation property and that the natural topology is finer than the empirical topology on the pose space. In order to have the equivalence between the topologies, there must be a limitation on the "size" of the natural topology. This is achieved through the introduction of a new hypothesis.

The hypothesis of compactness of the motor configuration space Definition and statement

A sufficient way to put the restriction on the natural topology the following hypothesis on the motor configuration space (M, τ M ) with the natural topology.

Hypothesis 6 (Compactness of the motor configuration space). The motor configuration space (M, τ M ) with the natural topology is compact. Chapter 5. The empirical structure as a representation of physical continuity.

example is that continuous maps on a compact metric space are uniformly continuous. In our case, this allows to restrict the natural topology so that there always exists finite sub-covers.

Indeed, this hypothesis is very common when dealing with configuration spaces of robotics mechanical linkage, (see [START_REF] Farber | TOPOLOGY OF ROBOT MOTION PLANNING[END_REF]. As an example, the topological space of the simple 2DoF agent shown in Figure 5.1 is a torus which is a compact space. In fact, any configuration space that is obtained from revolute joints is a compact space.

The Hausdorff property can therefore be seen as the property of having "enough" open sets for separation between points and sets, and the compact property for "not to many" open sets in order for structural information to be valid globally on the space. The combination of both properties together with the assumption of continuity of the bijective map f -1 P allows for a very strong topological result.

Topological equivalence of the internal representation with the natural topological structure of the fundamental observable pose space.

Proposition 24 (Main result). If Hypotheses (H5) and (H6) are satisfied then the natural topologies and the empirical topologies coincide and the spaces (M/ P , τ σ * ) and (X / P , τ X / P ) are homeomorphic.

Proof.

Lemma 24.1. Compactness is a topological property preserved by continuous map.

Hence from Lemma 24.1 the pose space (X , τ X ) and the fundamental observable pose space (X / P , τ X / P ) are compact spaces.

Lemma 24.2. Any continuous map from a compact space to a Hausdorff space is closed.

A map is said to be closed if the image of closed sets are closed sets.

Proof. The theorem is very standard and its proof can be found in N. R. [START_REF] Howes | Modern Analysis and Topology[END_REF]

Summary of the topological constructions 119

The map f -1 P is a continuous map from the compact space (X / P , τ X / P ) to the Hausdorff space (M/ P , τ σ * ) so, from Lemma 24.2, it is a closed map. Then, for any closed set C in (X / P , τ X / P ), then f -1 P (C) is also a closed set in (M/ P , τ σ * ). However f -1 P (C) is the preimage of C by f P . So that the preimage in (M/ P , τ σ * ) of closed sets in (X / P , τ X / P ) by f P are closed sets. Hence f P is a continuous map. On the considered spaces, f P is a bijective continuous map with a continuous inverse so it is a homeomorphism and the spaces (M/ P , τ σ * ) and (X / P , τ X / P ) are homeomorphic.

We have shown that if the sensory dissimilarity p * is continuous on the natural topology of space and if the motor configuration space is compact then the internal representation (M/ P , τ σ * ) is topologically equivalent to the fundamental observable pose space with its natural topological structure. Hence the refinement process on the whole motor configuration space allows for the agent to capture all the topological properties that can theoretically be captured from the sensory invariants of the agent's interaction with the environment.

Summary of the topological constructions

In the last sections, we have attempted to propose a chronological and detailed approach to the hypotheses brought to the problem of topological representation for non specialists in topology. These results and properties can be summarized in a more condensed way.

First, the structures can be summarized in the following list,

• (M, τ M ) is the motor configuration space with its natural topology.

• (X , τ X ) is the pose space with its natural topology.

• (M, τ σ ) is a pseudometric space and with its empirical topology.

• (X , τ p ) is a pseudometric space and with its empirical topology.

• (M/ P , τ σ * ) is a metric space and with its empirical topology.

• (X / P , τ p * ) is a metric space and with its empirical topology.

Chapter 5. The empirical structure as a representation of physical continuity.

• (X / P , τ X / P ) is the observable fundamental pose space with its natural topology.

The relations between some of the spaces can be shown with the following diagram,

(M, τ M ) (M, τ σ ) (X , τ X ) (M/ P , τ σ * ) (X / P , τ X / P ) id. M π M f π X f P .
(5.24)

Where f is the forward model, π M and π X are quotient map from motor configurations or poses to their equivalence classes in the observable fundamental sets, and f P is the map that links the internal representation (M/ P , τ σ * ) to its represented space (X / P , τ X / P ). In blue are the natural topological structures and in red the empirical topological structures.

The first hypothesis is the Hypothesis of continuity of action (H4). It implies that there exists a topology τ M in the motor configuration space M such that the actions induced by the agent's commands are necessarily continuous.

Then, the natural topological structure τ X in the space X is the topology co-induced (see Definition 13) by the continuity of the forwards model f : (M, τ M ) → X . It makes the forwards kinematics model, so that the laws of classical mechanics are satisfied.

Similarly, the natural topological structure τ X / P in the observable fundamental space X / P is again co-induced by the composition of π X • f where π X is the quotient map from X to X / P . So that the continuity of actions is again satisfied from paths in X / P .

Then, we consider the empirical topological structure induced by the statistics of sensory invariant, i.e. the sensory dissimilarity σ on M. The topology τ σ on M is the one generated by family of open balls in the pseudometric σ while the topology τ σ * on the observable fundamental motor configuration set M/ P is the one generated by the family of open balls in the metric σ * .

The second hypothesis is the Hypothesis of topological coherence (H5) such

Illustration

121 that the natural topology is always finer than the empirical topology. It basically assumes that if there is continuous variation of the sensory dissimilarity between motor configurations, it must have been generated by a continuous change in the motor configuration. This hypothesis implies the inverse mapping f -1 P is continuous.

The third hypothesis is the Hypothesis of compactness of the motor configuration space (H6). This hypothesis together with the previous hypothesis imply that f P is a homeomorphism so that, in the observable fundamental set, the empirical and natural topological structures are equivalent. Therefore the space (M/ P , τ σ * ) built from sensory invariants is topologically equivalent to the space (X / P , τ X / P ). The space (X / P , τ X / P ) represent the set of "sensitive" sensors' poses, i.e. the sensor poses that can be distinguished with probability 1 from their sensory inputs, which possess the natural topological structure obtained from physical continuity. Chapter 5. The empirical structure as a representation of physical continuity.

Illustration

Figure 5.3 illustrates the internal representation obtained for illustrative environments with 3 different distribution of straight lines. In this figure, are represented the three internal representation (in pink) obtained from the sensory dissimilarity 1 σ * , 2 σ * and 3 σ * on the observable fundamental spaces M/ P 1 , M/ P 2 and M/ P 3 . In these environments, the natural topological structure in the motor configuration space is a torus, as shown in Figure 5.1. The torus being a compact space, Hypotheses (H4) and (H6) are verified. Furthermore, the co-induced topology in the pose space is naturally the standard topology in the 2-dimensional Euclidean space induced by the Euclidean metric.

For environment 1, the set M/ P 1 as well as X / P 1 are the set of reachable position D from the planar agent. Therefore, the represented space, i.e. the space (X / P , τ X / P ), is simply a 2 dimensional disk in the Euclidean space. Moreover, it has been shown in the previous chapter on structure emergence that the metric 1 σ * actually deforms the Euclidean neighborhoods according to their radius in the pose space but the neighborhoods are preserved so that Hypothesis (H5) is verified. The resultant space is a surface that is represented as a projection in the 3 dimensional Euclidean space with little deformation from the original space. The internal representation is still a 2-dimensional surface which is of course topologically equivalent to the 2 dimensional disk.

For environment 2, again the set M/ P 2 as well as X / P 2 are the set of reachable positions in D so that the represented space is also a disk in the Euclidean space. Moreover, the metric 2 σ * is directly proportional to the Euclidean distance so that Hypothesis (H5) is verified. The internal representation (M/ P 1 , σ * ) is also a disk, a little smaller but still topologically accurate.

Then, for environment 3, the observable fundamental sets M/ P 3 as well as X / P 3 have been shown to be the set of points in a horizontal segment line. Therefore, natural topology co-induced into the quotient set X / P 3 is the standard topology on the interval [-1, 1], i.e. the topology generated by the euclidean distance in 1 dimension, so that the represented space is a 1dimensional segment line. The metric 3 σ * being exactly the length between two points, Hypothesis (H5) is verified and the generated topology is also the standard topology on the interval. Hence, the internal representation is also topologically accurate.

Conclusion 123

Finally, environment 4 is a counter example, to see what happens when at least one hypothesis is not verified. In this environment, the straight lines distribution are the same that in environment 2, so that the statistics of sensory invariant are normally uniform in the pose space (see the internal representation for environment 2). However, one part of the pose space has been colored in black. When a sensor's pose is inside the black ring, the sensory input is always "black". In this environment, the natural topological structure of the observable fundamental pose space has been explained in Figure 5.2 to be topologically equivalent to hollow sphere attached to a surface, as shown in Figure 4.1 with the blue and pink surfaces. However, because a sensor pose on the black ring has a 50% chance to be sensory equivalent to any other pose irrespective of its distance, the statistics of the sensory invariant are discontinuous at the vicinity of the patch, separating the observable fundamental motor configuration space into 2 disconnected surfaces and a single point. In this case, the Hypothesis (H5) of topological coherence is not verified. Therefore, the internal representation is not path-connected and thus not topologically representative of the physical continuity. Even though, we can notice that, if the agent constrains the motor configurations so that they correspond to points inside one connected surface, then the topological properties are still verified. However, this discontinuity, due to the independence of the sensory invariants to a change in environment may have a very interesting interpretation. Indeed, the sensory invariants that does not change with the environment may be caused to the agent interacting with something relatively static such as its own body.

Conclusion

This chapter has attempted to give an interpretation to the internal space built through the refinement process. Indeed, this space is not directly a representation of the configuration space of the sensor. However, this space is shown to have nice topological representation properties under some topological assumptions. The first two important topological assumptions concern the motor configuration space, it must preserves the continuity of action and be compact. Secondly, the statistics of sensory invariants must vary continuously when the agent performs continuous actions. Then, the internally Chapter 5. The empirical structure as a representation of physical continuity.

built space actually represent something which has a physical reality, i.e. the space of observable fundamental poses endowed with the natural structure of continuity in the physical space. Furthermore, the empirical space being an accurate topological interpretation, it should be exploitable for path planning. Indeed, for a planned path in the internal representation, the agent has the guarantee that there exists continuous motor commands that follows the planned trajectory.

Now, that the theoretical properties of the refinement process have been formalized and studied, we can introduce a more experimental version of the refinement. Indeed, the experiences of the agent must be limited in time and in space, additionally, the environment may not be still during the exploration. All these considerations might lead to very different internal representations than in the provided theory. Indeed, real internal representations should, at best, be distorted versions of the theory. The next chapter consider the agent's interaction in its limited and distorted forms.

Part II

Experimental framework and applications

Chapter 6

Evaluation of the Internal Representation: an Experimental Framework

So far, theory have shown that, under some specific assumptions, the agent can build an internal representation of the "sensitive" sensors poses by exploiting sensory invariant during a repeated interactions with its environment. The theoretical development assumed that the agent was exploring the totality of its motor configuration space. However, in reality, the agent perform an active exploration of its motor configuration which is, of course, limited in time and space. Hence, the sensory statistics can only be computed for a subset of its motor configurations. Therefore, the internal representation contains a finite set of points which can be refined a finite number of time. In this context, the agent does not obtain strict topological structures.

In this chapter, we shall show that even under these limitations, the internal representation can still have some representative properties, via the preservation of local structures between finite graphs. We shall present two criteria to evaluate the structural correctness of the internal representation inspired from Nonlinear Dimensionality Reduction techniques. Then, some simulations shall be presented as proof of concepts, with different environments of different levels of complexity. Moreover, we shall also provide some simulated examples on how the internal representation can be exploited by the agent for path planning applications.

Chapter 6. Evaluation of the Internal Representation: an Experimental Framework

Introducing an experimental point of view on

the refinement process

Experimental setup

To begin with, let's consider a situated and embodied agent with commendable actuators and sensors in and on its body. The agent is able to perform actions via commands sent to its actuators, an action is a function that changes the motor configuration, such that it is a map

a : M → M. (6.1)
The agent starts at a "resting state" given as a motor configuration m 0 . Then, the agent performs active exploration of its motor configurations. This exploration is performed naively as a naive babbling. The babbling is done by generating a set of successive random actions A = [a 1 , a 2 , a 3 , • • • , a N exp ] and then goes back to its home state m 0 with a "resting behavior" which is assumed to exist. The motor configurations obtained through this exploration form a set is called the motor exploration set, which is a finite subset of points in M, such that

M = {m i } i=1,••• ,N exp (6.2)
where

m k = a k (m k-1 ). (6.3)
At each motor configuration m i , the agent receive a sensory input s i which depends on the current environmental state at the moment when the input is read.

Then, the agent repeats the exploration of the set M, each repetition being parameterized by an index k. Thus, at repetition k and motor configuration m i , the agent sensory input is noted

s i [k].
At the end of each repetition of M, the agent can compare the sensations between all pairs (m i , m j ) of all motor configurations in the exploration set M. For repetition k, one can define the N exp × N exp sensory dissimilarity matrix 6.1. Introducing an experimental point of view on the refinement process129

D[k] of all comparisons whose elements, noted D ij [k], are computed such that D ij [k] = σ k (m i , m j ) = 1 k k ∑ l=1 1 s i [l] =s j [l] . (6.4)
The elements D ij [k] of the dissimilarity matrix D[k] represent the frequency for two motor configurations m i and m j of having generated different sensory inputs during the k repetitions.

The hypothesis of static environmental state during the exploration

If during each exploration, the environmental state is static, then the zeros in D[k] actually represent the sensory equivalence classes [ . ] σ k as defined in equation 4.6. So that, at a repetition k, from the agent point of view, we have the following equivalence between motor configurations

m i ∼ m j when D ij [k] = 0. (6.5)
Thus, the equivalence classes in M are defined such that

[m i ] D[k] = {m j ∈ M; D ij [k] = 0}. (6.6)
Therefore, before any exploration, i.e. when k = 0, D[0] can be initialized as a null matrix: all motor configurations have not been distinguished from each other. Then, as the number of repetitions increases, the agent shall notice a refinement of the sensory equivalent classes of the motor configurations. We can notice that, if the environment state is static during the exploration and if the interaction satisfy the hypothesis of convergence of the statistics introduced in Chapter 3, then the sensory dissimilarity D ij [k] converges to the value of the asymptotic sensory dissimilarity σ(m i , m j ). For the sake of clarity and when there is no ambiguity, D[k] will be simply noted D.

Chapter 6. Evaluation of the Internal Representation: an Experimental Framework

The condition of having a static environment during a repetition has been introduced to ease the development of the formalization. Dealing with a reallife scenario requires to take into account a continuously changing environment. These changes might add some variance on the sensory dissimilarity D ij , so that it is possible that it does not properly converge towards the theoretical dissimilarity σ(m i , m j ). We shall introduce a criterion that evaluates the presence of distortion in the internal representations.

The internal representation

Similarly to the previous chapters, the internal representation is the quotient of M with the equivalence relation being a sensory dissimilarity D equal to 0. When there is a 0 at row i and column j with i = j, then the motor configurations m i and m j must have always generated the same sensory inputs. Hence, rows and columns i and j of D are necessarily identical. Then m i and m j shall be regrouped into an equivalence class. Let's call M/ D , the quotient set of M that regroups the motor configurations with dissimilarity D equal to 0. Then, we can extend the dissimilarity matrix to the quotient set with D * such that

D * ([m i ] D , [m j ] D ) = D(m i , m j ). (6.7)
Then the only 0 in the distance matrix D * are placed on the diagonal. Therefore, the internal representation is the space (M/ D , D * ) and it is a finite metric space (similarly to Proposition 13).

A finite metric space can always be represented by a real, positive, symmetric matrix (e.g the distance matrix) or by an undirected, weighted, finite complete graph. A Complete graph is a graph where any pair of graph vertices is connected with an edge. Therefore, the internal representation can be represented by the dissimilarity matrix D * or the complete graph G D = (V, E).

The set of vertices V is the set of equivalence classes in M/ D and the set of edges E is such that E(i, j) = (v i , v j , w ij ) where weights w ij correspond to the distances D * ij between the equivalence classes represented by v i and v j . Therefore, we shall adapt the evaluation of the representation using properties of graphs.

Evaluation of the representation: preservation of local structures

So far, it has been proven that, under some hypotheses, the agent is theoretically able to capture topological properties of the quotient pose space. In the experimental case, it is not possible to assess the convergence of the refinement process at a topological level because the explored points are discrete, and the discrete topology is trivial. The evaluation based on topological continuity cannot be performed and should be replaced by the evaluation of preservation of local structures in the form of "small" neighborhoods. Points that are close in one space should correspond to close points in the other space. Indeed it has been shown that the internal representation is a finite metric space which is equivalent to an undirected, no loop, weighted, finite complete graph. Until now, the quotient pose space has only been given a topological structure. In order to evaluate the internal representation metric, it is mandatory to define a metric on the quotient pose set. This will allow a proper evaluation of the metric distortion of the representation.

Evaluation metric on the represented space

Let's assume that the represented space, e.g. the quotient pose space X / P , is endowed with a metric ρ, called the evaluation metric, known to an external viewer and compatible with the natural topological structure in the quotient pose space X / P , i.e. the topology generated by the open balls of the metric is equivalent to the natural topology. This metric could be used in traditional applications to derive cost functions for tasks such as path planning.

In this experimental framework, when the agent explores the set of motor configurations M, it runs through a discrete set X = f (M) of poses. After taking the quotient by regrouping points that are theoretically not distinguishable from a sensory point of view, we obtain the discrete subset X/ P of quotient pose set X / P for which ρ is also a metric. Therefore, the space to be represented by the agent is the discrete set X/ P with the distance matrix R whose elements R ij corresponds to the distances between elements in X/ P : ρ([ f (m i )] P , [ f (m j )] P ). The discrete metric space (X/ P , R) can also be represented by the distance matrix R, or as a weighted complete graph.

Chapter 6. Evaluation of the Internal Representation: an Experimental Framework

Then, the refinement being evaluated after a finite number of explorations, the space M/ D may have less points than the represented set X/ P because the refinement is possibly incomplete and some points in X/ P may not be distinguished in M/ D . Therefore the map between the metric spaces (M/ D , D * ) and (X/ P , R) can be called an embedding between metric spaces and it is an injective map.

The internal representation being a finite graph, the preservation of the topological structure translates naturally to the preservation "small" neighborhoods. In the next section, we will discuss some state of the art techniques for the evaluation of local structure-preserving embeddings to justify the choice of an evaluation criterion for the internal representation.

Evaluation of the quality of local structure preservation of embeddings

The topological similarity between discrete models such as graphs or Self Organizing Maps [START_REF] Kohonen | Self-organized formation of topologically correct feature maps[END_REF] usually deals with the evaluation of the preservation of neighborhoods (T. Villmann et al., 1997). Most of the techniques used for this purpose comes from the field of NonLinear Dimensionality Reduction (NLDR) whose objective is to embed a set of points with known pairwise distances into a smaller dimension space, generally Euclidean, while preserving some structural properties. This can be useful for applications in data visualization but also feature size reduction. Typical techniques of NLDR such as Sammon's mapping [START_REF] Sammon | A nonlinear mapping for data structure analysis[END_REF], Isomap [START_REF] Tenenbaum | A Global Geometric Framework for Nonlinear Dimensionality Reduction[END_REF], Curvilinear Component Analysis (CCA) (Demartines and Herault, 1997), Locally Linear Embedding [START_REF] Roweis | Nonlinear dimensionality reduction by locally linear embedding[END_REF], t-SNE [START_REF] Maaten | Visualizing data using t-SNE[END_REF] aims at minimizing predefined cost functions that generally aim at preserving local structures while keeping far distances within certain bounds. Optimization procedures are then ran such as gradient descents or iterative methods to find the most fitting embedding space for the data points. The cost function is designed so that when optimized, it should provide an embedding with nice structural properties. However, in the literature, there is a lot of different solutions for the evaluation of the quality of an embedding. Some evaluations focus on the preservation of pairwise distances such as in Sammon's mapping, also known as the Sammon's stress, or with an emphasis on small distances, as in CCA, proximities or ranks in [START_REF] Shepard | The analysis of proximities: multidimensional scaling with an unknown distance function. I[END_REF] and [START_REF] Kruskal | Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis[END_REF] or topological connectedness [START_REF] Villmann | Topology preservation in self-organizing feature maps: General definition and efficient measurement[END_REF]T. Villmann et al., 1997). More recently, there have been some approaches attempting to generalize the notion of quality of an embedding by evaluating both its trustworthiness and its continuity [START_REF] Venna | Neighborhood preservation in nonlinear projection methods: An experimental study[END_REF][START_REF] Venna | Neighborhood preservation in nonlinear projection methods: An experimental study[END_REF][START_REF] Venna | Neighborhood preservation in nonlinear projection methods: An experimental study[END_REF]Kaski, 2006). These approaches has inspired the development of a quality criterion called the "local continuity metacriterion" [START_REF] Chen | Local Multidimensional Scaling for Nonlinear Dimension Reduction, Graph Drawing, and Proximity Analysis[END_REF]. These techniques has been finally unified and improved in John A Lee and Verleysen, 2009 and John A. Lee and Verleysen, 2010 which shall be used in the current work.

Evaluation criteria for preservation of local structures

We propose two evaluation criteria to evaluate the structural similarity between the two metric spaces (M/ D , D * ) and (X/ P , R). The first criterion evaluates the number of equivalence classes obtained from the refinement with respect to the number of equivalence classes it should theoretically obtain. The second criterion, gives an evaluation on the preservation of local structures of the embedding between the represented and the representative metric spaces.

The refinement criterion

The refinement criterion C 1 is defined as the ratio of the pairs of configurations (m i , m j ) that have not been distinguished by a sensory difference yet:

D ij = σ n (m i , m j ) = 0, (6.8)
but are distinct in the quotient pose space:

ρ([ f (m i )] P , [ f (m j )] P ) = 0, (6.9)
where ρ is the evaluation metric on the quotient pose space X / P . C 1 can then be computed as

C 1 = |{(m i , m j ); D ij = 0 and ρ([ f (m i )] P , [ f (m j )] P ) = 0}| N 2 . (6.10)
Chapter 6. Evaluation of the Internal Representation: an Experimental Framework

The finest refinement is then obtained when C 1 = 0, meaning that all distinct points in the quotient pose space have been distinguished in the internal representation.

Local structure similarity criterion

The local structure similarity criterion C 2 measures of how well small neighborhoods are preserved between the sensory dissimilarity D * and the evaluation metric R and is based on the adjusted Locally Continuous Meta-Criteria (LCMC) used for Local Multi-Dimensional Scaling (LMDS) in [START_REF] Chen | Local Multidimensional Scaling for Nonlinear Dimension Reduction, Graph Drawing, and Proximity Analysis[END_REF]. The adjusted LCMC is mainly used in the context of nonlinear multidimensional reduction. It evaluates both the preservation of continuity and the trustworthiness of an embedding from measures of dissimilarity on a dataset to a low dimensional Euclidean space. The choice of the adjusted LC metacriteria is also justified by the fact that it is a non-metric criterion, as it uses ranks in the dissimilarities and not metric information. This makes the evaluation more robust to the choice of evaluation metric because any metric that preserves the ordering between the points would give the same value for the criterion. This encourages the fact that the internal representation does not represent metric information but rather local structures.

The adjusted LCMC is computed as follows. Let's N D * K (i) = {j 1 , • • • , j K } be the K-Nearest Neighbors (K-NNs) of configuration i with regard to dissimilarity matrix D * , and N R K (i) = {k 1 , • • • , k K } the K-NNs with regard to the evaluation metric R. Then the neighborhood similarity for point i is simply the cardinality of their common K-NNs:

N K (i) = |(N D * K (i) ∩ N R K (i))|. (6.11)
The adjusted LCMC is given in its global form by a normalized and adjusted average over all points by

Q(K) = 1 KN N ∑ i=1 N K (i) - K N -1 . (6.12)
A value of Q(K) close to 1 indicates a high similarity between all the K-NNs in both spaces. However the adjusted LCMC is a function of the number of nearest neighbors: the higher K, the bigger the considered neighborhoods. But the only interest of the criterion C 2 is to evaluate similarity on "local structures" associated to a small value of K. This can be achieved thanks to the approach by [START_REF] Lee | Scale-independent quality criteria for dimensionality reduction[END_REF], which consists in finding the value K max of K which maximizes the adjusted LCMC along

K max = argmax Q(K). (6.13)
Then, the criterion C 2 is computed as the average below K max of Q as

C 2 = 1 K max K max ∑ K=1 Q(K).
(6.14)

The quantity C 2 assesses the similarity of local structures, e.g. the preservation of neighbors insides the K max -NNs. Then K max represents the scale of the locality of the structures. A value C 2 = 1 indicates that all the neighborhoods of size inferior and equal to K max are perfectly preserved. The value C 2 = 0 can be interpreted as a random permutation of the points where all local structures are lost.

An additional arbitrary criterion: low dimensional data visualization

Additionally to the two criteria introduced previously, it is also possible to project the dissimilarity matrix D * to a low-dimensional Euclidean space by using multidimensional dimension reduction algorithms. This projection can then be visualized to qualitatively assess the structure resemblance between the projected internal representation and the quotient pose space. However, this visualization can not replace the two quantitative criteria C 1 and C 2 in the general case, since the metric space cannot always be embedded into a 2D or 3D Euclidean space without distortions. The algorithm used for the projection is Isomap [START_REF] Tenenbaum | A Global Geometric Framework for Nonlinear Dimensionality Reduction[END_REF]. Given a value K, Isomap performs Classical MultiDimensional Scaling using geodesic distances on the K-NN graph of the dissimilarity matrix D. The visualization being not a criterion per se, any non-metric multidimensional scaling algorithm that preserves local structures such as Local-MDS, SOM, LLE, tSNE Chapter 6. Evaluation of the Internal Representation: an Experimental Framework or Curvilinear Component Analysis can be used. The choice of Isomap is based on its simplicity and the fact that the neighborhood scale K is already available from the computation of the local similarity criterion K max in Equation (6.13).

The process for building the internal representation has now been formalized. Two criteria have been introduced to evaluate if the space (M/ D , D * ) is a good representative of local structures in (X/ P , R). The next section shows the results of the implementation of the refinement process for a simulated agent in different environments.

Simulated results and discussion

This section aims at providing a proof of concept on how a naive agent can build an internal representation of its working space in an unknown environment with an uninterpreted sensorimotor flow. Therefore, the simulated agent will be tested on different environments. The detailed simulation setup is described in a first subsection. Then, quantitative and qualitative evaluations of the obtained representations for two different scenarios of increasing complexity are proposed.

Simulation setup

The agent used in the simulations is the simple 2 DoF serial and planar agent also used for the illustrations in the previous chapters. For all scenarios, the agent's motor exploration set M is chosen in the following way. First, let's recall that the motor configurations of the agent is represented with the tuple m = (θ 1 , θ 2 ), where θ 1 and θ 2 are the two joint angles of the serial arm. The agent starts from a resting position m 0 = (0, 0) with its single-pixel sensor being at the far right of the working space. It then generates random actions sampled from a uniform probability distribution in [0, 2π] after which it goes back to its resting position m 0 . After the i-th action, the agent receives the motor configuration m i . The explored pose space is a set of random points inside a 2D disk. These explored poses are represented by black points in figure 6.1. FIGURE 6.1: LEFT: the 2DoF planar agent. RIGHT: The random exploration points in black and, in color, some forced exploration points that are linked to form topological clues to be represented in the internal representation.

For a better visual interpretation, some topological structural clues have been added by forcing the agent to explore certain configurations. These forced exploration poses are shown with linked colored points in Figure 6.1. Then the explored points inside these rings have been removed from the exploration set. This manipulation allows a better visualization of the capacity of the agent to represent a specific topological structure without loss of generality. At the end, the explored motor configuration space M is composed of N = 500 motor configurations and sensor's poses for every scenario. At the end of repetition k, the agent computes the dissimilarity distance D[k] with equation (6.4) between all pairs of explored configurations. For all the simulated environment, the observable fundamental pose space to be represented is directly the position space of the sensors:

X/ P = X = f (M) = {(x, y) ∈ D}
where (x, y) are the Cartesian coordinates in the Euclidean plane and D is the reachable set of sensor's positions. Then, the chosen evaluation metric is the standard Euclidean distance between the poses

R ij = ρ(x i , x j ) = ||x i -x j || 2 .

Environments description and results

The simulated environments are separated in 2 scenarios of increasing complexity. All environments are black-and-white or gray-scale backgrounds on Chapter 6. Evaluation of the Internal Representation: an Experimental Framework the pose space of the 2D agent. The first scenario is composed of 2 different environments. In this first scenario the refinement process matches with the conditions of the theory: the environmental states are kept static during the exploration of M, the sensory inputs are either 0 or 1. The second scenario is more realistic and is composed of 3 different environments. In this second scenario, the environmental states are allowed to change during the exploration as it would happen in a realistic environment, and the agent's sensory inputs can take more values than 0 or 1.

Scenario 1, static environment during exploration FIGURE 6.2: Scenario 1: (straight lines), a "black and white" straight lines environment random environmental states with a homogeneous distribution on the pose space.

Straight lines environment

In this first environment, the environmental states are identical to those in the environment 2 in the illustrative example and are depicted in Figure 6.2(b). Each environmental state is randomly chosen as a straight line crossing the pose space separating the background into one black and one white areas. The distribution of these straight lines is taken so that they homogeneously cover the working space. The agent's sensory input is either 1 or 0 according to which side of the straight line the agent's sensor is in. At the end of each repetition, a new straight line is randomly chosen and the refinement process continues. This process is repeated until a number of k = 10 6 repetitions of the explorations of M. Figure 6.3 plots the evolution of the refinement criterion C 1 . The finest refinement is obtained after almost 1400 repetitions, meaning that the environment takes a very long time to distinguish points in the representation. Indeed, at each environmental states, the sensory inputs take only two value, additionally, the region of space where these values are constant are localized, i.e. close poses are very likely to be sensory equivalent. In Figure 6.3 is also shown the evolution of the local similarity criterion C 2 between the measured dissimilarity D[k] at repetition k and the Euclidean distance R between the poses of the target pose space shown in SubFigure 6.1(a). The criterion C 2 Chapter 6. Evaluation of the Internal Representation: an Experimental Framework starts very low for the first exploration and then converges towards a value of C 2 = 0.98 which is almost a perfect match of local neighborhoods. Indeed, for this environment the statistics of sensory invariants are invariant to translations or rotations in the 2D Euclidean working space [START_REF] Jaynes | The Well-Posed Problem[END_REF] and it has already be shown in equation 4.23 that in this environment, the measured dissimilarity converges in probability to the Euclidean distance between the poses, up to a constant factor. Due to this convergence, the final internal space should actually be totally equivalent to the represented space endowed with Euclidean space, however the convergence is very slow due to the limited structural information brought at each environmental state.

In order to interpret the values of C

Blobs environment

In this second environment, the environmental states are composed of randomly generated background images with black and white blobs stretched in the bottom left/top right direction of the working space, as shown in Figure 6.4. The blobs are generated using a random noise generator from a procedural Perlin noise with anisotropic filtering (steerable Gaussian filter) oriented at 45 degrees [START_REF] Goldberg | Anisotropic noise[END_REF]. The resultant noisy image is then thresholded to give black and white blobs. The agent sensations are either 1 or 0 depending on the color of the blob the sensor is looking at. After each repetition, a new black-and-white image is randomly sampled. This process is repeated until k = 10 6 explorations of the motor exploration set. Chapter 6. Evaluation of the Internal Representation: an Experimental Framework

Looking at the refinement criterion C 1 represented in Figure 6.5, one can see that the finest refinement is obtained about ten times quicker than for the straight lines environment. Indeed, far poses have a clear tendency to be separated quicker than in the previous environment since blobs are more localized in space. For a given agent, one could then compare environments with respect to the number of repetitions required to reach the finest refinement: the smaller this number, the "richer" should be the environment. The richness of an environment can thus defined as its ability to quickly separate pairs of poses of an agent's explored pose set. In Figure 6.5, the local similarity criterion C 2 converges to a value of C 2 = 0.77 indicating a distortion of local structure in the internal representation D with respect to the evaluation metric R. Indeed, the sensory invariants have a higher probability to occur for a pair of points aligned along the top left/bottom right direction because the blobs are stretched in this direction. Thus, these pairs of points have a lower dissimilarity and are considered closer than in the orthogonal direction. This statistical anisotropy implies different neighboring relations between the poses.

The projected internal representation in Figure 6.5 shows as expected a stretch in the direction of high sensory invariants variance. However, after convergence, the structural cues are visually well preserved indicated by a high C 2 = 0.77, but not as well as in the previous environment.

Until now, the environmental states were kept fixed during the exploration. This gives us an insight into the representation distortion as well as into the notion of richness of the environment. Let's now consider more realistic dynamical environments.

Scenario 2, dynamical environment (Movie) In this last scenario, the environmental states are composed of cropped images of a black-and-white movie frames as shown in Figure 6.6. The agent hovers its sensor across the image; for that purpose, the agent has been centered inside the movie frame and scaled so that the diameter of its working space corresponds to 40 pixels in the image. In order to simulate a spatially continuous environment, the value given by its sensor is the linearly interpolated gray value at the 2D sensor pose from adjacent pixels. In order to account for the possibility for the sensors to have values different from just 0 and 1, the sensory input is quantified on a 0 to 15 gray scale (the range of gray levels, 16 instead of the originally 256, has been adapted to suit the illustration purposes). Of course the agent's sensory inputs are still uninterpreted: it cannot know that a value of 3 is closer to 4 than to 15. Each sensory value is actually seen in this framework as a single symbol so that sensory equivalence occurs only if the agent obtains two 3 or two 4, etc. To simulate the dynamics of the environment, the image is refreshed with the next frame of the movie after a given number of agent's movements. In the environments 3, 4 and 5 the agent respectively moves at a speed of 22 motor configurations per frame (mpf), 6 mpf and 1 mpf: thus, a high mpf indicates low dynamics of the environment. In order to obtain sufficient frames, the movie file 1 has Framework been played 3 times, resulting in 479166 frames, which in turns corresponds to 21082 repetitions for environment 3, 5749 repetitions for environment 4 and 960 repetitions for environment 5. Repeating the same movie allows for more samples but with the same sensory statistics. Indeed, a different movie shall possess different sensory invariant statistics based on the way it is filmed, if it is an action movie, a documentary etc. Here, the movie is the environment in which the agent is, so that's its internal representation is directly representative its sensory statistics. Criterion C 1 plotted in Figure 6.7 implies that the refinement is generally quicker in scenario 2 which indicates richer environments. The first reason comes from a higher number of sensory values: 16 gray scale values instead of the binary 0 and 1, making it less probable to have sensory invariants.

The 1 mpf environment reaches the finest refinement the quickest, indicating that quick refinement is also caused by a higher relative dynamics of sensory changes in the environment with respect to agent's movements. Moreover, Figure 6.7 exhibits that the 22 mpf, 6 mpf and 1 mpf environments respectively converges to C 2 = 0.62, C 2 = 0.4 and C 2 = 0.14. In these dynamical environments, a pair of close points in space but explored after a long time might have a distortion caused by the possible environment changes during the reaching time, thus the higher the dynamics the higher the distortion after empirical convergence. At the bottom of Figure 6.7 are shown the final internal representations obtained at the end of the interaction. Visually the structural clues are well represented when dealing with slow relative dynamics (22 mpf and 6 mpf movie environments). However, considering a high relative dynamics of 1 mpf, the agent's movements are too slow with respect to changes in the environment. As a consequence, two close poses in space that have a high probability to generate the same sensory input if reached quickly, have now a smaller probability to produce the same sensory input as the environment is likely to change during the agent movement. The agent cannot capture the spatial order of the poses anymore and the representation is almost a random permutation of points, which is obtained when C 2 = 0 by definition. In this last case, the internal representation is certainly not exploitable for any task defined in the working space because the agent cannot plan continuous trajectories. However, the distortion in the visualization can also be caused by the the Isomap projection. Indeed, the internal representation may not be always be perfectly projected in 2 dimension leading to a visually distorted representation. That's why the criterion C 2 is much more objective.

Application for path planning

The internal representation can be used by the agent for path planning applications. Path planning is a method that allows a mobile robot to go from Chapter 6. Evaluation of the Internal Representation: an Experimental Framework one position to another position in what is called a free space or working space. The path is usually computed in a way that it minimizes a distance function as well as to avoid collisions. A path planning problem can be translated from the free-space, where the agent moves, to the configuration space, i.e. the space of all configurations of the agent, such as its joint space in the case of a serial arm. In our present case, the agent is not able of performing locomotion, so that its working space is directly the image by the forward kinematics of its motor configurations, i.e. the pose space in which its sensors move. Therefore, the path planning problem is to go from one sensors' pose to another sensor's pose while minimizing the path length.

Constructing the neighboring graph

After the refinement process, the agent obtains a stable internal representation (the finite metric space (M/ D , D * )). Hence, it can measure the length of a path in the graph representation. Indeed, the shortest path in a graph is known as a graph geodesic and its length is computed as the sum of edges' lengths that are followed during the path. However, the graph being complete, every pair of vertices are connected together so that a possible graph geodesic can be the edge that connects directly the vertices together with out passing through any intermediate points. However, this can pose a problem in path planning as an obstacle can be present in-between these points and nothing guarantees that the path between them is in the free space. Generally, one need to compute a neighbor graph in order to find a more robust feasible path, i.e. a path that can be followed by the agent without collision.

A neighbor graph is a graph where connections between vertices are allowed only if they are in the same graph neighborhood. Indeed, a graph neighborhood of a vertex connects all the vertices that are close enough according to a certain neighboring criterion. This closeness is generally computed from two different manners: as the set of vertices that lie below a certain distance which is called the epsilon-nearest neighbor graph epsilon-NNG, or as the k nearest vertices according to the metric defined on the graph, which is the knearest neighbor graph K-NNG. Once this neighbor graph is computed, there is no more edges between far vertices so that the path between two vertices must run through local paths in the graph.

We would like to find the K-nearest neighbor graph of the internal representation graph, noted G D * , without an a priori choice of K. Because the application is path planning, we shall require that this neighbor graph is fully connected, so that K must be big enough. However, when K is big, the shortest path can run through longer edges, which is not ideal because the longer are the edges the more risks there is of a possible collision in the free space. Hence, the value of K shall be small enough but not too big. Hence, we will choose K as the smaller value that fully connects the K-nearest neighbor graph. Thus, the neighbor graph, denoted by G K D * , is computed as follows.

Let G D * be the graph corresponding to the metric space (M/ D , D * ) with G D * = (V, E) where V = M/ D is the set of vertices and E the set of edges such that

E(i, j) = (v i , v j , D * ij ), (6.15)
where

v i = [m i ] D , v j = [m j ] D and D * ij = D * ([m i ] D , [m j ] D )
is the sensory dissimilarity between sensory equivalence classes in M. Then, we follow algorithm 1.

Algorithm 1 K-NNG connected graph G K D * from G D * = (V, E) 1: initialize the neighbor parameter K = 0 2: initialize the graph G K D * = (V, {∅}) 3: while G K D = (V, E) is not connected do 4: K ← K + 1 5:
initialize the edge set E = {∅} 6:

for each vertex v i in V do 7:

compute the set of k neighbors N D * K of v i according to distance D * 8: add, if missing, the edges between v i and its neighbors N D * K with their distances D * in the edge set E. 9: end for 10: end while Notice that in state 8 of algorithm 1, because the distance D * is symmetric, there is only one possible weight for the edge between any pair of vertices. Connectivity of the graph can be checked by any graph traversal algorithm such as Breadth-first search [START_REF] Moore | The shortest path through a maze[END_REF].

Computing the shortest path in a neighbor graph consists at finding a sequence of connected vertices starting and ending at specified vertices that Chapter 6. Evaluation of the Internal Representation: an Experimental Framework minimizes the sum of the followed edge weights. This step can be performed by Dijkstra algorithm [START_REF] Dijkstra | A note on two problems in connexion with graphs[END_REF].

Illustration

The agent

To present the path planning application of the refinement process, we shall introduce a more complex agent. The agent introduced has the same mechanical structure, namely it is a planar 2 DoF serial arm with two limbs of the same length, but instead of having a single pixel camera as an end-effector, it possesses a squared camera of size c × c with p = 9 pixels as seen in Figure 6.8. The pixels are organized on a grid which is centered at the end-effector position so that the center of the middle pixel position is exactly at the end-effector of the agent's arm. The motor configuration space M is the set of possible joint angles (6.16) Therefore, the pose space X = f (M) of the agent is the set of end-effector positions and the corresponding end-effector orientations such that x = f (m) = f ((m 1 , m 2 )) = (x, y, θ), (6.17) where

M = {(m 1 , m 2 ); m 1 , m 2 ∈ [-π, π]}.
x = l cos(m 1 ) + l cos(m 1 + m 2 ), (6.18)

y = l sin(m 1 ) + l sin(m 1 + m 2 ), (6.19) θ = m 1 + m 2 restricted to the interval ] -π, π].
(6.20)

The environment

The environment chosen to illustrate path planning is the simple straightlines environment that separates the Euclidean plane into a black and a white area with the homogeneous distribution shown in Figure 6.8(right). The straight lines cover homogeneously the set D of all positions in space where the agent can sense something, i.e. the set

D = (x, y) ∈ R 2 ; x 2 + y 2 ≤ 2 • l + √ 2 • c 2 .
(6.21)

The exploration and the sensory comparison

Similarly to the previous experimental consideration, the agent performs random babbling to build an exploration set M with N exp = 1000 configurations. The environment is also chosen to be static during the explorations. A new straight line is chosen uniformly from the homogeneous distribution at the end of each exploration.

The sensory input generated at the explored motor configuration m i , is given as the vector of all 9 binary pixels inputs such that 6.22) where s p i ∈ {0, 1} gives 1 when the background of pixel p is white and 0 if it is black depending on which side of the plane is the center of the pixel.

s i = [s 1 i , s 2 i , s 3 i , s 4 i , s 5 i , s 6 i , s 7 i , s 8 i , s 9 i ], (
Chapter 6. Evaluation of the Internal Representation: an Experimental Framework

Without a priori knowledge on the structure of the sensory input, the sensory comparison at a repetition k, between sensory inputs s i [k], s j [k] is computed such that each possible value taken by the sensory vector is considered as a 'sensory symbol'. Therefore, two symbols are either identical or different such that:

δ(s i , s j ) =    0, if s p i = s p j
, for all pixels p ∈ [1, 9], 1, otherwise.

(6.23) Then, after a repetition n, the sensory dissimilarity matrix D is computed between all pairs m i , m j ∈ M of explored motor configurations such that

D[n](m i , m j ) = 1 n n ∑ k=1 δ(s i [k], s j [k]). (6.24)
The quotient space and the evaluation metric

In order to be able to evaluate the internal representation, we need to describe the represented space, i.e. the observable fundamental pose space X / P and its evaluation metric.

In this environment, similarly to what was shown in the previous illustrations, all different poses can be distinguished by a set of straight lines with a probability different from zero, so that the observable fundamental pose space is equal to the pose space, X = X / P . Now, let's describe the natural topological structure in the pose space. One can notice that each pose x = (x, y, θ) can be seen as an element of the Special Euclidean group SE(2) in two dimensions, i.e. the set of translations in the Euclidean plane followed with a rotation, applied from any point in the plane. According to our definition of continuity given under the form of Hypothesis (H4), a continuous transformation in the pose space must be obtained from a continuous action in motor configuration space. Therefore, because the forward model is continuous, any continuous changes in M according to the natural torus topology in M described in Figure 5.1, corresponds to continuous transformation in SE(2) in the pose space. Moreover, the topological structure of SE(2), similarly to the topological structure in the Euclidean space, can be generated by an infinite number of metrics [START_REF] Zefran | Metrics and connections for rigid-body kinematics[END_REF].

One family of such metric is given in the following form. For very γ > 0, the function ρ c is a metric on SE(2), such that for any x = (x, y, θ), x = (x , y , θ ) ∈ SE(2),

ρ γ (x, x ) = (x -x ) 2 + (y -y ) 2 + γ • min{|θ -θ |, 2π -|θ -θ |}. (6.25)
The evaluation metric ρ γ is composed of the Euclidean distance between the Cartesian positions in the Euclidean space in addition with the absolute angular difference between the angles inside [0, π], which are scaled by the constant γ. The constant γ changes the sensibility of the metric to the difference in rotation. As an example, angles are coded either in radian or in degrees which can make the huge difference in the obtained metric, this can be compensated by a change in γ. Then, a higher value of γ makes the distance more sensitive to rotation. In a context of path planning, when the length of the path between two points is given by the metric ρ γ , the choice of γ impacts the optimal trajectory. However, there is no "good" choice of γ, it depends on the application. It may be interesting to evaluate which evaluation metric is closer to the metric in the internal representation.

Results and evaluation of the obtained internal representation

The represented space is the set of explored poses X = f (M) with the metrics ρ γ , which gives the distance matrix noted R γ between pairs of poses in X.

The chosen parameters for the simulation are the following:

• the arms lengths are l = 0.5,

• the camera size is c = 0.5 with p = 9 pixels,

• the random motor exploration is performed for N exp = 1000 with commands on the joints taken uniformly in [-π, π].

• the exploration is repeated for a number of N rep = 10 5 . FIGURE 6.9: The explored pose space. In red are the positions of the end-effector and the arrows represent the orientation of the camera.

The explored poses are represented in Figure 6.9. The distribution of the poses shows a higher density of positions at the center of the pose space. This is naturally due to the kinematic structure of the agent. Indeed, while the exploration is uniform in the motor configuration space M, nothing guarantees that it stays uniform in the pose space. Furthermore, these changes in density may cause some local distortions when evaluating the preservation of local structures, as some spatial areas might be less sampled than others. However, some improvements can be made in future works by adapting the exploration set to obtain the most uniform internal representation possible based on the sensory dissimilarity.

Then, as the refinement is finished after the number of N rep repetitions, let's compute the local similarity criterion C 2 between the final internal representation and the pose space with different metrics obtained with different values of parameter γ indicating different sensitivity to the rotation angles in the pose space. The results are shown in In this table, we can notice that there is an optimal value of γ, i.e. γ = 0.09 for which the internal representation and the pose space have a very high similarity in their local structure. Indeed, when γ = 0.001, the metric in the pose space is only sensitive to the Cartesian position of the sensor so that the orientation almost not influences the pose space's metric. This metric does not preserve the local structure obtained from the rotations, however some local structures are still preserved in the internal representation indicated by a value C 2 = 0.71. This is due to the fact that, the camera being relatively small and centered at the end-effector position, when rotated, the pixels are still relatively close to each other indicating a high probability of generating the same sensory inputs. However, when the pose metric is mainly sensitive to rotation, i.e. when γ = 10, we see an important decrease in local similarity as the Euclidean distance between sensors' positions can be neglected relatively to the distance in orientation.

Chapter 6. Evaluation of the Internal Representation: an Experimental Framework In Figure 6.10 is represented the evolution of criterion C 2 along with the repetitions of the exploration for the optimally representative value of c = 0.09 in the pose space (X, R γ ). We can empirically make the observation that the internal representation does converge towards a stable state, highly representative of the local structures in (X, R γ ). Moreover, the refinement seems to be quicker than in the case of the simple pixel camera in the same environment, as shown in Figure 6.3. However the finest refinement is obtained after nearly 1000 repetitions because one pair of poses was really close in the pose space. Following Algorithm 1 we obtain four different graphs:

Results of path planning

G 7 D * , G 10 R 0.001 , G 7 R 0.09
and G 9 R 10 . Then, shortest paths can be computed while following geodesic paths on the neighbor graphs.

In Figure 6.11 are shown the results of the shortest paths between two poses chosen at the extremities of the pose space for the different metrics.

In SubFigure 6.11(a) is shown the results of the shortest path, in the internal representation represented by the connected neighbor graph G 7 D * , between the two equivalence classes corresponding to poses at two extremities of the pose space. This shortest path is represented in the pose space for better comparison. The path follows a line of with close sensor's pose positions while keeping a slow variation in the orientation of the camera (pointing arrows). The resulting path is very smooth in term of both positions and orientation of the end-effector. Therefore, we can conclude that the obtained path is a very good representative of a smooth motion of the sensors in the physical space. Proper interpolation in the motor configuration would grant a smooth and short trajectory in the pose space. Indeed, this path is actually identical to the one followed in the neighbor graph G 7 R 0.09 shown in SubFigure 6.11(c). This implies that the internal representation graph is almost identical to the graph representing geodesics in SE(2). This is mainly due to the nice statistical properties of the straight line environment, which is isotropic and uniform. Different statistics would of course alter these results by introducing distortions.

To compare with other evaluations metric, one should look at SubFigures 6.11(b) and (d) corresponding to the evaluations metric R 0.001 and R 10 . The first one is a metric that almost does not depend on the orientation in the pose. Therefore, the shortest path is the path that minimizes the Euclidean distance along the way. This leads to large perturbations in the camera directions, as can be seen in the zoom in Figure 6.12. On the other hand in SubFigure 6.11(c), the evaluation metric is mainly sensitive to orientation of the camera so that the camera direction is mostly preserved during successive points in the path. This leads to discontinuities and 'zigzag' in the sensor's poses positions as can be seen in the zoom in Figure 6.12.

Moreover, to be objective, the agent could also plan a smooth and continuous motion of the camera pose by performing a continuous motion in its motor configuration. If the agent is given such knowledge on its motor configuration, it could, as an example, plan a linear interpolation in the motor configuration space. FIGURE 6.13: Path of poses followed by a linear motor interpolation in the motor configuration space.

The resultant path is shown in Figure 6.13 with a linear interpolation of 20 points in the space [-π, π] 2 . The path is clearly admissible as smooth and continuous while being completely suboptimal considering geodesics in the pose space. The reason for this path to be acceptable is that there are no forward kinematics redundancies in this example. The presence of kinematics redundancies would possibly create loops in the linearly interpolated path because of the projection from the high dimensional motor configuration space to the lower dimensional pose space.

Chapter 6. Evaluation of the Internal Representation: an Experimental Framework

Discussion

We have provided a physical interpretation to the internal representation in the case of a simple agent with one kinematic chain and one sensor. However in general, the internal representation, as a representation of the physical structure in which all the agent's sensors are moving, is still difficult to interpret from both the internal and external point of view. [START_REF] Hoffmann | Body schema in robotics: a review[END_REF]. Furthermore, separating the sensors input requires additional interpretation in the sensorimotor flow. In the introduced approach, such improvement can be approached by adapting the sensory comparison operator δ, which computes the sensory invariants and is the only a priori sensory interpretation available to the agent.

Moreover, the internal representation is obtained without any supervised learning. Once obtained, it can be used as an intrinsic supervision information to guide any further improvement in the agent internal constructions.

As an example, it can be used to guide the explorations in the motor configuration in order to obtain a correct sampling in the pose space/observable fundamental pose space but also for higher level constructions, such as the discovery of sensory objects as invariants in successive environment states.

An other limitation in this approach is the fact that the environment must be rich enough for the refinement to occur at sufficient rate. Indeed, when an agent is placed in a room inside a building, then, it is very ambitious to assume that its environmental states are going to vary sufficiently for the internal representation to converge. The idea justifying this hypothesis is that, the agent must be capable of locomotion, i.e. displacement of its full body inside the environment. Also, it should be able, at a point of this life, to distinguish between its own actions leading to an absolute displacement and the actions leading to postural movements in a peripersonal space. If possible, once this distinction has been stated and discovered, then there is a duality between an environment state change and a full body displacement with respect to the refinement process. Hence, the agent should be able, to change the environment "at will" by performing absolute displacement, and thus it becomes easy to stabilize the environment while exploring the motor configurations. The same idea can be applied when dealing with serial kinematics chains. Indeed, if one actuator is situated at the root of a kinematic chain, e.g. in the 2DoF agent the actuator m 1 can change the position of the 2nd part of the arm while m 2 cannot, then the actuated root can be used as an absolute displacement, so that the agent can perform the refinement process on the motor configuration space of the following kinematic part and so on. However, to our knowledge, there hasn't been any work dealing with this kind of learning in the context of environment-agnostic agents (Georgeon and Sakellariou, 2012) with quasi-uninterpreted sensorimotor inputs.

Hereafter, the sensory inputs and motor actions are subject to noises and variabilities. All these perturbations should result in a more or less distorted internal representation. Hopefully, there is room for increasing the robustness of the approach, especially by adapting the set of explored motor configuration as more and more information about the motor configuration space is gathered.

Conclusion

In this chapter, we have provided an experimental approach to the construction of an internal representation of the space of sensors sensitive movements from quasi-uninterpreted sensorimotor inputs. This construction is performed by using the refinement process formally described in the previous chapters. Furthermore, we have explicit two criteria to evaluate the correctness of the internal representation, via the refinement criterion C 1 whose evolution can be seen as a measure of richness in the environment, and the Chapter 6. Evaluation of the Internal Representation: an Experimental Framework local similarity criterion C 2 which evaluates the preservation of local structures in the neighbor graph obtained from the sensory dissimilarity. Different simulations have been presented in some challenging environments, for proofs of concepts of the refinement process. Then some challenging environments show the limitations of a direct application of the proposed techniques. However, there is room for improvement, as an example by using both post-processing and exploration strategies to overcome such problems.

Then, we have also shown that the internal representation can be exploited by the agent in the form of path planning. When the internal representation is structurally accurate, we have shown that the shortest path between points in the internal representation can be very similar to the shortest path in the observable fundamental pose space given an adequate evaluation metric. Moreover, the obtained shortest paths possess all the properties of good candidates to obtain smooth and short trajectories in the sensors pose space.

Once a sufficiently stable internal representation is obtain, it can be used by the agent for higher level representations. Indeed, the agent has learned a sensorimotor forward model from the motor configurations space to a structured "sensitive" sensors pose space. Applications for learning the body schema can be considered. However, in order to obtain more structural information about the agent, the sensory inputs shall be separated into streams of sensory inputs from different sensors so that refinement process can be applied to each different sensory stream. Similarities in the internal representations shall therefore be exploited to find a common space, with identical local structure, to these different sensors "sensitive" poses which may be interpreted as the agent's peripersonal space.

Moreover, the proposed approach is also valid when replacing sensory inputs with sensory features. Indeed, the agent shall have the possibility to group the quasi-uninterpreted sensory inputs into clusters of symbols, or sensory features, that can be based on similarities in the different representations built from them. Hence, the agent's interpretation of the sensory inputs can possibly evolve during its interaction with the environment. This may lead to the concept of sensory objects in the environment and thus rigid transformations which can be used for the discovery of the geometry in space.

Chapter 7

Some applications: sensors topographic structure and the discovery of the self.

In the previous chapter, the refinement process has been developed in an experimental framework as an active exploration process to obtain an internal representation of the explored sensors sensitive poses. Because the sensory comparison is made on the sensory inputs of all the sensors at once, the agent has not given the possibility to separate the different sources of sensory inputs that come from different sensors. Therefore, the agent represents the spatial configurations of all its sensors as a single point in space and a change in the internal representation may represent a movement of all its sensors at the same time. Thus, one challenge for a naive agent is to be able to understand the structure of its sensor apparatus so that the overall internal representation can be organized for each different sensor.

Furthermore, an embodied agent may possibly interact with its own body. Indeed, its body is also a physical object in the environment so that sensors can be sensitive to changes in its configuration. Moreover, at the beginning of its interaction, a naive agent does not have the ability to understand a difference of interaction with its body or with an environment's physical object. However, the question of the separation of the "self" from the environment is very important in developmental robotics but also in cognitive biology as the body of cognitive organisms is the medium of interpretation of any interaction with an environment [START_REF] Hoffmann | Body schema in robotics: a review[END_REF] In this chapter, we shall Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

propose an approach that allows the agent to differentiate self-interaction from the other environmental interactions by exploiting discrepancies in the statistics of sensory invariants in the internal representation.

Discovery of sensors topographical structure

In order to obtain structural information about the different sensors, we shall assume that the agent receive a separated sensor stream from each of its sensor elements, called sensels. As an example, for a camera sensor, we can consider that there is as much pixels as different streams of sensory inputs. Then, each sensory stream can be interpreted independently and each generates an internal representation. Then, by exploiting the similarities between these representations it is possible to obtain the structure of their arrangement.

The approach

Similarly to the experimental framework described in the previous chapter, let's consider an agent that can generate commands that acts on its actuators whose states are parametrized with a proprioceptive input in the motor configuration space M. The agent has a number of different sensors that can be moved in space from its motor commands. The difference between the previous chapters is that here, each sensor element, or sensel, p sends a unique stream of sensory inputs s p ∈ S p where S is the set of sensory input of sensor p, so that the set of all possible sensory inputs for all sensels is the concatenation of all sensory sets:

S = S 1 × S 2 × S p × • • • S N s , (7.1)
where N s is the number of all sensory streams. Again, the sensory inputs

s = [s 1 , • • • , s N s ]
∈ S only depend on the current motor configuration m ∈ M of the agent and the current state ε ∈ E of the environment through the so-called sensorimotor law Ψ so that

s = Ψ ε (m). (7.2)
Then, it is assumed that the agent is able to perform direct comparisons between the sensory inputs inside each stream. This comparison is obtained from the comparison operation δ p for the sensory streams p ∈ [1, N s ]. In this work, we will consider that the the sensory comparison is the equality operator so that, for any sensory inputs s p , s p ∈ S p in sensory stream p, we have

δ p (s p , s p ) =    1 if s p = s p , 0 otherwise . (7.3)
Then, at each step of the refinement process, the agent explores a set of N exp motor configurations M = {m i } i∈[1,N] called the motor exploration set. At the kth-repetition, at motor configuration m i , the agent has access to the sensory inputs s

i [k] = [s 1 i [k], • • • , s N s i [k]] ∈ S.
p ij = 1 n n ∑ k=1 δ p (s p i [k], s p j [k]).
(7.4)

For each sensor stream p, the computation of sensory dissimilarities gives rise to the internal representation spaces (M/ D p , D p * ). Moreover, similarly to the evaluation of the internal representation presented in the previous chapter, it is possible to evaluate the preservation of local structures in-between the metric spaces obtained by each sensor stream. This is done by computing the local similarity criterion C 2 introduced in the previous chapter at §6.2.3. So that at repetition n, the agent can obtain measures of similarity between the internal representation of different sensory streams: C 2 (D p 1 * , D p 2 * ) for each pair of sensor streams p 1 , p 2 ∈ [1, N s ]. The resulting similarity is high when the sensor streams give similar local structures and low when they represent different topological spaces. This allows the agent to separate sensels streams controlled by different actuators but also to obtain the topological structure of sensels arranged in rigid arrays such as in camera, or range finder sensors.

Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

Justification

When two similar sensors or sensels, such as two pixels of a camera, are placed nearby on an end-effector, the statistics of their sensory inputs are generally similar. Indeed, if their receptive fields are near to each other, i.e. the portion of space where they are sensitive to physical changes, then they generally generate sensory inputs that are highly correlated. This property has been used for the calibration of camera sensors as an example for recovering the topological and metrical structure of the pixels of a camera [START_REF] Grossmann | Discrete camera calibration from pixel streams[END_REF]Orabona, 2010, Censi and[START_REF] Censi | Calibration by correlation using metric embedding from nonmetric similarities[END_REF].

To get back to the formalization in the previous chapter, two sensels p 1 and p 2 have poses x p 1 ∈ X p 1 and x p 2 ∈ X p 2 which are linked to the motor configuration space M through the forward models f p 1 and f p 2 .

For sensel 1:

M f p 1 -→ X p 1 , (7.5) For sensel 2: M f p 2 -→ X p 2 . (7.6)
Now, in the case where the two sensors are identical (two pixels, two range finders, etc.), arranged in a rigid array and close to each other, then the forward models lead to very similar pose spaces. Hereafter, the motor configurations leading to close poses in X / p 1 should also lead to close poses in X p 2 . Then by assuming that the sensory invariants statistics vary smoothly in the pose spaces, i.e. Hypothesis (H5), the internal representations obtained for both sensors should be locally very similar. On the opposite, if the two sensors have different pose spaces structures then, local properties cannot be preserved between them so that the sensory dissimilarities lead to different local structures and thus the agent can separate them based on the value of their local similarity criterion. Let's introduce an example serving as proof of concept for the reconstruction of sensors structure. The illustrated agent is shown in Figure 7.1. The agent is composed of two 2 DoF serial arms with four revolute joints controlled by four actuators with joint angles m 1 , m 2 , m 3 , m 4 ∈]π, π] forming two kinematics chains. At the end of each kinematic chain is a rigid array of sensors which is rigidly fixed so that the orientation of the sensors array changes with the orientation of the last part of each arm. The configurations (m 1 , m 2 ) controls the camera 1 which is a rigid array of identical sensors from s 1 to s 9 and (m 3 , m 4 ) controls the camera 2 from s 10 to s 34 . Each arm has lengths [0.5, 0.5] and the camera are both squares of size 0.5 so that camera 2 has a denser set of pixels. Each camera is centered at the end-effector point of their corresponding serial arm. At each repetition, the agent naively explores a set M of N exp = 200 motor configurations taken uniformly in M = [-π, π] 4 .

Proof of concept example

The environment is the straight lines environment with a distribution of straight lines that homogeneously covers the working space of both agent. Therefore each pixel will send binary values 1 of 0 according to the color in Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

the background environment at their poses. Because sensory inputs are considered to be symbols by the agent, it does not matter if they send a 0 or a 1 for a "black" background, indeed the sensory invariants are independent of the specific encoding of the sensory inputs as long as they stay invariant. Each environmental state is chosen randomly and independently from the distribution after each full exploration of M. The results of the pairwise comparisons is shown in Figure 7.2. The local similarity criterion shows that the internal representations obtained from the pixels of the first camera are very similar, with a minimum local similarity criterion of C 2 = 0.6 for the farthest pixels in the camera (p = 1 and p = 9). However, the similarity between pixels from camera 1 with pixels from the other camera are almost equal to 0. The same occurs for camera 2. The agent can clearly separate the sensory streams from both cameras based on the local similarities of their internal representation. Additionally, this separation could be done automatically with techniques from spectral clustering, as an example by using the Laplacian Eigenmaps [START_REF] Belkin | Laplacian eigenmaps for dimensionality reduction and data representation[END_REF]. Now, if we assume this separation is done, then we can project the similarities obtained for each camera into a low dimensional space for visualization. The results of the projection in 2D are shown in Figure 7.3, the projection is performed using Classical Multidimensional Scaling [START_REF] Torgerson | Theory and methods of scaling[END_REF] and the numerical implementation in MATLAB ver. R2018b whose details can be found in [START_REF] Seber | Dimension Reduction and Ordination[END_REF]. The projected results show a great preservation of the structures of the arrays of sensors in each camera, especially for pixels alignments. However, some distortion is still present, this is mainly due to the agent's architecture. Indeed the camera pixels at the bottom of each camera are always moving closer to the center so that they travel less distance than the pixels placed at the top of the cameras. As they travel less distance, their

Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

neighbor configurations are less perturbed which leads to a better local structure similarity. Therefore we can observe a slight increase in dissimilarity as we consider pixels that are generally farther from the center of the agent's working space (from left to right in Figure 7.3).

Discussion

It is interesting to notice that there is an analogy between the perceived internal representation in sensors' structures and the topographic maps in the brain. Topographic maps are projection of a sensory system, such as the retina (retinotopy), the cochlea (tonotopy) or the skin (somatotopy) to the cerebral cortex that preserves the topographic organization of the sensors. In this context we can see that the distortion is inherent in topographic representations as sensors are generally not distributed uniformly and can have different sensitivities. Indeed, perceptual distortion can be seen in somatotopical maps where different innervation densities on the skin are represented in the cortex by areas of different size [START_REF] Penfield | Somatic motor and sensory representation in the cerebral cortex of man as studied by electrical stimulation[END_REF]. In the context of topographic mapping, there is no requirement of accurate geometrical calibration of the sensors. Hence, biological organisms seem to not require metrical information, instead it seems that they just need an internal representation that preserves local structures of the sensory system. Hereafter, a vast majority of classical perception algorithm actually require accurate metrical calibration to reach good performance. As an example, metrical calibration of cameras is required for 3D reconstruction, motion estimation or augmented reality. This demonstrates one the fundamental differences between classical approach of perception in robotics and sensorimotor approaches that are more biologically related: sensorimotor representations are not metrically accurate when sensory spastics are not uniform in space [START_REF] Censi | Calibration by correlation using metric embedding from nonmetric similarities[END_REF]. Hence, the fact that biological perceptive system works pretty well implies that topographic information should be sufficient for cognitive systems to correctly operate in the world.

Conclusion

When the agent's has access to different sensory streams, the agent can obtain internal representation for each of these sensory stream. Then, we have 7.2. Self-interaction: the discovery of the self 169 proposed a way to represent the topographic structure of the sensors by computing local similarities between each pair of internal representations. If a set of sensor streams correspond to a rigid arrangement of sensels then, under the topological coherence assumption, the similarity between the sensory statistics directly represents the spatial proximity of the sensels. Moreover, if the sensels belong to sensors placed on different kinematics chains, they can be separated because of their difference in statistics. Then, the agent can obtain a specific internal representation for each rigid sensor, as a grouping of similar sensels and the map from the motor configuration space to each internal representation can be interpreted as a specific "sensitive" forward model for each rigid sensors. Therefore, this approach can be useful to discover the agent's kinematic structure and is a step forward for the construction of the agent's body schema in a strongly unsupervised setting Hoffmann et al., 2010.

Self-interaction: the discovery of the self

The agent is embodied in the environment, so it possesses a body that can possibly be perceived during the agent's motor explorations. A common idea in a lot of development robotics works is to make an agent that is able to distinguish the self from the environment. Usually, the distinction between the body and the environment can be based on the correlation from voluntary actions and sensory changes [START_REF] Asada | Cooperative behavior acquisition for mobile robots in dynamically changing real worlds via vision-based reinforcement learning and development[END_REF], multi-modal invariance with an example with vision and proprioception [START_REF] Yoshikawa | Does the invariance in multi-modalities represent the body scheme?-a case study with vision and proprioception[END_REF], prediction of the consequences of own actions based on internal models [START_REF] Schillaci | Is that me?: sensorimotor learning and selfother distinction in robotics[END_REF], temporal contingencies [START_REF] Stoytchev | Self-detection in robots: a method based on detecting temporal contingencies[END_REF] to cite a few. Anyway, from all these examples, the distinction between selfbased and environmental-based sensory changes is rooted in the assumption that the agent's interaction with its own body is invariant to environmental changes. In the current approach, we are very limited in the interpretation of sensory inputs, we don't have access to modality other than the separation proprioception/exteroception, but more importantly we don't have access to any meaningful sensory interpretation such has feature extraction, optical flow, or the timing of sensory events. However, we shall see that it doesn't require the separation of modalities, e.g. vision versus touch, to be able to Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

distinguish self-interaction. Instead, we can replace the concept of modality separation with separation of sensory streams.

The idea

The idea for the distinction between the body interaction and the environmental interaction is the following. Let's assume that they are motor configurations where a set of sensels reacts uniquely to the physical properties of the agent's body and other motor configurations where the same set sensels reacts uniquely to the environmental physical properties. Then, in between, there should be configurations where a part of the sensels reacts to the environment and the other part reacts to the body. In these configurations, the agent interacts partially with its body. Then, at the motor configurations that lead to this partial interaction, the structure of the internal representation obtained with the comparison operator on the set of sensels differs from the structure obtained with set of sensels that fully interact with the body.

Indeed, the internal representation of one sensel is a smooth representation of the sensel spatial configurations when it interacts with the environment, i.e. it is "smoothed" by the environmental changes. Moreover, when the sensel interacts with the body, the sensory invariants does not change with the environment and they depend on the fixed "texture" of the sensory invariants of the body. However, the sensory invariants on the body cannot be directly distinguished from intrinsic sensory invariants such as forward kinematics invariants or sensor symmetries. In order to perform this distinction, we need to consider the other sensels. Indeed, when the considered sensel interacts with the body, other sensels can still be sensitive to the environment changes so that the internal representation for the full set of sensors is still "smoothed" by the environmental changes. Hence, that local structure for the full internal representation is representative of spatial continuity while the local structure of the considered sensel is representing the sensory texture of the body. Thus, for these motor configurations, there is a local discrepancy between the internal representations, i.e. a change in neighborhood structures. By measuring these local differences, we can obtain the set of motor configurations where the considered sensel interacts with the body.

The approach

As in the previous section, the only interpretation the agent will be able to make comes from the local structures in its internal representations. As previously, we assume that the agent is able to make commands that acts on its motor configuration space M and it receives an array of sensory inputs s ∈ S Once computed, the neighborhood similarities C 3 represent the similarity between neighborhood structures obtained from a single sensory stream compared to the neighborhood structure obtained for the whole sensory stream. Furthermore, for a motor configuration m i , and a sensory stream p, a small value of C 3 represent a discrepancy between the local structure of the internal representation obtained for p and the local structure of the internal representation of the full sensors so that it is likely that p is interacting with the body. Otherwise, a high value of C 3 indicates that the local structure obtained with sensory stream p is similar to the full sensor representation which indicates an interaction of p with the environment. Let us illustrate these considerations with an example serving as proof of concept.

Proof of concept example for the discovery of self-interaction

Simulation's set-up Let's consider the planar agent with a 2 DoF serial arm with 2 limbs of same length 0.5, endowed with camera with 7 × 7 grid of pixels of full size 0.2 × 0.2 which is similar to the one shown in Figure 7.4(c). The camera is rigidly attached at its center to the position of the agent end-effector so that it rotates with the arm movements as shown in camera is sensitive to the background illumination at the exact pixel center (the black points in Figure 7.4(c)). The agent is assumed to have a simple body which is a fixed region of the background the agent's working space with a fixed black color.

In this simple case, the body can be assimilated to a static background. Indeed, in this approach the distinction between a body and a static background is not possible as the agent is not able to perform locomotion. The body is constructed as a black punctured square, as shown in Figure 7.4(a), of side 1 × 1 with a hollow interior of radius 0.3.

The agent explores a set M of N exp = 2000 motor configurations taken randomly in a uniform in the space [-π, π] 2 . The environment is taken to be the usual straight lines environment with homogeneous distribution in the 2D working space of the agent and after each cycle of exploration a new environmental state is chosen, randomly and independently, in the homogeneous distribution. The straight lines separate the 2D working space of the agent into a black and a white areas such that the sensory inputs the binary black and white values as shown in SubFigure 7.4(b).

When the agent changes its motor configurations m ∈ M, the sensel p travels through specific poses x p = f p (m) ∈ X p , which are 2D positions in the working space of the agent. The N exp = 2000 explored poses are shown in Figure 7.5(a).

The agent receives a sensory stream for each camera pixel, numerated as seen in Figure 7. where δ p and δ I are the comparison operators for a single and the whole sensory stream computed respectively with Equation (7.3) and Equation (7.7).

Then for a chosen sensory stream p and an a priori chosen neighborhood parameter K, the agent can compute the neighborhoods similarities C 3 at each explored motor configuration m i ∈ M according to Algorithm 2.

Local interpretation of structure discrepancies

First, we have chosen pixel 25 for our demonstrations. By removing the zeros in the sensory dissimilarity matrix D 25 , the agent obtains the quotient sensory dissimilarity matrix D * 25 and following Algorithm 2, it computes the corresponding quotient matrix D * {1,••• ,49} .

In Figure 7.6 is shown the value of dissimilarities for both pixel 25 and the full set of pixels obtained for the cases where the pixel 25 interacts with the environment (Figure 7.6(a)) and interacts with the body (Figure 7.6(b)). For the case where the pixel interacts with the environment, the perceived distances around the reference pose in red are quite similar and vary smoothly in space (similar smooth shading of blue as distance increases), i.e. the local structure is very similar as confirmed by a neighborhood similarity C 3 of 0.9 for a 10-neighborhood which means that 9 out of the 10 neighbors are the discovery of the self.

(a) C 3 = 0.9 with K = 10 same. However, for the case where the agent interacts with the body, the sensory dissimilarities are quite different around the reference pose. Indeed, for the sensory dissimilarity of pixel 25, the poses on the body are all situated at a perceive distance equal to 0 (dark blue) while the sensory dissimilarity of the whole set of pixels has been "smoothed" by the pixels interacting with the environment (smooth gradation from dark to light blue around the reference pose), so the neighborhood similarity C 3 for K = 10 is equal to 0 which means that the first 10 neighbors are totally different.

Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

Conclusion

In this chapter, we have proposed different ways for the agent to built some higher level interpretations on its interaction with its environment in the case where the sensory inputs are separated in different sensor streams. Namely, the agent can discover the structure of its sensors via the computation of similarities between the internal representations obtained by each individual sensory stream. Based on this technique, it has been shown in a simulated interaction, that the agent is able to separate the pixels from two different cameras and build a topographic representation of the spatial arrangement of the pixels. Later, this results could be used to separate kinematics chains in order for the agent to build a representation of body schema and its peripersonal space.

Then, when the agent is physically embodied in the environment and able to interact with its own body, we have presented an approach that enables the agent to distinguish when its sensors are in interaction with its body. This is done by exploiting the local discrepancies between the internal representation obtained for the whole set of sensors and the internal representation obtained from a single sensory stream. We have proposed an algorithm to compute these local differences via neighborhood similarities C 3 at each explored motor configuration. Then, we obtained some convincing results in a simulated simple environment. Indeed, the agent correctly represents when a particular pixel is sensitive to the agent's body. These results can be used later as a tool for the agent to build a representation of its body image (Head and Holmes, 1911 pp. 186-189). Indeed, it should be able to internally represent the topological shape of the body as well as the sensory textures on it.

From our perspective, all the proposed results, while obtained in very simple environment simulations, can be reproduced in more challenging real life experiment thanks to the low level of prior required. Indeed, no prior have been required on the sensor apparatus neither on the kinematics structure, the agent can be a 18 DoF snake with tactile, audio or visual sensors the current approach is perfectly applicable. The main problem would be the difficulties of the built internal space as is it difficult to formalize all the sensory invariance. However, thanks to the last two proposed application, such internal representation can be split and organized in a much simpler way.

Once the internal representations and self-interaction information has been extracted from the refinement, further work could focus on designing intrinsically motivated sensorimotor tasks defined in the internal representation, such as reaching or predicting specific sensory inputs at certain configurations [START_REF] Oudeyer | Intrinsically motivated learning of real-world sensorimotor skills with developmental constraints[END_REF].

Finally, the internal representations are only valid on a finite number of points.

It would be interesting to extend the finite internal representation from a discrete explored motor configuration space to the full manifold of motor configurations.

Conclusion

The proposed work aimed at studying the emergence of an perception for a robotic agent interaction with an environment. The taken approach follows a constructivism path inspired by the development of cognitive processes in infant from Jean Piaget (Piaget, 1937) and recent developments in the sensorimotor approaches of perception by K. J. O'Regan (J K O' Regan and Noë, 2001). In its theoretical developments, it extends the concepts of internal representations of sensor's spatial configurations proposed by A. [START_REF] Laflaquière | Approche sensorimotrice de la perception de l'espace pour la robotique autonome[END_REF].

The taken approach is based on the study of a naive agent, with inputs from a sensorimotor flow constructed with proprioceptive and exteroceptive sensors, in interaction with its environment. In order for this agent to be the more general as possible, only basic interpretations has been assumed on the sensorimotor inputs. Indeed, the agent has only be able to make interpretations about the equality or difference on the inputed elements.

Facing the conceptual difficulties of interpretation of the possible representations obtained by such agent, an theoretical framework has been developed in a first part. This commitment to interpretation is in opposition with the current trend of "black box" learning algorithms, such as deep learning, where the obtained representations can hardly be interpreted from a human point of view. Indeed, the approach is part of the field of developmental robotics where cognitive processes are obtained from progressively richer concepts, hence the importance of providing an understandable grounding for higher levels considerations.

In this theoretical framework, the agent is able to obtain a hierarchical refinement of previously uninterpreted motor configuration inputs based on the sensory invariants obtained during its successive environment interactions. This refinement has been characterized theoretically and its limits have been Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

studied. The limit of the refinement provides an interesting concept of perceptive points represented by the sets of motor configuration that can never be distinguished from a sensory input. Thanks to this, we can already understand the principle that "what can't be perceived can't be represented".

Then, in the context of a dynamical interaction with the environment, it has been shown that, under some explicit assumptions, the agent can theoretically represent more than just points. Indeed, from statistical measurements obtained by repeating the same set of motor configurations, it has been shown that a structured representation can emerge from statistics of sensory invariants. However, the hypotheses are not yet sufficient to obtain physically accurate representations.

The next step has been to explicit the sufficient hypotheses, in the field of topology, that are required to obtain an internal representation that respects and preserves the physical laws of continuity in space. These hypotheses being satisfied, the agent theoretically obtains a first real step towards the physical reality of its internal representation. Indeed, now, the internal representation and the physical space share a common property, we can then say that the agent perceives continuity in space. From there, the theoretical considerations stops and the focus switches to building an experimental realistic framework for the validation of previously introduced results.

Hence, a realistic agent is necessarily limited in its explorations. Thus, its internal representation is reduced to a finite sets of points and finite number of interactions with the environment. However, it has been shown, through multiple examples, that after a sufficient number of interactions, the internal representation obtained through the experimental refinement process can still be accurate. The topological properties being adapted to the preservation of local structures for the spatial configurations of the agent's unknown sensors. However, some challenging environments may introduce distortions between the real physical structure and the internal representation. Future works shall extend the proposed refinement framework to obtain more robust representations before seeking any real robotic application. Due to its fundamental nature and the limited motor exploration, the proposed refinement process lacks direct useful applications in real robotics. Indeed, usual

Conclusion

185 tasks such as navigation, cannot be proposed to the agent because of its limited spatial knowledge. However, following the hierarchical progression of cognitive processes, the agent can still obtain higher level representations from the refinement.

Consequently, in a last chapter, by simply assuming the separation of sensory inputs into different sensory streams from different sensor elements, the agent has the ability to build stronger structures. Thus, by exploiting the similarities between the internal representations obtained from each sensory stream, the agent obtain measures of similarity between different sensels. This is the first step towards the representation of sensors geometries and the discovery of the kinematics chains. When exploited under the correct assumptions, the proposed approach can thus lead the agent towards the discovery of a body schema which is one of the main goals in developmental robotics. Another main goal of cognitive development, is the discovery of the self. In the last chapter, and again just from the similarities between the internal representations obtained from different sensory streams, it has been shown that the agent has the ability of making a distinctions between a self-interaction and an interaction with the environment. This ability is also a step forward to the discovery of the self and body image.

Thanks to the introduced methods and under the explicit assumptions, the naive agent, from almost no a priori information about itself or the world, can obtain an internal structured, representation of its sensors spatial configurations. Additionally, this representation can also be split and organized in topographical representations of the sensors apparatus and possibly kinematic chains. Finally, self-interaction can be represented inside the previously obtained representation.

The most obvious future works should tend towards the full notion of body schema and peripersonal space because of their natural extension of the proposed applications. Furthermore, the agent has been limited to being fixed in the world and it can't perform locomotion. A other extension can be obtained by allowing locomotion abilities. Indeed, these could allow the agent to perform controlled changes of the environment and thus obtain more robust internal representations. However, the separation between postural actions and actions of locomotion is not as obvious as its seems, in this limited

  FIGURE 1.1: Model of the classical paradigm of perception in robotics.

  FIGURE 1.2: Model of the sensorimotor paradigm of perception in robotics.

FIGURE 1 . 3 :

 13 FIGURE 1.3: Strict sensory compensation of rigid motions can give rise to an internal, kinesthetic, geometrical representation of space. Figure extracted, with the kind permission of the authors, from Gas and Argentieri, 2016.

FIGURE 2 . 1 :

 21 FIGURE 2.1: View from the top of a simple planar agent represented in the 2D Euclidean plane. The limit of the reachable set of positions is represented by the dashed line. In blue are the two segments of the articulated arm and its revolute joints and in red a single-pixel camera.

FIGURE 2

 2 FIGURE 2.4: Illustration of the different types of sensory invariance. (a) In a fixed environment, when different regions of space make the sensor generates identical sensory inputs. (b) When different motor configurations always gives the same position and orientation of the sensor in the physical space (here we have added a 3rd degree of freedom to the serial agent for the illustration). (c) When different sensor's spatial configurations always generates the same sensory inputs independently to the environmental state.

FIGURE 2

 2 FIGURE 2.6: LEFT: the composition of environmental states ε 1 and ε 2 shown with different colors for the different possible sequences of sensory inputs. RIGHT: the corresponding sensory invariants represented in the motor configuration space for the sequence of the two environmental states.

FIGURE 2

 2 FIGURE 2.8: TOP: three straight line environmental states ε 1 , ε 2 and ε 3 separating the plane into a 'black' and a 'white' areas. BOTTOM: The corresponding sensorimotor states.

FIGURE 3

 3 FIGURE 3.1: Parametrization of a straight line by its podal point.

  FIGURE 3.2: Three environments with three different distributions of the straight lines on the disk D and their podal points distribution with 1000 environmental states samples. TOP: environment 1 gives a heterogeneous covering of the working space by the straight lines, environment 2 gives a homogeneous covering of the disk D and environment 3 is a homogeneous covering of the working space but with only vertical straight lines. BOTTOM: the podal points are distributed homogeneously on the disk in environment 1, heterogeneously on the disk for environment 2 and homogeneously on the horizontal line in environment 3.

Chapter 3 .

 3 Stochasticity on the refinement process P-measure 0. This allows us to define a new equivalence relation = P as m = P m if and only if

  FIGURE 3.3: The set of positions of the podal points in D of the straight lines that separates the poses f (m) and f (m ).

FIGURE

  FIGURE 3.5: Environment 2

FIGURE 3 . 7 :

 37 FIGURE 3.7: The set of positions of the podal points that separates the poses f (m)and f (m ) when they are vertically aligned.

  .7) where m and m are any representative of the respective equivalence classes [m] σ n and [m ] σ n . One can verify that relation (4.7) is well-defined as changing the representatives of the equivalence classes is a transformation inside the null-space of the sensory dissimilarity function σ n . Then the tuple (M/ σ n , σ * n ) is the current motor quotient space.

  (a) d(x, y) ≥ 0 (non-negativity). (b) d(x, y) = 0 if and only if x = y (identity of indiscernible) (c) d(x, y) = d(y, x) (symmetry).

  (d) d(x, z) ≤ d(x, y) + d(y, z) (triangle inequality).

FIGURE 4 . 1 :

 41 FIGURE 4.1: Emergence of the empirical structure in the binary straight lines environment. The agent explores 3 times its motor configuration space M and computes the sensory invariants (second column) with randomly chosen straight lines at each exploration (first column). The empirical structure is a finite discrete metric space which is represented with a fully-connected graph weighted by the sensory dissimilarity between the sensory equivalence classes (third column). On the last column is shown the corresponding sensory equivalence classes in the pose space.

  .13) where A(m, m ) is the region of the separating podal points as seen in Figure 4.3. The region A(m, m ) corresponds to the symmetric difference of the two disks of respective diameter O f (m) and O f (m ).

  FIGURE 4.4: Distortion between the open ball neighborhoods computed from the asymptotic sensory dissimilarity 1 σ * and the Euclidean distance d, at different poses x.

FIGURE 5 . 1 :

 51 FIGURE 5.1: The 2 DoF planar agent and its natural topological structure. LEFT: the 2 DoF serial agent with 2 controlled arms of same length and the single pixel sensor at the end. MIDDLE: the motor configuration set with angles from ]-π, π].RIGHT: the motor configuration space with its natural topological structure in the form of a torus.

5. 5 .

 5 The representation of physical continuity in the empirical structure. 111 Internal point of view, asymptotic empirical topological structures External point of view, natural topological structures

  [x] P , [x ] P ∈ X / P , p * ([x] P , [x ] P ) = p(x, x ), for any x ∈ [x] P and x ∈ [x ] P .(5.18)Proposition 17. The map p * is a metric in X / P .

FIGURE 5 . 3 :

 53 FIGURE 5.3: Illustration of the internal representations obtained with 4 different environments.

  FIGURE 6.3: TOP: evolution of the refinement criterion C 1 and the local similarity criterion C 2 for (straight lines) environment. The vertical line shows at which repetition the finest refinement (C 1 = 0) is obtained. BOTTOM: visualization of criterion C 2 with the corresponding internal representations projected in 2D using Isomap with K max -NNs during the agent's life.

  2 and C 1 , let's visualize the 2D projection of the obtained dissimilarity matrix along the agent's life using Isomap. The results of the visualization for the different environments are shown at the bottom of Figure 6.3. The internal representation starts with a few number of distinct points. After 20 repetitions, the number of points in the representation has increased, and the structural cues starts shaping with C 2 = 0.4. They are visually well preserved starting from C 2 = 0.6 and 50 repetitions. After the almost convergence of C 2 around 10 4 repetitions, the internal representation is visually a quasi-perfect reproduction of the working space which is confirmed by a C 2 close to 1.

FIGURE 6 . 4 :

 64 FIGURE 6.4: Scenario 1: (blobs), an environment made of black and white blobs slightly deformed at a 45 • orientation.

  FIGURE 6.5: TOP: evolution of the refinement criterion C 1 and the local similarity criterion C 2 for the (movie) environment. The vertical line shows at which repetition the finest refinement (C 1 = 0) is obtained. BOTTOM: visualization of criterion C 2 with the corresponding internal representations projected in 2D using Isomap with K max -NNs during the agent's life.

FIGURE 6

 6 FIGURE 6.6: Scenario 2: (movie), an environment made of grayscale background frames from the movie 'Phantom of the Opera' (1943). The agent is allow to explore n-Motor configurations Per Frame (mpf) so that the environmental states change during the exploration part of the refinement process.

  FIGURE 6.7: TOP: evolution of the refinement criterion C 1 and the local similarity criterion C 2 for the (blobs) environment. The vertical line shows at which repetition the finest refinement (C 1 = 0) is obtained. BOTTOM: final values of C 2 with the projected internal representations for 3 different dynamics (mpf = movements per frame) of the movie environments. The 3 final representations are obtained after 10 4 , 5700 and 960 repetitions respectively.

FIGURE 6 . 8 :

 68 FIGURE 6.8: The 2DoF planar serial agent with a multi-pixels camera in the straight lines environment with spatially homogeneous distribution.

  FIGURE 6.10: Evolution of the local similarity criterion C 2 and the refinement criterion C 1 for the comparison of the internal representation with the pose space with metric space (X, R 0.09 ). The vertical dashed line correspond to the repetition at which maximum refinement is obtained.

  Let's compute the neighbor graph representations for the internal representation (M/ D , D * ) and the represented spaces (X, R γ ) for multiple values of γ. The value of k for the k-NNG graphs is computed with Algorithm 1. The results are shown in

  FIGURE 6.11: Path planning results on the neighbor graphs of the internal representation and multiple evaluation metrics.

FIGURE 7 . 1 :

 71 FIGURE 7.1: An agent with multiple sensor streams from camera sensors and multiple forward models.

  FIGURE 7.2: Local similarity criterion C 2 between each sensory stream internal representation after N rep = 1000 exploration repetitions.

FIGURE 7 . 3 :

 73 FIGURE 7.3: The original pixel structures and the 2D projected perceived similarities between pixel streams after N rep = 1000 exploration repetitions

Algorithm 2

 2 1 × • • • × S p × • • • × S N sgenerated from multiple sensors which are linked to the environmental states ε ∈ E through the sensorimotor law s = Ψ ε (m). The agent is also able to perform sensory comparisons trough sensory comparison operator δ p on each sensor stream. Then, following the refinement process, the agent explores a set M of motor configurations and the exploration is repeated until convergence of the internal representations (M/ D p , D p * ) where D p * is the sensory dissimilarity for sensor stream p on the motor configuration equivalences classes in M/ D p . The agent can also build an internal representation for a set of sensors, i.e. it can create a sensory comparison operation δ I where I ⊆ {1, • • • , N s } is a set of chosen sensor streams such that, for two sensory inputs s, s ∈ S,with s = [s 1 , • • • , s N s ] and s = [s 1 , • • • , s N s ], δ(s p , s p ) = 1 for all p ∈ I, agent can also build an internal representation, through the refinement process, based on the statistics of sensory comparisons obtained with the comparison operator δ I . A particular case is whenI = {1, • • • , N s },where the agent builds an internal representation of the whole set of sensors 'sensitive' poses spaces.Hence, once the refinement process has converged, the agent obtain the dissimilarities matrices D p for each sensor stream p ∈ I with comparison operator δ p as well as the sensory dissimilarity for all the sensors as D I where I = {1, • • • , N s } obtained from comparison operator δ I . Then, for each motor configuration m i inside the motor exploration set M, the agent can compare the local structures between the different internal representations. A local structure is just the set of neighbors of m i obtained from the sensory dissimilarity distance from m i to every motor configurations and the comparison Chapter 7. Some applications: sensors topographic structure and the discovery of the self. Neighborhood similarities C 3 between sensory stream p and full stream I Require: sensory dissimilarity matrices D p and D I of sizes N exp × N exp and K the size of the neighborhoodsStep 1: label the connected components for the equivalence classes in D p 1: initialize the graph G = ({1, • • • , N exp }, {∅}) 2: for all elements of coordinates i, j ∈ [1, N exp ] in D p do 3in G between vertices i and j 5:end if 6: end for C ← set of connected components in G %the connected components are the connected subgraphs, it can be a vertex. Each connected component of C is labeled with an index i * ∈ 1, |C| 7: for all vertex i in G do 8: L(i) ← i * %where i * is the label of the connected component in which vertex i belongs 9: end forStep 2: compute the quotient sensory dissimilarity matrices according to zeros in D p 10: initialize matrices D p * and D I * as matrices of size |C| × |C| 11: for all i * , j * in [1, |C|] i, j are any index such that L(i) = i * and L(j) = j * 13: D I * i * j * ← D I ij where i, j are any index such that L(i) = i * and L(j) = j * 14: end for 15: %the matrices D p * and D I * should have 0 only on their diagonalStep 3: compute the neighborhood similarities between the quotient matrices 16: for each row i * in D p * do 17:N p (i * ) ← index of the K-lowest values > 0 of row i * in D p * 18:N I (i * ) ← index of the K-lowest values > 0 of row i * in D I * 19:C * 3 (i * ) ← 1 K • N p (i * ) ∩ N I (i * ) 20: end forStep 4: paste the values of similarities for all motor configurations 21:for all i in [1, N exp ] do 22: C 3 (i) ← C * 3 (L(i)) 23:end for is made by looking at the common neighbors obtained from two different sensory dissimilarity matrices. The measures of local similarity between the internal representation is thus called the neighborhood similarities C 3 . These similarities are computed at each motor configuration m i ∈ M between an isolated sensory stream p and the full stream I. The computation details are shown in Algorithm 2. (a) the agent and its black body (shown in gray for illustration clarity) (b) sensory inputs for different motor configurations in a straight line environmental state (c) the actual 7 × 7 camera with pixels centers (black points) and their numeration.

FIGURE 7

 7 FIGURE 7.4: Agent with a camera and a 'punctured square' black body in the straight lines environment

  Figure 7.4(a) and (b). Each pixel of the Chapter 7. Some applications: sensors topographic structure and the discovery of the self. The N exp = 2000 explored poses for pixel 25 at the center of the camera in the 2D working space (b) the neighborhood similarities C 3 of pixel n • 25 and K = 10 represented in the pose space of pixel 25.

FIGURE 7 . 5 :

 75 FIGURE 7.5: Correspondence between the body and neighborhood similarities for pixel 25.

  5(c). At repetition k, pixel p sends a sensory input s p [k] to the agent. Then, as the refinement process converges, the agent computes the sensory dissimilarity matrices D p for each sensory stream p ∈ {1, • • • , 49} as well as D I for the whole sensory stream I = {1, • • • , 49} such that: s i [k], s j [k]), (7.9)

  FIGURE 7.6: Values of neighborhood similarity and sensory dissimilarities for single pixel 25 and the whole set of pixels {1, • • • , 49} from a reference pose (in red). In (a), the pixel 25 interacts with the body. In (b) the pixel 25 interacts with the environment. The sensory dissimilarities are represented in the set of poses of pixel 25.

  FIGURE 7.7: Neighborhood similarity C 3 between sensory stream 25 with the whole sensor stream in the motor configuration space. When the neighborhood similarity is low (black points), the motor configurations represents when pixel 25 interacts with the body.

  Towards a representation of sensitive sensor's spatial configurations 29

FIGURE 2.5: LEFT: two different environmental states ε 1 and ε 2 with different spatial sensory invariance. RIGHT: the corresponding theoretical sensorimotor state.

Proposition 1. Let

  's consider to environmental subsets E andE of E . If E ⊆ E then the quotient set M/ E is a refinement of M/ E , or equivalently M/ E ≤ M/ E . Proof. For any motor configuration m ∈ M, [m] E is the equivalence class of m in M/ E and [m] E its equivalence class in M/ E . Let's show that [m] E ⊆ [m] E . Itis clear from relation 2.4 that for any pairs m , m of [m] E we have that m = E m and equivalently ∀ε ∈ E , Ψ ε

	{1, 2}, {3, 4, 5}}
	and
	β = {{0, 1, 2}, {3, 4, 5}}
	are both partitions of the set {0, 1, 2, 3, 4, 5}, but α is also finer than β. Indeed,
	the subset {0, 1, 2} in β is refined into the two subsets {0}, {1, 2} in α and
	α ≤ β
	.
	The refinement process is the fact of refining the motor configuration set M
	of the agent into successive finer partitions along with new experienced en-
	vironments. The agent first interacts with a single environmental state ε and
	builds the quotient set M/ ε from the sensory invariants in this state, it is a
	partition of M. When the agent experiences a second environmental state
	ε , it can build the multiple environments quotient set M/ {ε,ε } , as defined
	in 2.7. It is intuitive that the partition M/ {ε,ε } should be finer than M/ {ε} as
	the more environmental states the agent explores the smaller are the config-
	uration invariants, as shown in the previous section §2.2. This proposition is
	shown true in a more general case.

  .15) Proof. At environmental state ε, the motor configuration m generates sensory input s = Ψ ε (m). Then the pose x = f (m) also generate the same sensory input s = Ψ ε (m) = φ ε ( f (m)). All the motor configurations that generates sensory input s form the equivalence class [m] ε . Then naturally, all the poses obtained from elements of [m] ε must generate the same sensory input s. So that m

2.19) Proposition 4. For any

  subset E of environmental states in E , f E is a bijection.

	Proof. For any k ∈ M/ E , let's take a representative m ∈ M of the equiva-
	lence class k

such that m ∈ k and k

  Euclidean distance. This guarantees that, asymptotically the set of sensory equivalence classes converges towards the set of reachable sensor positions in the Euclidean subspace D. Therefore, we can naturally replace the observable equivalence class [m] P 1 by the pose f (m) without loss of generality.

	(4.18)
	One can notice that, accordingly to the point made on observable fundamen-
	tal sets at §3.2.3,

).

1 σ * ([m] P 1 , [m ] P 1 ) = 0 if and only if 1 σ(m, m ) = 0, (4.19) if and only if d( f (m), f (m )) = 0. (4.20)

Asymptotically, two motor equivalence classes belongs to the same equivalence class if and only if their poses are at a 0

Table 6

 6

	.1.

TABLE 6 .

 6 1: Evolution of the local similarity criterion C 2 with the value of parameter γ of evaluation metric ρ γ in X.

  Table 6.2. metric space (M/ D , D * ) (X, R 0.001 ) (X, R 0.09 ) (X, R 10 )

	k	7	10	7	9
	TABLE 6.2: The minimum k values to obtain connected k-Nearest Neighbor
		Graphs for the different finite metric spaces.

  The main problem comes from the fact that the sensory inputs, possibly from different sensors based on different kinematics chains, are regrouped into a single symbol and all the sensors' configuration space are regrouped into a single space. While, the theoretical results are still valid in this case, the direct exploitation by the agent is not obvious. A solution to this problem comes by finding a way to separate the sensors kinematics. The extraction of the agent's kinematics from sensorimotor inputs is a field of research in itself, especially in developmental robotics, and is part of what is know as the learning of a body schema, see Hoffman et al. for a review on this subject in robotics

  For each sensory stream p ∈ [1, N s ] the agent can compute the sensory dissimilarity matrix D p between pairs of motor configurations in M. So, for any pair m i , m j ∈ M the elements D

	p ij are
	computed for repetition n such that
	D

For more details about the theory of quotient sets and equivalence relations, we advice the lecture of the book "Naive set theory" by P. R. Halmos[START_REF] Halmos | Naive Set Theory[END_REF].

uncountable set: a set that contains too many elements so that they can't be indexed with the set of natural numbers N = {1, 2,

3, • • • }

The selected movie is Phantom of theOpera (1943) which is in the public domain.

Remerciements

Empirical structure for environment 1

Asymptotic sensory dissimilarity for environment 1 FIGURE 4.2: Environment 1 with method 1. For each exploration, the straight line's podal point is chosen independently from a uniform distribution on the disk D.

The notation σ 1 being already taken, let's call 1 σ the asymptotic sensory dissimilarity for environment 1. For two motor configurations m and m , the

Results of the neighborhood similarity

Then, in Figure 7.5(b) is shown the computed values of neighbor similarities C 3 for all pixel 25 poses with a K = 10. The value of K has been chosen a priori but we believe that additional work on the matter could provide an automatic way to chose this value. We can see that the neighbor similarities represented in the pose space of pixel 25 follow the shape of body. Therefore we can say that the neighbor similarities successfully represent when the agent interacts with its own body.

A small value of K makes the neighbor similarity less robust to the various density of explored poses and too large values of K makes the neighbor similarity less representative of local structures as it takes into account far configurations. In this experiment, values of K between 5 to N exp /2 have provided very consistent results.

We can notice that the highest values are obtained on the edge of the working space. Indeed, for the poses of pixel 25 at the edge, the camera has a fixed orientation (the 2 DoF arm is at full extension) so that the sensory dissimilarity for the whole set of pixels does not depend on the rotation of the camera.

As it only depends on the positions, the sensory dissimilarity for the whole set of pixel is very similar to the sensory dissimilarity for the single pixel 25.

On the opposite, near the center, we can remark that the sensory dissimilarity for the whole set of sensors mainly depends on camera rotation (the arm is folded) which creates a local difference with the sensory dissimilarity of the pixel 25 which only depends on the camera's center position. However, the neighborhood similarity is still relatively high thanks to the preservation of spatial continuity.

From the agent's internal point of view, the agent directly obtains an internal representation of the interaction of sensory stream 25 with the body. This representation is shown in Figure 7.7 in the motor configuration space. This representation can be obtained for all the sensory stream individually but it is also possible to combine multiple sensory streams.

Conclusion

Chapter 7. Some applications: sensors topographic structure and the discovery of the self.

a priori context. Currently, this work is in progress. Furthermore, when locomotion is allowed, the agent has the possibility to compute compensatory transformations that are, according to Poincaré's intuition [START_REF] Poincaré | L'espace Et la Géométrie[END_REF], at the origin of our perception of the geometrical structure of space.

Appendix A

Annex -Proof of relation 4.23

If the straight lines are distributed uniformly in the planar region D then the probability to intersect a given line segment inside the disk is invariant to any translation or rotation of this line segment as long as it stays inside the disk. As an example in environment 2, if the poses f (m) and f (m ) are at distance d and the poses f (m ) and f (m ) also are at distance d then the probability to obtain a straight line separating both poses, and then making them generate a different sensory input, is the same for both pairs: P(E c (m,m ) ) = P(E c (m ,m ) ). Thus we have that the sensory dissimilarity σ 2 (m, m ) is function of the euclidean distance d( f (m), f (m )). Furthermore, if we increase the length between f (m ) and f (m ), let's say d( f (m), f (m )) = d + d , with d > 0, d > 0 and d + d < 1, then the probability for a straight line to intersect the line segment between them is the sum of the probability of intersecting the first part of length d plus the probability of intersecting the second half of length d . So that the sensory dissimilarity is an additive function of the distance d( f (m), f (m )) and when the euclidean distance is 0, the sensory dissimilarity must be 0 too as the poses are the same. The sensory dissimilarity is also a continuous function of the distance d( f (m), f (m )) as its a function of the area . increases, the sensory dissimilarity increases too as they are more straight lines intersecting the line segment, thus the sensory dissimilarity is monotone. Hence we can write σ 2 (m, m ) = ad( f (m), f (m )). To find the constant a, let's use a particular example. Consider the motor configurations m = [0, π] and m = [0, 0]. Their corresponding poses have the following cartesian coordinates: f (m) = (0, 0) and f (m ) = (0, 1).

Appendix A. Annex -Proof of relation 4. 23 FIGURE A.1: The set of podal points that separates f (m) = (0, 0) and f (m ) = (0, 1). This set correspond to the points inside the circle C of radius 1/2 and center (0, 1/2).

The set of podal points that separates poses (0, 0) and (0, 1) is given in Fig- ure A.1, it corresponds to the regions inside the circle C of radius 1 2 and center (0, 1 2 ). Let's compute the probability to obtain a podal points inside the circle C using the formula of P 2 given at (3.9). In polar coordinates, the equation of the circle C is given by: r = cos(θ), for θ ∈ [-π/2, π/2].

(A.1)

Hence the P 2 -measure of the podal points is the area of C in the polar plane such that