Frédéric Alexandre

A behavioral framework for a systemic view of brain modeling

Introduction

As stated by the French biologist and physician Henri Laborit [START_REF] Laborit | Eloge de la fuite[END_REF], the motivation of living beings is ... being, i.e. maintaining their organic structure. Whereas this statement is obvious for primitive animals, it is too often neglected when studying high level cognitive functions, particularly in humans. Most of the time in computational neuroscience, such cognitive functions are associated to specific regions of the cortex with no references to the bodily dimension or even to subcortical structures. Their characteristics are described as resulting from purely cortical dynamics, with no references to motivational or emotional groundings. Some authors and paradigmatic approaches have stressed the limitations of such corticocentric views, compared with a myopia [START_REF] Parvizi | Corticocentric myopia : old bias in new cognitive sciences[END_REF], minimizing the essential role of subcortical structures. The domain of embodied Artificial Intelligence has shown through robotic experiments [START_REF] Pfeifer | How the body shapes the way we think : a new view of intelligence[END_REF] that complex behaviors may result from elementary loops between sensors and actuators, exploiting the properties of the body instead of a complex representation of information. It is for example proposed in [START_REF] Taouali | A parsimonious computational model of visual target position encoding in the superior colliculus[END_REF] that the non homogeneous distribution of sensors in the retina can explain some visual target selection principles in a more parsimonious way than cortical mechanisms.

More fundamentally and more anchored in biology, enactivism, the theory of enaction [START_REF] Varela | The embodied mind : Cognitive science and human experience[END_REF], stresses principles like autonomy and ecological meaning of the behavior. In this theory, autonomous behavior is a central characteristic and is considered at different time scales. Fundamentally, a living being must choose on its own and at each moment the most adapted behavior and can only rely on previous learning (ontogeny) and on pre-established abilities (phylogeny). Ecological meaning refers to the motivational and emotional bases of behavior that have to be taken into account. Cats chase mice because they have such motivations, needs and goals -because they are cats.

In spite of their important role to define needs of the body and goals to be reached, the motivational and emotional dimensions of behavior are little studied in computational neuroscience and in cognitive science. Building on interoceptive information like visceral signals but also on somatosensory information (pain, pleasure, temperature), the insular cortex is reported to play a central role in defining motivations of the body to act, like feeding, breeding, preserving the integrity of the body (A. [START_REF] Craig | How do you feel -now ? the anterior insula and human awareness[END_REF]. Biologically significant events important for survival signaled by such interoceptive signals but also by sensory information (e.g. related to the perception of a predator or of social signals) can be associated by learning with other neutral events that will elicit emotions useful to anticipate the former ones and to detect goals to be pursued or avoided. [START_REF] Gros | Cognition and Emotion : Perspectives of a Closing Gap[END_REF] suggests a specific role of information of reduced complexity for emotions that can become conscious feelings, also described as mental experiences of body states [START_REF] Damasio | The nature of feelings : Evolutionary and neurobiological origins[END_REF]. Such a body of principles renders humans closer from animals. It also underlines the strong links between the brain, the body and the environment and, within the brain, is a strong motivation to consider loops instead of cortical regions in isolation and to consider the multiple learning mechanisms at work in these loops. At the functional level, this is also a plea for defining a global cognitive architecture in which any cognitive operation in consideration should be delineated. Decision making, planning, selective attention or perceptual identification shouldn't be studied, and models of the corresponding cerebral circuitry shouldn't be elaborated, without a reference to global plans relating cognition and the brain, seen as a whole and in relation to the body and the environment. Else, the risk 1 is to just study mechanisms apart from the rationale for their existence and consequently to forget some of their fundamental characteristics and resources.

In agreement with these considerations, we present here in a systemic view a general framework of brain modeling that has been elaborated in our team as a basis for several modeling studies of the brain, of learning mechanisms and of cognitive operations. It is also intended, for future specific studies of brain-inspired cognitive mechanisms, to serve as an outline in which each of these studies should be placed, for a better understanding of its contribution in general cognition and for consistency in this systemic view of cognition that we affirm here to be essential. This view is kept simple not only because of the reduced size of this paper but also because it is intended not to give a very detailed description of all the underlying phenomena (beyond our knowledge) but to set their main principles.

Three worlds to conciliate

The brain is facing three complex and dynamic worlds, on each of which it can sense information and it can act, possibly resulting on modifications in that and other worlds. We call these worlds the external environment, the extended body and the internal body.

The external environment corresponds to the external world, including objects subject to the laws of physics and beings subject to the laws of nature, possibly including intentionality. These agents exist in space and time and can be sensed by external sensors (i.e. seen, heared, touched, tasted or smelt), defining exteroception.

The extended body considers the body as an agent in the external environment, in which it may act. The extended body is composed of parts (e.g. limbs, head) carrying the external sensors. Their positions in space can be sensed by proprioception and can be modified by elementary actions and integrated motor programs (e.g. walking, grasping, speaking).

The internal body refers to all the machinery that makes the body work internally at the visceral, chemical, hormonal levels, i.e. eat, drink, breath, digest, etc. This defines the fundamental needs of a being, depending on internal states that can be sensed by interoception. Homeostatic mechanisms and other internal and external actions can modify these states.

As sketched in figure 1, the brain has consequently exteroceptive, proprioceptive and interoceptive sensors to get information about these worlds and their inner dynamics. It can act on them through a series of pro-cesses that we will call actions at large. These actions can be voluntary or involuntary and be applied to the extended or internal body, respectively corresponding to motions of parts of the body with skeletal muscles (e.g. speaking or blinking) and to the activation of endocrine or exocrine glands (e.g. releasing an hormone or crying), of smooth muscles and of the heart. Impacts of these actions in the three worlds can also be perceived by the sensors.

For both motor and sensory aspects, the central nervous system (including the brain) is connected to these worlds by the peripheral nervous system, including a somatic part (for exteroception and proprioception and for external actions) and a visceral part (also known as autonomic nervous system, for interoception and internal actions). The brain can be interpreted as an information processing system receiving signals from the three worlds and triggering actions impacting them. The role of the brain can be discussed with regard to the ways the signals are processed (building internal representations, convenient for subsequent processing) and the actions triggered.

Signals that are received can be used to trigger directly actions, based on their intrisic value or on their capacity to activate internal representations. They can also be used to modify internal representations or to create new ones, following several learning processes that will be described in the next sections. These internal representations correspond to different kinds of memories that might also participate to triggering actions at various moments. As we will also discuss below, the characteristics of these memories, directly linked to the information flows feeding them and the criteria governing the corresponding learning rules, will have a strong influence of the kind of information that will be represented in the brain and available for action selection.

In summary the processes for the transduction of signals into actions and for the elaboration of internal representations of information are based on the signals received from the three worlds, on the current state of the memories and on the architecture of the cerebral structures.

Before describing more technically these processes, it is fundamental to remind that they have been selected in an evolutionary scheme, particularly to enable living beings to maintain their structure, to optimize survival and reproduction. This sets a special emphasis on the internal body world that has been designed and complexified by evolution to represent special body states indicating critical situations and giving specific incentives for that aim. Particularly, the autonomic nervous Figure 1 -In an enactive view of cognition, the brain is not seen in isolation but in systemic inter-relation with the external environment and with internal states. This leads to the specification of two facets of the body. The extended body interacts with the external environment through proprioception and exteroception and through external actions. The internal body feels internal states by interoception and can trigger internal actions. Altogether, this defines the sensory and motor information flows of a brain+body system, acting and learning in the external environment to satisfy some fundamental needs, expressed as internal states.

system is divided in two parts, the parasympathetic system responsible for the feed and breed activities and the sympathetic system for the fight or flight activities. In a simplistic view, this can be categorized in a dichotomic way with positive (pleasant) situations and incentives to exploit them for the nourishment and the reproduction of the body, opposed to negative (unpleasant or painful) situations and incentives to get rid of them for the integrity of the body. As we will discuss later, though partly overlapping, it would be too simple to directly associate this dichotomy to rewards and punishments, since for example a lower punishment than expected can be seen as pleasant. This can be more nicely integrated in a two opponent-process system with mutual inhibition between two classes with opposite properties, acting against a baseline [START_REF] Dickinson | The Role of Learning in the Operation of Motivational Systems[END_REF].

Pavlovian and instrumental conditioning

Considering the brain, as sketched in figure 1, as a system integrating different kinds of sensations to decide for different kinds of actions, several mechanisms of increasing complexity have been aggregated to this system along evolution. A first set of mechanisms directly associated to emotional learning is related to pavlovian (or respondent) conditioning [START_REF] Balleine | Parallel incentive processing : an integrated view of amygdala function[END_REF]. Some biologically significant stimuli also called unconditional stimuli US (e.g. a predator or some food) can be automatically identified (without learning) and trigger unconditional responses UR (e.g. freezing or salivation). Pavlovian conditioning corresponds to learn that some initially neutral conditional stimuli CS (e.g. a tone or a light) announce the arrival of US. Subsequently, the occurence of CS can generate two phases of behavior. In the preparatory phase, the CS is associated to motivational properties of the US and to its valence (aversive or appetitive) and triggers non specific responses like arousal, heart rate increase and approach. In the consummatory phase, the CS is associated to the sensory properties of the US and triggers specific responses like chewing and blinking.

At this stage, it can be interpreted that pavlovian learning is a way to anticipate, upon CS arrival, the negative or positive characteristics of the US and to prepare the body to this inevitable event. In the pavlovian scheme, responses are stereotyped (also called pavlovian reflexes) and are consequences of the learned association. Several mechanisms have been described, inserting actions in the pavlovian process [START_REF] Balleine | Parallel incentive processing : an integrated view of amygdala function[END_REF]. In autoshaping, an action can be triggered to more easily get a CS. In pavlovian instrumental transfer (PIT), animals exposed to a CS associated to a US trigger more frequently the action that was learned to yield that US.

To more flexibly insert actions in the behavioral process, in another learning scheme called instrumental (or operant) conditioning, the learned association is between an action and the outcome observed to be the consequence of the action. An action will be triggered more (resp. less) frequently when observed to lead to a positive (resp. negative) consequence. Alternatively, an action can be triggered more frequently if it leads to the avoidance or removal of a negative stimulus, giving also here a motivation to act. The two opponent process system mentioned above is a good basis to consider these negative cases where, at the end, no reinforcement is given : the absence of negative outcome can be considered as a positive outcome, on which to build learning with so called conditioned inhibitors. It has also been associated to the representation of safety [START_REF] Genud-Gabai | Safety Signals in the Primate Amygdala[END_REF].

Instrumental conditioning can be performed under the control (or conditional to) stimuli also called occasion setters, that can become conditioned reinforcers [START_REF] Cardinal | Emotion and motivation : the role of the amygdala, ventral striatum, and prefrontal cortex[END_REF], leading to chaining in behavioral goal-directed sequences toward primary reinforcers. These associations can be restricted to habits through extensive learning, where the conditional stimuli directly elicit actions without references to the outcomes to be obtained.

In summary, it has been argued here that the behavior is basically organized around motivations to act imprinted in the nervous system. We have mainly evoked extrinsic motivations, to get a desired outcome satisfying fundamental needs. This includes integrity of the body, seen as a positive motivation in the framework of the two opponent process system. With the elaboration of more complex internal representations that will be described below, expressing intrinsic motivations [START_REF] Oudeyer | Intrinsic motivation systems for autonomous mental development[END_REF] like curiosity or attention toward novelty, related to more abstract needs of exploration of a complex world and of monitoring of internal activity, will be made possible.

Beyond the automatic triggering of stereotyped behaviors associated to motivations, we have also evoked two fundamental learning processes that elaborate on them. Pavlovian conditioning learns CS that anticipate the US and instrumental conditioning learns to select actions that best exploit them. These learnings are not easy because they are to be performed in three mutually dependent, changing and dynamic worlds, the external environment and the extended and internal body. Through evolution, a task of the nervous system has been to learn to build more and more complex and adapted representations of these worlds, as we report in the next section.

Encoding and representation of information

In this section, we propose a general framework of information representation in the brain and explain how this is associated to questions to be addressed in more and more complex ways along three main evolutionary steps and some corresponding learning paradigms.

Four questions to be addressed

We have explained above that one important task of the brain is to explore three different worlds and to learn their contingencies. Overall, the goal is to get information about 'objects' in these worlds and about ways to act on them. We sketch these pieces of information as answers to four fundamental questions : what, why, where and how.

The why and what questions are useful to relate the internal body and external environment. The what question is useful to encode a CS and its emotional impact. The CS can be seen as the current goal of the behavior (for example a bottle of water). It is consequently important to encode its physical characteristics (shape, color) to be associated to its emotional value (like, dislike). Several regions of the posterior ventral cortex have been reported to be selectively responsive to such physical characteristics [START_REF] Rousselet | How parallel is visual processing in the ventral path ?[END_REF]. The why question corresponds to the characteristics of the motivational impact on the body and is useful to encode the value of the US or the bodily cost of an action to get it. Consider for example the level of water deprivation or the intensity of a pain. Such information is represented in the posterior insular cortex (A. [START_REF] Craig | How do you feel -now ? the anterior insula and human awareness[END_REF]. It can motivate the behavior, explaining 'why' we act (for which purpose) and why we accept to spend our energy.

The where and how questions relate the external body and the external environment. Answering the where question provides information about the position of an 'object' and particularly with regard to (some parts of) the body. The how question refers to the need to learn how objects can be modified (e.g. moved, manipulated) by the action of some body parts. The posterior dorsal cortex has been reported to be involved in both functions [START_REF] Milner | The visual brain in action[END_REF], respectively in its superior and inferior parts.

These questions give the main ingredients of a simple behavior : We describe the goal of our action (what are its characteristics), explain why it is appropriate for our current motivation and localize it (where) for consumption (how). But of course, in the real world, things are not so easy. Several motivations and goals can be in competition. Their characteristics can be difficult to extract. Variable delays can exist between the main ingredients of the behavior (the US, CS and actions) and some abstract reasoning can be needed (the bottle of water is in the fridge and I have to find the kitchen beforehand). All these elements correspond to increasingly complex behaviors, made possible along evolution, as we now describe.

Elementary association to actions

Before describing how integrated behaviors can associate answers to the four questions to efficiently satisfy the needs, it must be underlined that in addition to regions of the posterior cortex that have been mentioned above as elaborating an internal representation in reference to these questions, key subcortical regions, existing in primitive animals without a cortex, can be associated to them, directly associating actions to similar (though more rough) representations of sensory information.

What -The amygdala : The amygdala is a set of structures with sensory and motor aspects, instead of an homogenous system [START_REF] Swanson | What is the amygdala ?[END_REF]. Among its nuclei [START_REF] Ledoux | The amygdala[END_REF], the lateral nucleus receives a wide spectrum of sensory inputs from the thalamus and the cortex and is generally reported as the place for storing CS-US associations. The central nucleus of the amygdala (CeA) is the main output region for the expression of innate emotional responses and related physiological responses, particularly in relation to the periaqueductal gray (PAG) and the lateral hypothalamus. Another major nucleus is the basal nucleus, particularly in charge of information exchange with higher level structures like the prefrontal cortex and the hippocampus [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF]. Neurons in this nucleus encode a variety of information for aversive and appetitive stimuli, related to the sensory nature of the US, to conditioned inhibitors and, for instrumental conditioning, related to conditioned reinforcers [START_REF] Belova | Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala[END_REF]. It also encodes the level of arousal, ambiguity and unpredictibility of information [START_REF] Pessoa | Emotion processing and the amygdala : from a 'low road' to 'many roads' of evaluating biological significance[END_REF].

Altogether, the lateral and basal nuclei, also called the basolateral complex (BLA), can consequently be considered as a primitive place for representing the valence and the value of emotional stimuli and a relay for more elaborated processing in the cortex, ventral striatum and hippocampus, in direct association with the central nucleus CeA responsible for the pavlovian response and also involved in more elaborated emotional responses [START_REF] Cardinal | Emotion and motivation : the role of the amygdala, ventral striatum, and prefrontal cortex[END_REF].

Why -The lateral hypothalamus : The lateral hypothalamus contains nuclei evaluating needs of the organism and responsive to appetitive US, and nuclei promoting actions related to digestive functions, blood pressure and other visceral functions (A. D. [START_REF] Craig | Interoception : the sense of the physiological condition of the body[END_REF]. In the framework of the two opponent process system, it is also strongly linked to the PAG for pain control. Similarly, on the aversive side, the PAG also encodes corresponding US and mediates defensive responses [START_REF] Bandler | Columnar organization in the midbrain periaqueductal gray : modules for emotional expression ?[END_REF]. Both structures are consequently reported as low-level homeostatic centres and are also involved in preparation of the sensory inputs to the insula and in the expression of more elaborated motivational behaviors [START_REF] Cardinal | Emotion and motivation : the role of the amygdala, ventral striatum, and prefrontal cortex[END_REF].

Where -The superior colliculus : The superior colliculus (also called the tectum in primitive animals) is a structure mainly studied for its involvement in eyes movement and gaze orientation [START_REF] Lee | Population coding of saccadic eye movements by neurons in the superior colliculus[END_REF]. It is composed of several layers, some receiving mainly visual information from many regions in the brain, including directly from the retina. The sensory layers are topographic maps of the surrounding environment and are in direct association with deeper motor layers for eye movements towards the place elected by competition in the sensory layer [START_REF] Taouali | A parsimonious computational model of visual target position encoding in the superior colliculus[END_REF]. It has been remarked that this structure can also perform direct sensorimotor associations for orientation of the whole body for tracking novel stimuli, for defensive movements and flight in case of a danger [START_REF] Dean | Event or emergency ? Two response systems in the mammalian superior colliculus[END_REF]. For more complex oculomotor behavior, the superior colliculus remains an essential stage between the retina and the posterior dorsal cortex and the frontal eye field [START_REF] Sommer | What the brain stem tells the frontal cortex. I. Oculomotor signals sent from superior colliculus to frontal eye field via mediodorsal thalamus[END_REF] in the frontal cortex.

How -The cerebellum :

The cerebellum is a cerebral structure known for its role in sensorimotor control [START_REF] Manto | Consensus paper : roles of the cerebellum in motor control-the diversity of ideas on cerebellar involvement in movement[END_REF]. Its granular cells are sensory inputs arranged in somatotopy and receiving most kinds of sensory information including from proprioception.

They are directly associated with purkinje cells projecting to all output structures responsible for motor control, from movement execution to planning. Particularly, these circuits have been shown to be involved in limb movements, manipulation, speech, both for direct automatic movements (postural adjustment, balistic movements, conditioned reflexes) and for the control of voluntary movement and even more abstract cognitive functions, through higher level centres [START_REF] Middleton | Basal ganglia and cerebellar loops : motor and cognitive circuits[END_REF].

In summary and in accordance with an enactive view of cognition, it is important to consider why the behavior is triggered, toward what goal, where it is situated and how it can be accessed. Each of these questions can be tackled independently by a simple sensorimotor association and we have reported here evidences that for each question one cerebral structure is particularly involved in elaborating such an association. We have also indicated that, in each case, other higher level structures can build more complex relations on the association, in a classical framework of imbricated sensorimotor loops [START_REF] Guillery | Anatomical pathways that link perception and action[END_REF], convenient both for incremental learning and for responding at anytime. Before bringing more information about other levels of imbrication, we first evoke another problem, related to a need of consistency between the selection of answers to these questions.

Selection of action

Obviously, to display an adapted behavior, these four questions cannot be addressed independently. Firstly, we can generally perform only one action of the same level at a time (we cannot at the same time go to the kitchen to eat and to the bedroom to have some rest). Secondly, the decisions taken at various levels must be consistent (imagine what happens if we decide that the need to address is hunger, if we select a salient visual goal like a predator approaching and if we grasp a bottle located nearby...). P. Redgrave and colleagues have nicely addressed this problem in [START_REF] Redgrave | The basal ganglia : A vertebrate solution to the selection problem ?[END_REF] and have proposed that the basal ganglia (BG) is the cerebral structure responsible for solving this kind of conflict in the brain, underlining that, even if some internal processing are distributed, the process of action selection is fundamentally centralised, which is rather rare in brain functioning.

The inner process of the BG is very complex, involving a variety of internal structures, pathways and mechanisms, as evoked in [START_REF] Redgrave | Basal Ganglia[END_REF], which are still topics of intense research. Basically, the BG can also be described as a sensorimotor set of nuclei, with the striatum as an input structure receiving sensory information and the internal part of the globus pallidus and the substancia nigra pars reticulata (called together GPi-SNr in primates ; can also correspond to other structures in other species) acting as an output motor structure. In accordance with most of the literature, we will first evoke here some characteristics of the loops between the BG and the cortex but we will also mention that the subcortical structures evoked above are also involved in these loops [START_REF] Mchaffie | Subcortical loops through the basal ganglia[END_REF].

Several parallel loops (five in [START_REF] Alexander | Parallel organization of functionally segregated circuits linking basal ganglia and cortex[END_REF]) have been described between the cortex and the BG, performing a generic function of action selection. In summary (cf. section 7 and 8 for details), the cortex can be separated in a sensory pole in the posterior cortex and a pole representing actions or tasks in the anterior (or frontal) cortex [START_REF] Fuster | The prefontal cortex. anatomy, physiology and neurophysiology of the frontal lobe[END_REF]. Depending on the current state in the sensory pole, actions in the frontal cortex can be triggered and maintained until some changes in the sensory pole are obtained. Actions can be chosen on a frequency basis (the action most often associated with a sensory event). Alternatively, for some context, other more specific decision might be taken to display a more adapted behavior (and get consequently more reinforcement). In the first case, frequencies can be estimated by learning sensorimotor contingencies in antero-posterior cortico-cortical connections but this is not possible in the second case because there is not a unique cortical region merging contextual and reinforcement information with the corresponding sensorimotor information. This is exactly what is proposed by the architecture of BG-loop, well adapted for a such a contextual sensorimotor learning controlled by the level of reinforcement.

For each BG-cortex loop, a region of the striatum receives afferent information from frontal and posterior cortical regions and one among the tasks represented in the frontal region is going to be selected by a funneling effect from the striatum to GPI-SNr and back to the frontal cortex after a thalamic step, depending on the nature of the sensory information received in the striatum [START_REF] Alexander | Parallel organization of functionally segregated circuits linking basal ganglia and cortex[END_REF]. This process of action selection is generally related to reinforcement learning, with a prominent role for the dopamine, sent by the ventral tegmental area (VTA) and the substantia nigra pars compacta (SNc) to modulate cortico-striatal connections and minimize reward prediction errors [START_REF] Joel | Actor-critic models of the basal ganglia : new anatomical and computational perspectives[END_REF].

The loops are displayed on a posterior-anterior axis in the frontal cortex and also correspond to different and well-identified regions of the striatum, as confirmed by many data reporting the topographical organization of projections and of information representation in these circuits [START_REF] Alexander | Parallel organization of functionally segregated circuits linking basal ganglia and cortex[END_REF][START_REF] Parent | Functional anatomy of the basal ganglia. I. The cortico-basal ganglia-thalamo-cortical loop[END_REF]. The loops have been named depending on the frontal areas mainly engaged and consequently on the kind of tasks selected by the loops [START_REF] Alexander | Parallel organization of functionally segregated circuits linking basal ganglia and cortex[END_REF]. In the motor loop, the dorsolateral striatum receives information from the motor cortex and proprioceptive information from the sensory cortex. The loop is somatotopically organized, which makes it organized to select different classes of actions, e.g. involving the face, the arm or the legs [START_REF] Alexander | Parallel organization of functionally segregated circuits linking basal ganglia and cortex[END_REF]. The oculomotor loop [START_REF] Hikosaka | Basal Ganglia orient eyes to reward[END_REF] participates in gaze orientation and also involves regions of the dorsolateral sensorimotor striatum receiving projections from the posterior dorsal cortex together with the frontal eye field frontal area, known to encode gaze movement [START_REF] Sommer | What the brain stem tells the frontal cortex. I. Oculomotor signals sent from superior colliculus to frontal eye field via mediodorsal thalamus[END_REF]. The lateral prefrontal cortex loop, also called the cognitive loop, involves the dorsomedial striatum and associative regions of the posterior and prefrontal cortex and is mainly engaged in cognitive control [START_REF] Koechlin | The Architecture of Cognitive Control in the Human Prefrontal Cortex[END_REF], related to the ability of the prefrontal cortex to manipulate abstract rules, as described below in section 8. In the anterior cingulate loop, the ventral striatum (mainly the core of the nucleus accumbens) receives sensory information about the needs of the body and the US experienced and is rather involved in the level of 'wanting' associated to the needs, with the anterior cingulate cortex in the medial prefrontal cortex monitoring performance with regards to the association of actions and reinforcement, to decide for a cost of action (fatigue, risk) adapted to the level of wanting (energizing role of motivation [START_REF] Niv | Cost, Benefit, Tonic, Phasic : What Do Response Rates Tell Us about Dopamine and Motivation[END_REF]). In the orbitofrontal loop, the ventral striatum (mainly the shell of the nucleus accumbens) receives information from the posterior ventral cortex and from the hippocampus, giving sensory details about objects, and from the orbitofrontal cortex, reported to encode the reward value of objects, typically of the CS, and to define the 'liking' of objects or their hedonic value [START_REF] Kringelbach | The human orbitofrontal cortex : linking reward to hedonic experience[END_REF].

The anatomical organization of the BG has been described by the parallel processing made by these segregated loops but also by a convergence of information [START_REF] Parent | Functional anatomy of the basal ganglia. I. The cortico-basal ganglia-thalamo-cortical loop[END_REF] due to several characteristics. Within loops, a funneling effect is obvious when the reduction in the size of data flow from the cortical input to the output of the loops in GPi-SNr is considered [START_REF] Joel | Actor-critic models of the basal ganglia : new anatomical and computational perspectives[END_REF]. Between loops, overlapping schemes can be deduced from several principles, like the spi-ral principle proposed in [START_REF] Haber | Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum[END_REF], where pathways between loops can be observed through dopaminergic projections and through overlapping prefrontal representations from one loop to the next. This is also the case, considering the participation of the subcortical structures mentioned in section 4.2 in the loops. They can be functionally associated, one to one, to the loops as proposed in figure 2, whereas anatomical data suggest a wider scheme, for example with the cerebellum linked to the motor and oculomotor loops [START_REF] Hikosaka | Basal Ganglia orient eyes to reward[END_REF] or the amygdala linked to orbitofrontal and anterior cingulate loops [START_REF] Cardinal | Emotion and motivation : the role of the amygdala, ventral striatum, and prefrontal cortex[END_REF].

As it is proposed in figure 2, taking appart the cognitive loop that will be described in section 8, each of the other four BG-cortex loop can be seen independently as selecting an action (or a task or a decision, depending on the nature of information) with regards to its afferent information and as participating to the answer to one of the four fundamental questions. In the orbitofrontal loop, What is the goal of my behavior ? In the anterior cingulate loop, Why should I spend energy satisfying the corresponding need (and up to which level) ? In the oculomotor loop, Where is this object ? In the motor loop, How should I behave (which action should I trigger) to get it ?

Depending on the complexity of the task and on the richness of the environment, these decisions can be constrained and articulated in different ways. On one extreme, we are in the domain of goal-directed behavior, when there are several answers to each questions, and when loops must interact to find the best global solution : contingencies between local decisions and their consequences must be known and corresponding reinforcements must be evaluated and compared. This can be associated to the domain of model-based reinforcement learning [START_REF] Dayan | Reinforcement learning : the good, the bad and the ugly[END_REF] and to the domain of hierarchical planning [START_REF] Pezzulo | Thinking as the control of imagination : a conceptual framework for goal-directed systems[END_REF], with the classical steps of deciding for goal, motivation, strategy and execution and of backtracking in the hierarchy when one step is impossible. On the other extreme, we are in the domain of habits, where the current state is enough to directly trigger the action with no need to refer a priori to a model of sensorimotor transitions or to the value of the outcome, as it is the case in model-free reinforcement learning [START_REF] Dayan | Reinforcement learning : the good, the bad and the ugly[END_REF]. Many behavioral experiments [START_REF] Packard | Learning and memory functions of the basal ganglia[END_REF] have shown that both types of learning are present and in competition in the brain and probably that the longlasting learning of the later is dependent on a supervision by the former. It is consequently important to wonder how the rich representa-tions of states and of sensorimotor contingencies needed by goal-directed behavior are built by learning and have been complexified through evolution, as we evoke now.

6 Keeping a memory of past episodes

Basically, we have explained above that, to give an ecological meaning to our behavior, our direct sensorimotor capabilities (being able to orient toward an object of interest (the where question) and being able to exploit the object with the body (the how question)) are enslaved by the motivational and emotional analysis of the situation (the why and what questions). At a first level of complexity, this can be performed by primitive structures (amygdala, PAG and hypothalamus) learning simple pavlovian associations and having strong relations with the ventral striatum. In the simplest case, when the goal of the behavior has been identified in the sensory region of the amygdala (BLA) and is directly available for consumption, BLA activates the amygdalar output CeA for pavlovian response and sends also projections to the shell of NAcc for the corresponding consummatory behavior. Anatomical and functional considerations underline how these responses are similar. There is in fact anatomical continuity between CeA and the shell of NAcc with a proposed similar functional organization [START_REF] Cassell | The Intrinsic Organization of the Central Extended Amygdala[END_REF] including strong dopaminergic innervation and projections to the same regions of motor output (including PAG and lateral hypothalamus). When the target of the behavior is not directly identified, the general class of motivation can give information to energize a preparatory behavior that will result in selecting the target. This is allowed by projections from CeA to the core of NAcc and can result in simple autoshaping or in more complex goal-directed behavior. This view gives to the ventral striatum (or NAcc) a central role at the interface between pavlovian and instrumental learning. That is the reason why it is particularly interesting to remark that, considering more elaborated information that was incorporated to the system along evolution and particularly in birds, episodic memory originating from the hippocampus is projected to the striatum mainly in its ventral division [START_REF] Voorn | Putting a spin on the dorsal-ventral divide of the striatum[END_REF].

Within the medial temporal lobe generally reported as dedicated to declarative memory, the hippocampus is more precisely associated to episodic memory [START_REF] Tulving | Episodic and semantic memory[END_REF], that allows us to remember specific Figure 2 -This scheme of the brain underlines some important anatomical and functional characteristics to better understand how information flows are processed in the brain. It is proposed that five kinds of neuronal structures bring more and more complexity and flexibility along phylogeny : (i) extracortical structures (the Amygdala, the Hypothalamus, the Superior Colliculus (SC) and the Cerebellum), (ii) the Basal Ganglia (with the striatum composed of its dorsolateral part DLS, dorsomedial part DMS and limbic part, also called Nucleus Accumbens NAcc with a shell and a core part ; with output structures like the internal Globus Pallidus and the substantia nigra pars reticulata GPI-SNr and the ventral pallidum VP ; with dopaminergic regions ventral tegmentum area VTA and substantia nigra pars compacta SNc) ; (iii) the hippocampus with its main internal structures dentate gyrus DG, CA3 and CA1 and its input cortical structures ; (iv) several regions of the sensory cortex receiving inputs from the sensory thalamus ; (v) several regions of the frontal cortex including the ventromedial prefrontal cortex vmPFC, the dorsomedial prefrontal cortex dmPFC, the lateral prefrontal cortex with ventral and dorsal parts vlPFC and dlPFC, the frontal eye field FEF and the motor and premotor regions. As explained in more details in the text, colors refers to the major implication of certain regions in these structures, to answer to fundamental questions for the selection of goals (what goal and why ?) and for their spatial access (where and how ?). These colors also refer to preferential projections between these regions, particularly forming loops between the structures, even if the text also explains that some complex functions result from interactions between different loops. events in their context. Through its input structure, the entorhinal cortex, the hippocampus receives cortical information from the posterior ventral cortex related to the what and why questions (via the perirhinal cortex) and from the posterior dorsal cortex related to the where and how questions (via the postrhinal or parahippocampic cortex) and aggregate them, including their organization in time [START_REF] Jensen | Hippocampal sequence-encoding driven by a cortical multiitem working memory buffer[END_REF], in an episode or event. This association of arbitrary information is made possible by the unique recurrent architecture of the hippocampal region CA3 that makes it work as an associative memory, learning very rapidly an event [START_REF] Kassab | Integration of exteroceptive and interoceptive information within the hippocampus : a computational study[END_REF]. Decision to memorize an event can be taken intrinsically on the basis of its novelty and from extrinsic afferents, particularly originating directly or indirectly from the amygdala [START_REF] Paz | Physiological basis for emotional modulation of memory circuits by the amygdala[END_REF], signaling errors of reward prediction and consequently a need for a more precise learning. Errors might be due to ambiguities in the conjunction of features [START_REF] O'reilly | Conjunc-tive Representations in Learning and Memory : Principles of Cortical and Hippocampal Function[END_REF] or in their temporal ordering, as the hippocampus is also particularly critical for sequence and delay learning [START_REF] Jensen | Hippocampal sequence-encoding driven by a cortical multiitem working memory buffer[END_REF]. Conversely, in the recall process, the hippocampus can be activated from partial information, evoke the complete episode and facilitate reactivation of other brain regions [START_REF] Gruber | Context, emotion, and the strategic pursuit of goals : Interactions among multiple brain systems control-ling motivated behaviour[END_REF] via its output structures CA1 and the subiculum. This has been for example reported as contextual signals sent to the amygdala for the extinction of pavlovian conditioning [START_REF] Moustafa | A model of amygdala-hippocampalprefrontal interaction in fear conditioning and extinction in animals[END_REF] or as predictive signals of possible paths sent to the prefrontal cortex and the ventral striatum during navigation of rats in a maze [START_REF] Gruber | Context, emotion, and the strategic pursuit of goals : Interactions among multiple brain systems control-ling motivated behaviour[END_REF].

From its ability to store and later detect and recall complex multimodal episodes, particularly including delays between their constituents, the hippocampus provides the ventral striatum and the amygdala with more complex features than simple sensory cues sent by the thalamus or the cortex. It is for example reported that hippocampal inputs are critical to the amygdala in pavlovian trace conditioning [START_REF] Paz | Physiological basis for emotional modulation of memory circuits by the amygdala[END_REF], when the CS and the US are separated by a delay. This also allows to create conditioned reinforcers in the amygdala, corresponding to subgoals or intermediate steps in a sequence of behaviors, sent to the ventral striatum and evoking surrogates of rewards when the actual reward is distant, as it is often the case in instrumental conditioning [START_REF] Cardinal | Emotion and motivation : the role of the amygdala, ventral striatum, and prefrontal cortex[END_REF].

The distinction evoked above between the posterior ventral cortex (the what and why questions) representing perception for recognition and the posterior dorsal cortex (the how and where questions) rather re-presenting perception for action [START_REF] Milner | The visual brain in action[END_REF] has also been clearly reported in the hippocampus [START_REF] Fanselow | Are the Dorsal and Ventral Hippocampus Functionally Distinct Structures ?[END_REF], with a dorsal region rather involved in navigation, with neurons coding for location (place cells) or head direction and a ventral region rather involved in emotional aspects and massively projecting to the amygdala and to the ventral striatum (mainly the shell). It must be noted that the dorsal hippocampus also projects to the core of the ventral (and the dorsomedial) striatum and to the anterior cingulate cortex [START_REF] Pennartz | The hippocampal-striatal axis in learning, prediction and goal-directed behavior[END_REF], which underlines the special position of this BG-cortex loop, intermediate between pavlovian and instrumental conditioning and associating basically actions and outcomes. This will be discussed in more details below.

In summary, the hippocampus can represent complex events, corresponding to specific episodes, introducing rich and complex sensory information in pavlovian and instrumental conditioning. This gradient of complexity in sensory inputs, from specific cues encoded in the sensory cortex to cognitive maps and emotional episodes in the dorsal and ventral hippocampus is very nicely illustrated in [START_REF] Voorn | Putting a spin on the dorsal-ventral divide of the striatum[END_REF], gathering anatomical information in rats about hippocampal projections to the striatum, the amygdala and the frontal cortex, also ordered along that gradient. Such complex information allows birds and mammals to learn pavlovian associations with a complex pattern in time. It is also critical in goal directed behavior which requires the prospective evocation of perception-action contingencies and of outcome values, as it has been reported in the hippocampus and the ventral striatum [START_REF] Bornstein | Multiplicity of control in the basal ganglia : computational roles of striatal subregions[END_REF].

Experiments in rats [START_REF] Packard | Learning and memory functions of the basal ganglia[END_REF]) have shown that rapid and flexible goal-directed behavior involving the hippocampus and the dorsomedial striatum can be replaced by repetition by an habitual behavior involving the dorsolateral striatum and corresponding to a simple stimulus-response association insensitive to reward devaluation. Since the dorsolateral striatum has no hippocampal but only cortical sensory inputs, it can be thought that the slow habitual learning is constrained by the time for consolidation from the hippocampus to the sensory cortex, of the critical events triggering the response. In fact, when habits have been learned, the same experiments [START_REF] Packard | Learning and memory functions of the basal ganglia[END_REF] show that both goal-directed and habitual learning coexist and are in competition. In a very interesting view [START_REF] Penner | Neural systems analysis of decision making during goal-directed navigation[END_REF] using the actor/critic framework where reinforcement learning is decomposed in an actor applying the current po-licy (rules of behavior) and a critic learning from errors of prediction the value of the outcomes and modifying the policy correspondingly, the dorsolateral striatum is proposed to be the actor for habitual behavior and the dorsomedial striatum to be the actor for goal-directed behavior. The shell, corresponding to the consummatory behavior and learning explicitly the value of the outcome, is proposed to be the critic of the dorsomedial striatum, learning explicitly the model of the world for goal-directed behavior, whereas the core, associated to preparatory behavior not specific of the outcome and learning only to associate an action to a motivational value, should be the critic of the dorsolateral striatum, associating directly in a habitual mode states to actions.

All these pieces of information give a very important role to the ventral striatum at the interface between limbic and motor systems. [START_REF] Mannella | The nucleus accumbens as a nexus between values and goals in goal-directed behavior : a review and a new hypothesis[END_REF] describes the ventral striatum as the place where motivational values are assigned to goals from their pavlovian value given by the amygdala and their salience and novelty given by the hippocampus. This results in associations between the outcomes and their motivational value in the shell and between actions and the outcomes in the core, and the corresponding energizing effect on instrumental behavior. The dorsomedial striatum is also a key player in instrumental behavior and its role will become more clear as more details are given about the prefrontal cortex in the next section.

Building abstract categories

Beyond the memory of specific episodes, an important innovation has been brought by the cortex to build structured high-level information over simple signals : the elaboration of abstract categories composing a semantic memory. In the posterior cortex such a representation is built on data flows corresponding to the sensory dimensions evoked by the four questions discussed above (cf. also figure 3). This results in hierarchical cortical areas with neuronal populations responding to more and more complex objects [START_REF] Rousselet | How parallel is visual processing in the ventral path ?[END_REF], building more and more abstract categories in the ventral information flow relating the exteroceptive and interoceptive poles, for the What and Why questions. Based on considerations on the timing of information propagation [START_REF] Nowak | The timing of information transfer in the visual system[END_REF], the information flow is described as parallel rather than serial in the dorsal pathway to elaborate categories between the exteroceptive and motor poles, related to the ques-tions Where and How, even if intermediate strategies are also observed in associative areas, between a purely constructivist hierarchical and a purely purposive specialized view of information processing [START_REF] Norman | Two visual systems and two theories of perception : An attempt to reconcile the constructivist and ecological approaches[END_REF]. This intricate representation is particularly useful to account for selective attention, a function of the posterior cortex particularly critical in primates [START_REF] Fix | A Dynamic Neural Field Approach to the Covert and Overt Deployment of Spatial Attention[END_REF], associating selection of spatial regions and implicit or explicit (covert or overt) involvement of body parts in the dorsal regions together with an anticipation of the subregion of the sensory space that will be available and the focused processing of critical features in the ventral regions.

In these associative regions, one crucial (and still open) question is about the choice of the compound objects to be represented since the combinatorics is obviously too large for a systematic representation. This selection is made by learning and in an ecological view, a simple (but vague) criterion is : « Those which are the most useful to the organism ». A more precise specification must rely on the mechanisms triggering sensory learning in the posterior cortex [START_REF] Ahissar | Attentional control of early perceptual learning[END_REF], including the role of cholinergic modulation triggered by the amygdala, in case of error of prediction, to favor attentional process in the cortex [START_REF] Pauli | Attentional control of associative learning-a possible role of the central cholinergic system[END_REF], and the role of reinstatement of episodes in the cortex, driven by the hippocampus in the consolidation process [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex : insights from the successes and failures of connectionist models of learning and memory[END_REF].

Another important actor in the processes described in this section is the thalamus [START_REF] Sherman | The thalamus is more than just a relay[END_REF], for the critical role of its sensory part in the activation of the posterior cortex, conciliating feed-forward sensory input and feed-back cortical expectations, and also in cortical learning of new categories, particularly involving multimodal features. Nevertheless, it will not be described in details in this paper. Nor will be described the motor thalamus, even if it also has a critical role in the functioning of the frontal cortex presented below.

Building flexible sequences

Among the characteristics of the frontal cortex, the capacity of focused populations to sustain their activity in a working memory process is one of the most critical [START_REF] Fuster | The prefontal cortex. anatomy, physiology and neurophysiology of the frontal lobe[END_REF]. The organization of the frontal lobe can be described in reference to regions of the posterior cortex in which frontal regions can control transitions of states [START_REF] Burnod | An adaptive neural network : the cerebral cortex[END_REF]. In the motor cortex, neurons arranged in stripes symmetrical to the somatosensory area can trigger elementary actions modifying the posi-Figure 3 -This figure integrates the enactive view of the brain-body-environment system and the functional view of brain structure in a behavioral organization, where sensory interoceptive and exteroceptive poles interact with the limbic and motor poles to decide for the main characteristics of the behavior. Basically, addressing the four fundamental questions results in specifying sensory constraints in the motor pole related to the position in space and to the body, and in the interoceptive pole related to pain and pleasure and to fundamental needs. This will define preferences and motivations in the limbic pole, generating directly a consummatory behavior or organizing a preparatory behavior with the motor pole that can particularly trigger movements and evoke selective attention to obtain desired changes in the internal and external world and, accordingly, in the sensory perceptions. tion of the bodily scheme until the sensory goal of the action (e.g. position of a limb, characteristics of the sound produced by the phonatory apparatus) is reached. Motor control is also reported in the premotor cortex, with a more integrated topography [START_REF] Graziano | The organization of behavioral repertoire in motor cortex[END_REF], corresponding to more ecological categories of the behavioral repertoire like climbing, reaching, etc. Similarly, in oculomotor regions like the frontal eye field [START_REF] Sommer | What the brain stem tells the frontal cortex. I. Oculomotor signals sent from superior colliculus to frontal eye field via mediodorsal thalamus[END_REF], transitions are between initial and final targeted eye positions.

The same process of control of transition between present and targeted states can be used to describe the functions of the limbic frontal regions. In the ventromedial prefrontal cortex, regions like the orbitofrontal cortex have been described to receive exteroceptive and interoceptive sensory inputs [START_REF] Kringelbach | The human orbitofrontal cortex : linking reward to hedonic experience[END_REF] to encode the value of emotional stimuli, seen as potential goals of the behavior. This is the basis for emotional control, with the selection of a desired goal that is sustained until that goal is consumed (or given up). Complementary to the consummatory behavior, the preparatory behavior is organized in the dorsomedial prefrontal cortex, with areas like the anterior cingulate cortex performing motivational control [START_REF] Kouneiher | Motivation and cognitive control in the human prefrontal cortex[END_REF]. Basically, these regions, described as associating actions to outcomes, are in charge of deciding if the energy required by the actions selected in the preparatory phase is worth the corresponding need. Accordingly, they are reported to energize the behavior, i.e. to evaluate up to which level it can be engaged and, when a strategy is selected, to maintain this selection until it is achieved.

It can be remarked that the medial prefrontal cortex is also structured in a ventral part, deciding for the selection of a goal from its features, and a dorsal part, selecting the dimension on which to apply the strategy (and accordingly the action). In both cases, when this selection is not trivial and requires elaborated rules, the needed cognitive control recruits additional circuits in the lateral prefrontal cortex [START_REF] Badre | Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes[END_REF]. This region, increasingly large in primates, is also distributed in ventral and dorsal regions and is reported to elaborate more complex rules with complexity defined as the level of sequential arrangement of actions in the dorsolateral prefrontal cortex and as the level of precision in the definition of cues in the ventrolateral prefrontal cortex [START_REF] O'reilly | The What and How of prefrontal cortical organization[END_REF], both together able to build a complex strategy, decomposing a goal and a level of engagement in subgoals and actions to get them. The same principle of maintenance of activity until satisfaction (or giving up) results in resistence to distraction, another strong characteristic of the prefrontal cortex. Some generic mechanisms of the frontal cortex, evoked in all the regions, can be re-interpreted now. Each region has been described as preferentially linked to posterior cortical regions and composed of actions (or tasks) monitoring a transition from one state represented in this posterior region to another (from an initial to a final position ; from need (e.g. water deprivation) to satisfaction of the need (satiety)). This can be interpreted with the scheme S 1 -A -S 2 , where S 1 is the initial condition eliciting A as a possible action (cf. [START_REF] O'regan | A sensorimotor account of vision and visual consciousness[END_REF] and the principle of affordance) and S 2 is the consequence that can be anticipated if A is preactivated. Conversely, if S 2 is a desired state, A is the action that has to be activated to obtain S 2 , which is possible if S 1 is compatible with the current state. Else, A can display a sustained activity, as in working memory state, and remain active until S 1 is satisfied. This interpretation is reminiscent of behavioral studies where antecedents and consequences of goal-directed actions are seen as beliefs and desires [START_REF] Balleine | The integrative function of the basal ganglia in instrumental conditioning[END_REF] and of more theoretical works on planning [START_REF] Burnod | An adaptive neural network : the cerebral cortex[END_REF][START_REF] Pezzulo | Thinking as the control of imagination : a conceptual framework for goal-directed systems[END_REF] explaining how goals (desires) can be decomposed in subgoals (S 1 becomes desired) and recursively executed in such S 1 -A-S 2 schemes. In our view, these intermediate steps with subgoals are built by cognitive control in lPFC : the goal remains active as a working memory in mPFC and activates subgoals and means (sequences of actions) to get them in lPFC until good conditions are met (e.g. finding the kitchen seen as a subgoal to open the fridge), without forgeting the initial goal, as ensured by the sustained activity insensitive to distraction. This view is very consistent with an interpretation of the role of the BG for the dynamic gating of PFC representations [START_REF] O'reilly | Computational models of cognitive control[END_REF], switching from the updating of the choice of the best action to be selected (from the prediction of the value of its consequence) to the maintenance of its sustained activity until this consequence is obtained. This also explains why goal-directed behavior is defined by its sensitivity to goal devaluation and contingencies of actions [START_REF] Balleine | The integrative function of the basal ganglia in instrumental conditioning[END_REF] : in the habitual mode, S 1 directly triggers A with no consideration of S 2 and of the value of the goal obtained at the end of the process.

Globally, this heavy and structured process of the frontal (=prefrontal + premotor and motor) cortex can be summarized as follows : Exteroceptive and interoceptive stimuli can elicit preactivations in the motor and limbic prefrontal cortex which can also evoke the anticipated consequences in exteroceptive and interoceptive terms. In simple and stable worlds, the elaborated model of the world can become of good predictive quality and at the end the initial stimuli can be sufficient to trigger directly the actions without evoking their consequences. This corresponds to the habitual mode, progressively shifting the control from the limbic to the motor loops [START_REF] Hélie | Learning robust cortico-cortical associations with the basal ganglia : An integrative review[END_REF] and in the long term, only mobilizing the motor cortex in a basic stimulus-response scheme. Nevertheless, in the early phases of learning or when the world is changing or when the best behavior to be selected does not correspond to the most frequent (for example in a specific context), a more precise analysis of the recent history of performance must be carried out, involving the limbic parts of the prefrontal cortex and of the basal ganglia. This is the reason why the dorsomedial prefrontal cortex is often reported to be involved in error detection and conflict monitoring [START_REF] Rushworth | Action sets and decisions in the medial frontal cortex[END_REF]) and the ventromedial prefrontal cortex to be sensitive to devaluation of outcome [START_REF] Kringelbach | The human orbitofrontal cortex : linking reward to hedonic experience[END_REF] for example in case of reversal and extinction. The interoceptive preactivation of the limbic loops can evaluate and supervise this goal-directed learning, depending if gains or losses are observed between anticipated and actually obtained punishments and rewards, and results in the selection of the current goal and motivation.

In this goal-directed process, the role of the basal ganglia is prominent as a critic to learn from errors of prediction and as an actor to explicitely trigger step by step the full plan of action, as it was explained above. Concerning the transition between loops, note for example that both ventro-and dorsomedial prefrontal cortex project to the dorsomedial striatum [START_REF] Gruber | Context, emotion, and the strategic pursuit of goals : Interactions among multiple brain systems control-ling motivated behaviour[END_REF] and that the exteroceptive preactivation of the motor loop is critical to offer affordances that help select the most appropriate preparatory behavior [START_REF] Pezzulo | Thinking as the control of imagination : a conceptual framework for goal-directed systems[END_REF], also supposed to be performed in the striatal region. The role of the dopamine must be also particularly emphasized here, with dopaminergic projections from VTA to the ventral striatum mainly for pavlovian aspects and from SNc to the dorsomedial striatum for the instrumental aspects, in interaction with the lateral prefrontal cortex [START_REF] Yin | Reward-guided learning beyond dopamine in the nucleus accumbens : the integrative functions of cortico-basal ganglia networks[END_REF]. These pathways are at the basis of the spiral principle by S. Haber evoked above [START_REF] Haber | Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum[END_REF], also concerned with the articulation between BG-cortex loops.

Discussion

In this paper, we have proposed a systemic description of the brain, as a general framework for integrating more specific models in computational neuroscience. Beyond its intrinsic interest, this framework is also necessary, else the risk is to build models of particular neuronal structures in isolation without reference to more global information flows and cognitive functions and consequently to neglect some characteristics of the structure or to overload it with functions carried out in other parts of the cerebral network.

This framework considers several fundamental aspects of the brain, seen as the device controlling the behavior of the body, as summarized in figure 3 : (i) In an enactive view, the brain has to elaborate loops with the internal and the external environments (cf. figure 1) and to ensure their stability for the general goal of survival. This circular causality has already been expressed in many systemic views, including in computational neurocience, a very interesting approach exploiting the powerful formalism of thermodynamics [START_REF] Friston | A Free Energy Principle for Biological Systems[END_REF]. It confirms also the fundamental organization of the brain in sensorimotor loops and structures [START_REF] Guillery | Anatomical pathways that link perception and action[END_REF]. (ii) To provide a more precise account of the various characteristics of behavior, we have structured brain functions as answering to four fundamental questions (what, why, where and how). From the basic what/where system [START_REF] Ungerleider | Two cortical visual systems[END_REF], this has a long history in neuroscience, including more recent and precise views [START_REF] Milner | The visual brain in action[END_REF][START_REF] O'reilly | The What and How of prefrontal cortical organization[END_REF]. In particular, a very close formulation is proposed in [START_REF] Verschure | The why, what, where, when and how of goal-directed choice : neuronal and computational principles[END_REF] but is less accurate concerning the mapping to brain structures and functions. (iii) As represented in figure 2, anatomical and dynamical considerations are added to explain how these questions can be addressed in brain structures, particularly taking phylogeny into account. Primitive animals have the same fundamental problems of behavior organisation for survival ; the main difference is about the quantity and quality of information that is provided to define the needs and elaborate the answers.

Elaborating such a framework is also interesting because it provides opportunities to put together many facts at various levels of description and to elaborate on them principles of brain functioning and of cognitive architecture. Basically, the behavior is elaborated from cross-talk between the motor and limbic poles (cf figure 3). The motor pole receives affordances from the exteroceptive pole that might result in triggering habits or at least preactivating some actions. The limbic pole receives emotional and motivational information from the interoceptive pole which can directly trigger pavlovian reflexes and consummatory behavior and lead to the definition of goals and motivations. Then, ins-trumental conditioning will allow to specify the more appropriate preparatory behavior, based on preactivated actions and learned contingencies.

As we might expect, this sketch gives a prominent role to pavlovian and instrumental conditioning, in this survival-oriented definition of behavior. Nevertheless, the specific contribution of other kinds of learning is obvious, specifically related to semantic memory in the posterior cortex, episodic memory in the hippocampus and working memory in the frontal cortex. More fundamentally, this also lays emphasis on the fact that cognition can be described as a dynamical system of interacting memories, some acting to provide information to others, to replace others when they are not efficient enough, or to help for the convergence of others. Such principles have already been described with a very strong impact in improving our understanding of cognitive mechanisms [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex : insights from the successes and failures of connectionist models of learning and memory[END_REF]. The framework that we have proposed here is a very convenient tool to study such mnemonic synergies, very difficult to delimit because not related to a unique cognitive function. We have for example seen that goal-directed behavior can generate a prospective memory (also called « memory of the future » [START_REF] Fuster | The prefontal cortex. anatomy, physiology and neurophysiology of the frontal lobe[END_REF])), as it is observed for places in the hippocampus and for rewards in the ventral striatum and can also generate a retrospective memory that can participate in training the habitual system [START_REF] Bornstein | Multiplicity of control in the basal ganglia : computational roles of striatal subregions[END_REF]. Precisely understanding how these processes work and interact is an important challenge for future research.

Describing the brain as an architecture of learning systems has also strong implications in Machine Learning. In addition to the definition of interacting mnemonic synergies as a basis to ensure really autonomous learning, which is very poorly addressed in classical Machine Learning, a more realistic view of pavlovian and instrumental conditioning can be very precious to revisit classical Reinforcement Learning. Such a contribution is for example proposed in [START_REF] Balasubramani | An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning[END_REF] where the selection of action is controlled by a utility function, defined by a weighted combination of value and risk. This can extend the classical actor/critic architecture and needs to be deepen, since many questions remain about the exact location of the critic (if any) and about the validity of the hypothesis of a double model-based and model-free actor-critic system evoked above [START_REF] Penner | Neural systems analysis of decision making during goal-directed navigation[END_REF].

Our systemic framework is also useful because it helps revisit the role of certain cerebral structures. Particularly, it appears from our analysis that the BG is best defined as a modulatory system that provides adaptive gating signals to the frontal cortex, instead of the prevalent ideas that BG directly encodes S-R associations and can be defined as a procedural learning system. This is confirmed by the (relatively) low impact of BG lesions in the behavior performances but rather in learning [START_REF] Packard | Learning and memory functions of the basal ganglia[END_REF]. One step further, the following phylogenetic interpretation could be proposed : Beyond simple reactions due to primitive extracortical structures, the hippocampus and the BG could be proposed as two structures endowed with rapid learning to adapt the behavior to specific cases, respectively corresponding to emotional episodes and to behaviors where the motivation must be explicitely reminded. But in both cases, the goal of the organism is to identify and learn criteria to automatize the behavior. In the same way as there is a consolidation from the hippocampus to the posterior cortex to create new categories in the semantic memory, appropriate to discriminate the world, the transfer from goal-directed behavior involving the BG to habits in the motor pole of the frontal cortex might be seen as a way to create motor routines, giving a behavioral repertoire adapted to our needs. At the end, this would result in a system parallel to the primitive extracortical structures, except that the sensory and motor characteristics would have been selected and learned in a slow process, from interactions with the world.

Our framework also indicates some specific structures, which can be seen as a kind of hub in the cognitive architecture, because they are fundamentally multimodal and coordinate transfers between memory systems. For different reasons, this is particularly the case with the ventral striatum and the hippocampus which woud have to be studied more deeply in that perspective. Another domain which remains not detailed enough is the precise definition of the mechanisms of cognitive control, with the elaboration of complex rules in the lateral prefrontal cortex and their specific dorsal and ventral aspects. Preliminary theories have already been proposed [START_REF] Badre | Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes[END_REF][START_REF] Koechlin | The Architecture of Cognitive Control in the Human Prefrontal Cortex[END_REF] and should be more deeply explored in specific behavioral applications for a better understanding.

It might also be said that this paper proposes a very « mechanical » view of the brain, neglecting highest cognitive functions of the brain, for example related to social aspects, language, not to evoke internal thought, mind wandering and consciousness. We think in fact that most of these functions rely on the same bases as the ones evoked here, even if of course our view still has to be consolidated and made more precise, particularly concerning the thalamus and neuromodulation, hardly described here.