Gregory Lafon

Scarlett R Howard

Benjamin H Paffhausen

Aurore Avarguès-Weber

Dr Martin Giurfa
email: martin.giurfa@univ-tlse3.fr

Haiyang Geng

Alexis Buatois

Isabelle Massou

Motion cues from the background influence associative color learning of honey bees in a virtual-reality scenario

Keywords:

Honey bees exhibit remarkable visual learning capacities, which can be studied using virtual reality (VR) landscapes in laboratory conditions. Existing VR environments for bees are imperfect as they provide either open-loop conditions or 2D displays. Here we achieved a true 3D environment in which walking bees learned to discriminate a rewarded from a punished virtual stimulus based on color differences. We included ventral or frontal background cues, which were also subjected to 3D updating based on the bee movements. We thus studied if and how the presence of such motion cues affected visual discrimination in our VR landscape. Our results showed that the presence of frontal, and to a lesser extent, of ventral background motion cues impaired the bees' performance. Whenever these cues were suppressed, color discrimination learning became possible. We analyzed the specific contribution of foreground and background cues and discussed the role of attentional interference and differences in stimulus salience in the VR environment to account for these results. Overall, we show how background and target cues may interact at the perceptual level and influence associative learning in bees. In addition, we identify issues that may affect decision-making in VR landscapes, which require specific control by experimenters.

Thanks are also due to Dr. Jean-Christophe Sandoz and Dr. Ricarda Scheiner for accepting to be part of my thesis committee and for their helpful advice and guidance during my PhD.

I'm extremely grateful to Haiyang Geng and Isabelle Massou who are responsible for all the cellular and molecular analysis that made chapter 2 and 3, and their associated publications, possible. They have done a staggering amount of work for this project and I consider myself lucky to have had the opportunity to work with them.

I am thankful to Alexis Buatois who established the initial VR setup for helping me get started with my PhD and for encouraging me to improve the VR.

I also wish to thank Dr. Mathieu Lihoreau for providing colonies, and material to help us get started on the bumblebee experiments. I'd like to thank Stephane Kraus and Mathilde Lacombrade for teaching me how to take care of the bumblebees. I am grateful to Benjamin Paffhausen and Marco Paoli for many fruitful discussions, not all of them about our research. Their kindness and enthusiasm for science made working on this project truly enjoyable.

I would also like to thank Maud Combe and Patrick Arrufat for providing the technical support that I needed to develop both the hardware and the software used in this thesis.

I am also thankful to Dr. Gabriella de Brito Sanchez for sharing her insights on bee gustation.

Thanks also to all the intern that I had the chance to supervise: Rodrigue Fouillet, Juliette Montet, Diane Sam Mine, Emma Giordanengo, Catherine Macri, Clemence Guinnement, Marin Nicolas, Naïs Judan, Eva Blot, and Karolina Pecharova. They have done an extensive amount of work for this project and I hope that their time with us was as beneficial to them as it was for me.

I am extremely gratefull to Dr Christian Jost for accepting to be my tutor in the DCCE program and helping me throughout my three years as a teaching assistant.

I am also gratefull to Dr. Jean-Marc Devaud, Dr. Cédrick Florian, Dr. Guillaume Isabel and Dr Christian Jost for giving me the opportunity to teach in their courses.

Finally, I am extremely grateful to my fellow PhD students for answering all my numerous questions about everything, from university paperwork to plant phylogeny, for helping me pick the color of my graphs and of course for many memorable nights out.

Aims and goals

The general objective of this doctoral thesis is to explore the underlying mechanism of visual learning in the honeybee Apis mellifera. To do so we decided to use virtual reality (VR), in order to move from the classical studies using free-ly flying bees to a controlled setup in which a tethered animal would learn visual discriminations. Our team had recently developed a new VR setup, which was far from being immersive as it allowed only translational stimuli movements (i.e. 2D VR). In order to be able to use VR to its full potential we first worked on upgrading the existing setup to a true 3D virtual environment. This introduced the possibility of enriching the VR with a background that could generated optic flow as the bee moves within the virtual world.

This new possibility inspired the first question addressed in this work, namely how do motion cues from the background influence associative color learning in bees in the 3D VR environment? To answer this question, we used our new setup to test if and how frontal motion cues generated in the VR and ventral motion cues generated by the movement of the treadmill below the bee affected color discrimination learning. In the first chapter, we present the answer to this question and identify issues that may affect decision-making in VR landscapes.

Answering that first question led us to refine both our setup and our conditioning protocols, thus raising our second question: What are the brain regions involved in visual learning? To answer it, we quantified expression of three Immediate Early Genes (IEGs) that serve as markers of neural activity in brain areas and that had been related to bee foraging and orientation: kakusei, Hr38 and Erg1. We analyzed the expression of these IEGs in the calyces of the mushroom bodies, the optic lobes and the rest of the brain after color discrimination learning in VR. More specifically, we asked if the nature of IEG expressiàon, and thus the areas involved in visual learning, change depending on the way in which the animal learns the visual discrimination. We thus compared IEG expression after learning in the 3D environment and after learning in the 2D environment which set more constraints in terms of stimulus movements. These two analyses are presented in two separate chapters

The study of the neural mechanisms underlying visual learning requires using invasive approaches to access the brain of the insects, which induces stress and can thus impair behaviors. To potentially mitigate this effect, we performed an additional study using bumblebees Bombus terrestris, which could constitute a good alternative to Apis mellifera as they are bigger and more robust and resistant to potentially harming procedures. In the last chapter, we explored the performance of bumblebees in a differential learning task in the VR and compared it to that of honeybees.

Overall, our work resulted in a novel and robust 3D VR system that is inexpensive, open source and supports experiments on both bumblebees and honeybees. This system represents therefore a qualitative advance for studies on honey bee visual learning. We also produced the first quantification of IEG expression in the bee brain as a result of associative visual learning and provide data showing the implication of mushroom bodies in this learning form. Taken together our results open the way for a deeper exploration of the mechansims of visual learning through VR experimentation.

Introduction

Honeybees are flying hymenoptera famous for their social organization and their important contribution to the pollination of crops and wild plants. Honeybees are central place foragers that are flower constant [START_REF] Grant | The fertilization of flowers[END_REF][START_REF] Chittka | Flower Constancy, Insect Psychology, and Plant Evolution[END_REF], meaning that they tend to constantly return to the same species of flower as long as they are available and profitable, even if other more rewarding flower species are available in the vicinity. In relation with both their eusociality and their status as central place foragers, honeybees have developed a complex system of communication relying on one hand on pheromones and on the other hand on stereotyped movements called "dances", which vary in shape according to the range of distances separating the hive and the food source [START_REF] Frisch | Dance language and orientation of bees[END_REF]. The most studied dance type is the waggle dance, which reports distance and direction of profitable food source to nest mates. It's the discovery of this surprising behavior that made ethologist Karl von Frisch famous [START_REF] Frisch | Dance language and orientation of bees[END_REF] and might have played an important role in consolidating the honeybee, among a few other invertebrates, has a major research model in neuroscience and behavior.

Given honeybees' flower constancy, they have the capacity to learn and memorize the essential traits that characterize the flower species they exploit at a time. Honeybees ability to identify and remember particular flowers species through numerous foraging bouts for periods of time that can span several days make them a very enticing species for studying learning and memory (Menzel, 1999). Even more so because their brain is made of about 950 000 neurons for a volume of about 1 mm 3 , which makes the underlying mechanism of their cognitive abilities more accessible. Hence, for more than fifty years now, honeybees' associative learning abilities have been extensively studied (Giurfa, 2007). Studies on honeybee learning have spanned mostly the visual and the olfactory domain but have reached a unique dimension in the latter given the fact that honeybees can be easily conditioned to respond appetitively to a particular odorant while being immobilized [START_REF] Takeda | Classical conditioned response in the honey bee[END_REF]Bitterman et al., 1983;Giurfa and Sandoz, 2012). For the conditioning of the Proboscis Extension Response (PER) [START_REF] Felsenberg | Behavioural Pharmacology in Classical Conditioning of the Proboscis Extension Response in Honeybees (Apis mellifera)[END_REF] each bee is restrained in an individual harness such that it can only freely move its antennae and mouth-parts (mandibles and proboscis). When the antennae of a bee are touched with sucrose solution, the animal exhibits the PER, i.e. it reflexively extends its proboscis to reach out to the sucrose. Neutral odorants blown to the antennae do not release such a reflex in naive animals. If, however, an odorant is presented immediately before sucrose solution (forward pairing), an association is formed which enables the odorant to elicit the PER in a following test. This effect is clearly associative and constitutes a case of classical conditioning (Bitterman et al., 1983), i.e. the odorant can be viewed as the conditioned stimulus (CS) and the sucrose solution as the rewarding, unconditioned stimulus (US) (Fig. 5). Within this framework, bees learn to associate the odorant with the sucrose reward. Immobilization is crucial in this context as it enabled the use of multiple invasive techniques to study the cellular and molecular underpinnings of olfactory learning and memory [START_REF] Mauelshagen | Neural correlates of olfactory learning paradigms in an identified neuron in the honeybee brain[END_REF][START_REF] Abel | Structure and response patterns of olfactory interneurons in the honeybee, Apis mellifera[END_REF][START_REF] Komischke | Partial unilateral lesions of the mushroom bodies affect olfactory learning in honeybees Apis mellifera L[END_REF]Boitard et al., 2015;[START_REF] Carcaud | Parallel Olfactory Processing in the Honey Bee Brain: Odor Learning and Generalization under Selective Lesion of a Projection Neuron Tract[END_REF].

Studies on olfactory learning have been mostly confined to the use of elemental protocols, that rely on the simple unambiguous association of at least two elements like in PER conditioning, although more recently, studies on non-elemental olfactory learning, protocol in which the reward or its absence is not associated univocally with the stimulus, have also revealed a capacity to solve non-linear discriminations, i.e. to go beyond simple forms of associative learning [START_REF] Meyer | Elemental and configural olfactory coding by antennal lobe neurons of the honeybee (Apis mellifera)[END_REF]Devaud et al., 2015a). Yet, at the same time as it offered the advantage of neural and molecular access, immobility represented a significant burden for the possibility of observing the richness and cognitive complexity of free behavior. Experiments with freely flying bees trained to solve discriminations in the visual modality showed precisely that the cognitive capacities of bees under these experimental conditions were highly elaborated and parallel to some abilities that were thought a prerogative of vertebrates (Avarguès-Weber et al., 2011a;Avarguès-Weber and Giurfa, 2013). For instance, free-flying honeybees were shown to be able to learn concepts, a relation between different objects that is independent of the physical nature of the objects linked by the relation [START_REF] Lamberts | Knowledge Concepts and Categories[END_REF][START_REF] Zentall | Categorization, Concept Learning, and Behavior Analysis: An Introduction[END_REF][START_REF] Doumas | A theory of the discovery and predication of relational concepts[END_REF][START_REF] Zentall | Concept learning in animals[END_REF][START_REF] Halford | Relational knowledge: the foundation of higher cognition[END_REF], in the visual modality such has sameness [START_REF] Giurfa | The concepts of 'sameness' and 'difference' in an insect[END_REF] or above/below (Avarguès-Weber et al., 2011b) and even combinations of those concepts (Avarguès-Weber et al., 2012a). However, the exploration of the underlying mechanisms of visual learning, be it simple associations or conceptual learning, has remained elusive due to the impossibility of accessing the nervous system of flying bees solving visual problems. At the same time, conditioning harnessed bees to visual stimuli has yielded low rates of success (Avarguès-Weber and Mota, 2016). A goal of this thesis consisted, therefore, in overcoming this historic limitation by establishing a new experimental paradigm allowing the coupled study of visual learning and neural analyses in bees.

Prior studies on honeybee visual learning

Honeybee visual learning has been studied for more than a century now, and many different techniques and setups have been developed to this end. In order to understand what needs to be done to allow us to progress in our understanding of their visual learning abilities we first need to know what has already been done, what worked, and what didn't. In this part, I will show that looking at the hundred years of scientific development (starting with von Frisch's seminal demonstration of color vision in free-flying bees) clearly outlines the great success of experiments on freely moving bees, which contrasts with the milder results obtained so far with immobilized bees.

Experiments with free-flying bees

In his pioneering work on honeybee vision, Karl von Frisch aimed at demonstrating that honeybees were endowed with color vision, although at that time the status quo was to claim that they were color blind [START_REF] Hess | Experimentelle Untersuchungen zur vergleichenden Physiologie des Gesichtssinnes[END_REF]. To this end, he trained bees to freely fly to an experimental set up where they could get sucrose solution in a small Petri dish placed on a blue piece of cardboard; then he tested the bees returning to the experimental site by offering at the same location several targets, one blue, the same used during training, and various shades of grey [START_REF] Frisch | Der Farbensinn und Formensinn der Biene[END_REF]. On all these visual stimuli, an empty Petri dish was presented. The rationale of this experiment was to show that bees would not confound the learned color hue with an achromatic intensity (a shade of grey) that would be common to the blue and grey cardboards (Fig. 1). He showed indeed that the majority of the bees chose to go to the blue target and not to the achromatic alternative displaying a similar intensity [START_REF] Frisch | Der Farbensinn und Formensinn der Biene[END_REF]. With this experiment, which he repeated for various color hues, he demonstrated unequivocally that honeybees are able to perceive colors. Bees were trained to feed on blue feeders and were tested on a multitude of feeders in random arrangement of blue and grey stimuli. The blue feeder received more visits than the achromatic ones. Taken from von [START_REF] Frisch | Der Farbensinn und Formensinn der Biene[END_REF] Since then, many experiments have studied different form of visual learning in free-flight conditions. In this type of experiment, trained honeybee foragers come directly from the hive to an experimental site where the stimuli to be learned and/or discriminated are offered, paired or not with a sucrose reward. The trained bee, marked with a color on its abdomen or thorax in order to identify it, will perform many flights between the hive and the experimental site to collect the food reward and thereby to answer the questions raised by the experimenter. This scenario can be labeled as mainly operant conditioning [START_REF] Rapaport | Instrumental deconditioning of the young rat conditioned to an ambient temperature of 40 degrees C with the operational technic of B. F. Skinner[END_REF] as obtaining or not the reward depends on the bees' actions because it is the decision of the bee to land or not on a given target that defines if the reward is obtained or not. Yet, it also includes Pavlovian associations between the visual stimulus and the reward (or absence of it) following the classic scheme of a CS-US association, and eventually associations between the visual stimulus and the response to be produced.

In 1967, Randolf Menzel initiated the study of color memory using free flying experiments [START_REF] Menzel | Untersuchungen zum Erlernen von Spektralfarben durch die Honigbiene (Apis mellifica)[END_REF]. In these experiments, bees were trained to collect sucrose solution on a horizontal table in which spectral filters illuminated from below provided the color cues to be learned. Bees were trained with one or more learning trials and varying alternative color (nonrewarded) presented adjacent to the rewarded color. The goal was to quantify the color memory resulting from this training for each wavelength trained. Memory retention was tested in extinction conditions (no reward provided) and presenting the rewarded color against a different color to assess the specificity of the color memory acquired. He demonstrated that bees are able to specifically learn the rewarded color and that different spectral colors are learned at different speeds. With violet (413,428 nm) being the fastest and the most reliably learned with up to 85% correct responses during the test, and blue-green (494 nm) being the slowest.

Presenting stimuli horizontally constituted a problem for the study of shape and pattern learning as bees could have only a partial view of a pattern perceived upon their approach. It was therefore decided that presenting the stimuli vertically and frontally would preclude this problem as bees would be forced to see the trained stimuli entirely [START_REF] Wehner | Pattern Recognition in Bees[END_REF]. Since then the study of visual learning in freely flying bees switched to a vertical form of stimulus presentation in the majority of the works that aimed at controlling visual perception properly.

Yet, another problem was realized later: what the bee would perceived depended on its distance to the target, i.e. on the visual angle subtended by the stimulus, which determined or not that the stimulus was resolvable for the bees' visual system [START_REF] Srinivasan | Spatial acuity of honeybee vision and its spectral properties[END_REF].

A way to solve this problem was found by adopting Y-mazes, in which not only stimuli were presented vertically on the back walls, but also which allowed controlling the distance from the stimuli to a decision chamber leading to both maze arms. In this way, the visual angle of the visual stimuli could be controlled [START_REF] Srinivasan | Spatial acuity of honeybee vision and its spectral properties[END_REF][START_REF] Hateren | Pattern recognition in bees: orientation discrimination[END_REF]M. Giurfa et al., 1996). In these experiments, forager bees are first trained to collect a sucrose reward at the end of each of the two arms of an experimental Y-maze. Then, during each conditioning trial, individual bees have to choose between a rewarded and a non-rewarded arm, distinguishable by adequate cues situated at the end of each arm (Fig. 2). Using these various protocols allowed determining that bees can not only discriminate colors but also a large variety of visual attributes such as shapes and patterns, depth, motion, light intensity, contrast and complex configurations [START_REF] Menzel | Untersuchungen zum Erlernen von Spektralfarben durch die Honigbiene (Apis mellifica)[END_REF][START_REF] Zhang | Maze Learning by Honeybees[END_REF]Giurfa and Menzel, 1997;[START_REF] Horridge | Seven experiments on pattern vision of the honeybee, with a model[END_REF][START_REF] Srinivasan | Honey Bees as a Model for Vision, Perception, and Cognition[END_REF]Avarguès-Weber et al., 2011a). Throughout the years free flight has been used extensively to analyze visual memory, discrimination and generalization [START_REF] Zhang | Pattern recognition in honeybees: local and global analysis[END_REF][START_REF] Zhang | Honeybee Memory: Navigation by Associative Grouping and Recall of Visual Stimuli[END_REF]Giurfa, 2004;[START_REF] Zhang | Visual working memory in decision making by honey bees[END_REF]Dyer et al., 2011;[START_REF] Dyer | The mysterious cognitive abilities of bees: why models of visual processing need to consider experience and individual differences in animal performance[END_REF][START_REF] Zhang | Visually Guided Decision Making in Foraging Honeybees[END_REF].

In the last two decades, however, a 'cognitive revolution' took place and visual learning moved to a different level, namely the study of higher order learning capacities in bees, capacities that until then were considered to be absent in the miniature brains of insects despite the fact that studies had already documented the ability of honeybees to generalize among visual stimuli [START_REF] Horridge | Generalization in visual recognition by the honeybee (Apis mellifera): A review and explanation[END_REF].

It started with the demonstration of symmetry categorization in bees [START_REF] Giurfa | Symmetry perception in an insect[END_REF]. Categorization consists in grouping together stimuli that are recognized as explicitly different but which are classified as similar based on shared attributes. Any unknown exotic bird will be recognized as a "bird" based on the presence of attributes defining this category such as wings, feathers, a beak, etc. In Giurfa et al. study bees were trained to collect a reward from vertically presented stimuli, as described earlier [START_REF] Wehner | Pattern Recognition in Bees[END_REF], that remained constant only in their degree of symmetry. One group was trained to collect reward on symmetrical stimuli and the other on asymmetrical ones. During the test both groups were able to correctly chose the novel symmetrical or asymmetrical stimulus respectively. Bees were thus able perceive the bilateral symmetry and generalize it to novel stimuli.

Later, free flying bees were studied for their capacity to learn conceptual relationships, meaning concepts that rely on relationships between stimuli rather than on physical features of the stimuli [START_REF] Zentall | Categorization, Concept Learning, and Behavior Analysis: An Introduction[END_REF]Avarguès-Weber and Giurfa, 2013). One particular protocol possible to test this capacity is the 'delayed matching-to-sample' task, in which an animal is presented with a 'sample' and subsequently with two or more secondary stimuli, one of which is identical to the sample. The animal is required to respond to the stimulus just encountered, i.e. to use the relational rule 'choose the same as the same previously seen', irrespective of the nature of this sample. The 'delayed non-matching-to-sample' is similar to the matching-to-sample task except that the animal is required to respond to the stimulus that is always different from the sample.

In both cases, broadly construed sameness and difference concepts are shown only if the animal exhibits positive transfer to a completely new set of stimuli, which it had not experienced during training [START_REF] Giurfa | The concepts of 'sameness' and 'difference' in an insect[END_REF]. This capacity was shown by Giurfa and collaborators, who trained honeybees, A. mellifera, in a delayed matching-to-sample paradigm to examine whether they could form a concept of sameness and a concept of difference [START_REF] Giurfa | The concepts of 'sameness' and 'difference' in an insect[END_REF]. In the sameness version, each bee entered the maze by flying through a hole in the middle of an entrance wall. At the entrance, the bee encountered the sample stimulus. The sample was one of two different stimuli, A or B, alternated in a pseudo-random sequence. The entrance led to a decision chamber, where the bee could choose one of two arms. Each arm carried either stimulus A or stimulus B as secondary stimulus. The bee was rewarded with sucrose solution only if it chose the stimulus that was identical to the sample. If the bees managed to learn the original discrimination, they were tested with a new sample and secondary stimuli in 'transfer tests' in extinction conditions (no reward provided): the bees had to choose between stimuli C and D, when the sample was either C or D. In such tests, bees that had been trained to match colors could match achromatic gratings and bees that had been trained to match achromatic gratings could match the colors with a success rate of about 70%, demonstrating thereby the capacity to learn the concept of sameness. Similar experiments demonstrated also the capacity to learn the concept of difference. Further work demonstrated that bees can also handle concepts such as above/below (Avarguès-Weber et al., 2011b), left/right and even process and learn both difference and spatial relationships at the same time (Avarguès-Weber et al., 2012a). More recently the Y-maze was used to investigate numerical cognition in free-flying honeybees. It was shown that bees are endowed with numeric competence and can count visual items until 4 or 5, different results were obtained in different experiments [START_REF] Chittka | Can honey bees count landmarks?[END_REF][START_REF] Dacke | Evidence for counting in insects[END_REF][START_REF] Gross | Number-Based Visual Generalisation in the Honeybee[END_REF][START_REF] Skorupski | Counting insects[END_REF]. Bees could also manage some basic level of addition and subtractions [START_REF] Howard | Numerical cognition in honeybees enables addition and subtraction[END_REF]. In this study honeybees were trained to enter a Y-maze and view a visual sample stimulus presented vertically containing a set of elements of a given color. The color defined the arithmetic operation to perform once in the maze. For instance, if three blue items were shown at the entrance of the maze, the blue color indicated addition of one; therefore, the bee entering the maze should choose a stimulus displaying four items and not two or five. If the items at the entrance were yellow, the arithmetic operation to perform was subtraction of one. Thus, if three yellow items were shown at the entrance, choice of two within the maze represented the correct option. The color of the elements, and thus the arithmetic problem to be solved, was randomly assigned per bee for each trial. Correct and incorrect options during experiments ranged from one to five elements, and the incorrect option could be higher or lower than the correct option (which also included the sample number as a possible incorrect option). The sample number of three elements was never shown during training and was only used as a novel sample number during testing.

Flying bees are able to perform very difficult tasks when flying through unknown environments relying mainly on the optic flow cues generated by their own motion [START_REF] Horridge | The evolution of visual processing and the construction of seeing systems[END_REF]. Optic flow is the speed of movement of an image on the retina, it can be used as a proxy of distance as object that are close appear to be moving faster than object that are further away. In 1989 Kirchner and Srinivasan trained bees to receive a food reward at the end of a tunnel where each wall displayed a pattern of vertical black bars and white gratings, to create a texture that would produce optic flow on the retina of the bees, one of the gratings could be moved either in the direction of the flight or against it, to reduce or increase the optic flow respectively [START_REF] Kirchner | Freely flying honeybees use image motion to estimate object distance[END_REF]. They showed that bees flied in the middle of the tunnel when the optic flow was equivalent on both sides but when the grating was moved in the same direction as the bees' flight, thus reducing optic flow, bees flied closer to the moving grating. Respectively when optic flow was increased bees flied further away from the moving grating. They thus concluded that honeybees use optic flow as a measure of object distance and use this measure to avoid collisions during flights. Since then it was shown that optic flow is also used to control speed, height, and to avoid lateral obstacle (Srinivasan et al., 1991;Baird et al., 2006;[START_REF] Baird | Visual flight control in naturalistic and artificial environments[END_REF]Baird et al., 2021). The impressive abilities of bees to fly through complex environment using optic flow to guide them has been reproduce in biomimetic robots that was shown to reproduce bees ability to avoid collision in narrow corridors and adjust their speed in wind condition [START_REF] Roubieu | A biomimetic vision-based hovercraft accounts for bees' complex behaviour in various corridors[END_REF], thereby showing that managing optic flow is sufficient to reproduce the honeybee flying abilities in flight corridors (Fig. 3). Free flights experiments have proven to be a very powerful tool in the investigation of honeybees' visual cognition and allowed to uncover the bee incredible learning abilities and cement it as a major model organism in neuroscience. However, because free flight experiments take place in the field they lack the finer control that lab experiments can allow and more importantly, they preclude the use of invasive methods to study the mechanisms of these intriguing performances.

Experiments with free-walking bees

The visual learning of honeybees can also be studied in more controlled paradigms in which bees walk, generally within reduced setups which force the animals to walk instead of flying.

For instance, Zhang et al [START_REF] Zhang | Eye-specific learning of routes and "signposts" by walking honeybees[END_REF] trained bees to walk through a narrow tunnel carrying visual stimuli on the two walls to study their capacity to learn routes based on visual stimuli presented to a single eye, and to then navigate these routes using the other (naive) eye.

Bees reaching the end of the tunnel had to turn right or left, one of these choices being correct and leading to reward while the other not. Using the narrow tunnel for a walking bee ensured that what was presented in the lateral walls of the tunnel was the only visible cue to a given eye.

They found that stimuli encountered by different eyes could be associated with different routes and that bees could learn to associate a color with a turning direction based on monocular cues.

A similar approach was used by Menzel [START_REF] Menzel | Serial Position Learning in Honeybees[END_REF] who trained bees to turn either left or right in a narrow T-maze depending on the sequence of colors (blue, yellow) experienced at four positions in the access arm. In this way he aimed at studying the learning of sequential visual configurations as predictive of reward. The results showed that visual cues differed in their capacity to predict reward when presented alone in a test at one of the four positions of the access tunnel. The position closest to the maze branching had the highest predictive value while that at the entrance of the maze had the lowest value. Thus, the four positions were equipped with different salience scores, which reflected probably their contiguity to reward, and which added up independently, although in some tests configuring of sequential patterns was also observed.

Walking setups have been also used to study aversive learning as they offered the possibility of delivering punishment (e.g. electric shock) via the tarsi of the walking bees. For instance, Nicholas H. Kirkerud and collaborators established an automated setup to study a passiveavoidance task that they called APIS, the Automatic Performance Index System [START_REF] Kirkerud | APIS-a novel approach for conditioning honey bees[END_REF]. It's an enclosed walking channel where the interior is covered with an electric grid, and where presentation of odors from either end can be combined with weak electric shocks to form aversive associations. To quantify behavioral responses, the movement of the bee is monitored by an automatic tracking system. Number of escapes from one side to the other, changes in velocity as well as distance and time spent away from the punished odor are measured to describe the bee's learning capabilities [START_REF] Kirkerud | APIS-a novel approach for conditioning honey bees[END_REF]. This setup was then adapted for color learning, where one half of the assay is illuminated with one color paired with electric shock and the other half was illuminated with light of a different wavelength and not paired with shocks. The unrestrained bee could run away from the light stimulus and thereby associate one wavelength with punishment, and the other with safety (Fig. 4a) (Kirkerud et al., 2017). A similar setup (Fig. 4b) was used by Agarwal and collaborators to explore the influence of dopamine (DA) and octopamine (OA) on avoidance learning [START_REF] Agarwal | Dopamine and Octopamine Influence Avoidance Learning of Honey Bees in a Place Preference Assay[END_REF]. In this study free walking honeybees had to learn the association between a mild electric shock and a special color cue. After five trials control bees successfully learned the association and stopped going to the color side associated with the shock. OA impaired avoidance learning as OA treated bees spent more time in the shock paired compartment and a lower proportion of insects reached complete avoidance. DA on the other hand improved the learning has treated bees spent less time in the shock paired compartment that control bees. Thus the reward and punishment pathways are inter-connected as OA is known to be involved in motivation, reward, and modulation of motor functions in insects [START_REF] Schwaerzel | Dopamine and octopamine differentiate between aversive and appetitive olfactory memories in Drosophila[END_REF]. Plath et al also used a setup adapted from APIS to explore the role of the central complex and of the mushroom bodies in aversive color learning (Plath et al., 2017). They found that silencing either the central complex or the medial lobes of the mushroom bodies (Fig. 15) impaired the ability of the bee to associate the light field with the shock. On the other hand, inactivating one collar region of the mushroom bodies calyx (Fig. 15.C) did not affect learning in this assay (Plath et al., 2017). Electric shock associated with color lights in a double-chamber setup was also used, yet not for aversive color discrimination learning, as in the cases mentioned above, but for phototactic suppression based on aversive learning. In this setup termed ICARUS bees learned to suppress their spontaneous attraction toward a blue lit compartment in which they would receive a mild electric shock (Fig. 4c) (Marchal et al., 2019). The setup is made of two illuminated chambers, initially both chambers are illuminated in red, not visible to the bee (640 nm), then the chamber not currently occupied by the bee is lit in blue, through positive phototaxy the naive insect will spontaneously enter the lit chamber and receive an electric shock. The delay between the beginning of the illumination and the moment the bee crosses the threshold is recorded and serves as a measure of learning. This protocol was sufficient to induce visual learning as bees were able to successfully repress their spontaneous phototactic response toward the blue light and to create long term memory as honeybees still repressed their attraction to blue light 24 h after the last conditioning trial. Finally, they performed RT-qPCR in individual brains of successfully trained animals focusing on expression levels of the three dopamine-receptor genes Amdop1, Amdop2, and Amdop3. Coherent with Agarwal's results on DA [START_REF] Agarwal | Dopamine and Octopamine Influence Avoidance Learning of Honey Bees in a Place Preference Assay[END_REF], found an upregulation of the dopaminergic receptor gene Amdop1 in the calyces of the mushroom bodies as a result of the conditioning (Marchal et al., 2019). Both APIS and ICARUS give more control over the timing of the experiments than under free flight conditions because the whole experiment can take place in the lab and the bees do not return to the hive. It is then possible to perform more invasive experiments like local inactivation of brain structures, using neuropharmacological blockade of target receptors [START_REF] Agarwal | Dopamine and Octopamine Influence Avoidance Learning of Honey Bees in a Place Preference Assay[END_REF], transiently inactivating specific brain structure with anesthetic [START_REF] Plath | Different roles for honey bee mushroom bodies and central complex in visual learning of colored lights in an aversive conditioning assay[END_REF], or quantifying relative levels of gene expression in key brain structures following aversive learning (Marchal et al., 2019).

In order to study appetitive conditioning in more controlled conditions than free flight it is possible to adapt the Y-maze used with free flying honeybees to a semi-restraining version in which bees move freely within a miniature maze but do not return to the hive in between trials (Buatois et al., 2018;[START_REF] Bestea | The short neuropeptide F (sNPF) promotes the formation of appetitive visual memories in honey bees[END_REF]. Such a maze had detachable end sections that could be closed and moved to the start of the maze after each choice. In this way, a bee was forced to do consecutive choices by translocating it repeated times to the start of the maze. This setup was used, among others, to control for the effect of manipulated appetitive motivation, which would change dramatically if the bee returns to the hive and unload the collected food [START_REF] Bestea | The short neuropeptide F (sNPF) promotes the formation of appetitive visual memories in honey bees[END_REF].

Thanks to those protocols it has been possible to investigate further the underlying mechanisms of visual learning by coupling controlled behavioral experiments with neural analyses of targeted regions of the brain. However, because the insect can move freely, the analyses performed are relatively crude as they proposed 'static' views of neural activation (via gene expression analyses) or because they targeted broad areas via pharmacological blockade. In all cases, what is missing is the possibility to couple the study of behavioral performances with an online recording of neural activity, which is of fundamental importance to characterize the neural signature of visual learning and memory.

Visual learning under full immobilization: conditioning of appetitive and aversive reflexes

One of the most widely used protocol to investigate learning and memory in bees is the appetitive reflex elicited by chemosensory contact of sucrose solution with sucrose receptors located on the antennae and the tarsi [START_REF] Minnich | An experimental study of the tarsal chemoreceptors of two nymphalid butterflies[END_REF][START_REF] Frings | The loci of olfactory end-organs in the honey-bee, Apis mellifera Linn[END_REF][START_REF] Frings | The Loci of Contact Chemoreceptors in Insects. A Review with New Evidence[END_REF]. This response termed PER (Proboscis Extension Reflex) has been used extensively to study olfactory learning and memory (Giurfa, 2007). PER conditioning as also been adapted to other hymenoptera like bumblebees (Laloi et al., 1999;Riveros and Gronenberg, 2009). As described earlier, individually harnessed bees learn the association between a neutral odorant and a sucrose reward such that the odorant ends up eliciting PER (Fig. 5). [START_REF] Scheiner | Standard methods for behavioural studies of Apis mellifera[END_REF].

The PER protocol was originally conceived to study visual learning. In its first version, published in 1957, Kuwabara introduced the protocol to study Pavlovian visual learning. Yet, he realized that learning would be only possible if the antennae of the bee were sectioned. He mentioned that this procedure was necessary to allow the acquisition of color-reward associations and consequent color-dependent PER response because restrained bees with intact antennae apparently developed unspecific PER responses to the water vapor from the small spoon used to deliver sucrose solution as reward [START_REF] Kuwabara | Bildung des bedingten Reflexes von Pavlovs Typus bei der Honigbiene[END_REF]. Fifteen years after Kuwabara's original report, another study reported results using visual-PER conditioning of honeybees with intact antennae. But it required up to 50-60 training trials to achieve learning [START_REF] Masuhr | Learning Experiments on the Use of Side -Specific Information in the Olfactory and Visual System in the Honey Bee (Apis mellifica)[END_REF]. Another study reported no significant acquisition of visualinduced PER in bees with intact antennae during a 6-trial pre-training phase in which bees had to associate a white-light stimulus with sucrose reward (Gerber and Smith, 1998). Only later, the group of Takeo Kubo reproduced Kuwabara's results on visual PER conditioning but sectioning the antennae again [START_REF] Hori | Associative visual learning, color discrimination, and chromatic adaptation in the harnessed honeybee Apis mellifera L[END_REF][START_REF] Hori | Associative learning and discrimination of motion cues in the harnessed honeybee Apis mellifera L[END_REF]. The first study compared visual learning performances of antennae-deprived and intact restrained bees, and found that only antennaedeprived bees were able to acquire significant visual-induced PER after 20 trials of classical conditioning using green (540 nm) or red light (618 nm) as conditioned stimulus [START_REF] Hori | Associative visual learning, color discrimination, and chromatic adaptation in the harnessed honeybee Apis mellifera L[END_REF] or, in a different work, motion cues which could simulate backward or forward optic flow [START_REF] Hori | Associative learning and discrimination of motion cues in the harnessed honeybee Apis mellifera L[END_REF]. Red light was used "to activate exclusively the L-receptor type", thus as a case of achromatic visual conditioning. In the case of motion cues, bees were able to be conditioned equally well to forward and backward movement, and were able to specifically respond to the conditioned motion as bees conditioned to backward motion responded significantly more (44.4%) to backward than to forward motions (14.8%) [START_REF] Hori | Associative learning and discrimination of motion cues in the harnessed honeybee Apis mellifera L[END_REF].

More recent works found that intact restrained honeybees can acquire visually-induced PER responses both in absolute and differential visual conditioning paradigms [START_REF] Dobrin | Visual Associative Learning in Restrained Honey Bees with Intact Antennae[END_REF][START_REF] Sakura | Evidence for instantaneous e-vector detection in the honeybee using an associative learning paradigm[END_REF][START_REF] Jernigan | Color-dependent learning in restrained Africanized honey bees[END_REF]Balamurali et al., 2015). Contrary to previous studies [START_REF] Hori | Associative learning and discrimination of motion cues in the harnessed honeybee Apis mellifera L[END_REF][START_REF] Hori | Associative visual learning, color discrimination, and chromatic adaptation in the harnessed honeybee Apis mellifera L[END_REF]T. Mota et al., 2011), some of these authors found that antennae amputation even impaired visual-PER acquisition [START_REF] Jernigan | Color-dependent learning in restrained Africanized honey bees[END_REF]. The potential bias due to responses to water vapor perception when training the bees with intact antennae was not discussed in these papers. Moreover, Dobrin and Fahrbach showed reduced discrimination performances when a wet toothpick was presented to the bees in the CS-trials instead of a dry toothpick [START_REF] Dobrin | Visual Associative Learning in Restrained Honey Bees with Intact Antennae[END_REF]. Thus, when the unambiguous presence of water could not be used as an additional predictive factor of reward, the bees' selective responses to the conditioned color dropped significantly suggesting a crucial influence of this factor. Thus, at the present time, the role and potential interference of the antennae on visual PER conditioning, and the causes for this interference remain unclear.

With or without antennae, visual learning performances in PER conditioning have never reached the levels usually observed in free-flying bees trained to visual stimuli, often characterized by fast acquisition rates and high percentages of correct choices (70-100%) at the end of training (M. Giurfa et al., 1996;[START_REF] Dyer | Simultaneous and successive colour discrimination in the honeybee (Apis mellifera)[END_REF]Avarguès-Weber et al., 2010) or the levels that are characteristic of olfactory PER conditioning, which allows reaching a plateau of 80-90% correct choices in the case of a salient odorant (Bitterman et al., 1983;Giurfa and Sandoz, 2012). Also, the number of conditioning trials required to reach a plateau in the learning curve is dramatically different between visual and olfactory conditioning of PER.

While few trials (usually three to five) are required for successful olfactory conditioning of PER (Bitterman et al., 1983;[START_REF] Guerrieri | Perceptual and Neural Olfactory Similarity in Honeybees[END_REF][START_REF] Matsumoto | Revisiting olfactory classical conditioning of the proboscis extension response in honey bees: A step toward standardized procedures[END_REF], a higher number (6 to 20) of trials is required for visual conditioning of PER and acquisition levels are substantially lower (40%) [START_REF] Hori | Associative visual learning, color discrimination, and chromatic adaptation in the harnessed honeybee Apis mellifera L[END_REF][START_REF] Hori | Associative learning and discrimination of motion cues in the harnessed honeybee Apis mellifera L[END_REF].

Therefore, visual PER in its current state does not meet the expectations set by olfactory PER as it is not only inconvenient to use, with high number of trial and antennae removal, but also has poor learning performances.

In order to study aversive visual learning in restrained bees, it is possible to use the sting extension response (SER), which is defensive behavior of bees elicited by potentially noxious stimuli [START_REF] Breed | Defensive behavior of honey bees: organization, genetics, and comparisons with other bees[END_REF]. It is possible to elicit SER in laboratory through the application of an electric shock to the thorax [START_REF] Núñez | Alarm Pheromone Induces Stress Analgesia via an Opioid System in the Honeybee[END_REF]. This lead to the implementation of an aversive olfactory conditioning protocol (Vergoz et al., 2007). Honeybees were fixed individually on a metallic holder so that they build a bridge between two electrodes through which a mild electric shock can be delivered to elicit the SER. A 2s electric shock served as the US and was paired with a 5s odor pulse as CS (Fig. 6) (Vergoz et al., 2007). This protocol was later adapted for visual learning (Fig. 6a) (Theo Mota et al., 2011a) as an alternative to visual PER. Bees were able to discriminate two colors, green and blue, that varied both in their chromatic and achromatic properties, reaching 40% of specific response to the reinforced color after 6 trials. They were also able to discriminate colors that varied only in either their chromatic or achromatic properties with the same percentage of success at the last trial. Lastly the authors showed that not only antennae ablation was not necessary for this conditioning but that it decrease bees' performances (Theo Mota et al., 2011a). However, due spatial constraints (the sting had to stay visible to the experimenter) the visual stimulus could only be projected on one eye, even though we know that bees can learn visual exposure through monocular exposure [START_REF] Masuhr | Learning Experiments on the Use of Side -Specific Information in the Olfactory and Visual System in the Honey Bee (Apis mellifica)[END_REF] this is limitation for further exploration of the underlying mechanism of visual learning. Moreover, Mota and collaborators' study showed a constant unspecific response of about 20% in addition to the low percentage of conditioned response (40%) so visual SER is pretty far from the performances observed with olfactory PER.

Thus visual SER does not meet the expectations set by olfactory PER for reasons similar to the ones mentioned for visual PER. The visual stimulus was presented on a white screen to the right eye of the harnessed bee. Taken from Theo Mota et al., 2011a. (b) Picture of a bee harnessed in a SER setup. Taken from (Tedjakumala and Giurfa, 2013) (c) Picture of a sting extension. Taken from (Tedjakumala and Giurfa, 2013) Overall, it is forceful to conclude that protocols with fully immobilized animals have not yielded satisfactory results for visual conditioning in honeybees. This cannot be related to the full immobilization as a stressful component as bees learn efficiently odorants under the same experimental conditions. One possible explanation is that full immobilization precludes the possibility of active vision, i.e. the active extraction of visual information from the environment by means of an animal's movements and sensing. Thus, an experimental approach in which immobilization exists but leaves some degrees of freedom for active sensing of visual stimuli could be a way to overcome the problems mentioned above.

Virtual Reality: an innovative approach to study visual learning in bees

Virtual reality (VR) is a scenario built on the basis of artificial sensory stimuli, often generated by computer, that gives the feeling of immersion in an actual world as it allows moving and interacting within that environment while being in fact stationary in the real world.

For walking insects the development of VR systems started more than 70 years ago with a pioneering setup published by Bernhard Hassenstein to characterize for the first time optomotor responses in beetles [START_REF] Hassenstein | Wandernde geometrische Interferenzfiguren im Insektenauge[END_REF]. This setup was not a true VR in the sense that it did not create a visual environment for the beetle under study. Yet the visual panorama was coupled to the beetle's movements. The insect was tethered onto a very lightweight 'Y-maze globe' made of thin straws, which turned below the beetle as the beetle 'walked' along a blade of straw, thus repeatedly confronting the beetle with Y-maze choices of diverging straws. The tethered beetle could then be exposed to highly controlled, moving visual stimuli, namely a cylinder with vertical black and white stripes, allowing the simultaneous recording of its directional choices on the globe in response to the moving stripes [START_REF] Hassenstein | Ommatidienraster und afferente Bewegungsintegration[END_REF].

Later a flight simulator was built by Götz to study the optical properties of the compound eye of Drosophila melanogaster. It consisted of a torque meter suspended in the middle of a cylinder with textured walls. The fly was suspended by its thorax to the torque meter which was thus able to measure the rotations of the fly as it reacted to the movements of the cylinder walls.

It was an open loop setup where the fly could react to the stimuli presented to it but could not control their movements (Fig. 7) (Götz, 1964). Measuring the optomotor response of the flies to movement of the walls, Götz was able to show that the perception of motion depends only on the temporal, not on the spatial phase relations between periodic intensity variations in neighboring ommatidia. Götz, 1964 Nowadays thanks to the advancement of computers, the range of virtual realities that could be generated became virtually infinite, ranging from simple shapes and colors to impossible world with non-Euclidian geometries. Thus new approaches and systems were produced to test a wide variety of hypotheses on bee visual behavior.

Studying navigation and attentional processes

VR for honeybees made its first significant start with a flight simulator build by Luu et al [START_REF] Luu | Honeybee flight: A novel "streamlining" response[END_REF]. A tethered bee was suspended in the middle of four LCD monitors that displayed a moving panorama. They could thus study the behavioral response of bees to being passively exposed to a moving panorama, as a way to examine whether and how optic flow affects body posture during flight. The authors were able to make the tethered bees fly in these experimental conditions and noticed that, upon such suspended flight, bees slightly raise their abdomen, a response that is interpreted as a 'streamlining response' (Fig8), presumably to reduce aerodynamic drag. This response was elicited by pure visual exposure [START_REF] Luu | Honeybee flight: A novel "streamlining" response[END_REF] and was strongest when the image motion was in the direction that would be experienced during forward flight and when it covered the full visual panorama of the bee. It was highly sensitive in the lateral rather than in the frontal and rear fields, and it was also modulated by air-flow cues simulating head-wind [START_REF] Taylor | Vision and air flow combine to streamline flying honeybees[END_REF]. (62 × 32 × 42 cm), with a LCD monitor at the floor of the arena. The movement of the bee where tracked in to two camera placed orthogonally to each other outside the arena. The surface displayed on the monitor could thus be updated according to the movement of the bumblebee thereby creating a virtual floor that could be positioned at an arbitrary height compared to the real floor (Fig. 9a) (Frasnelli et al., 2018). Bumblebees were first trained to feed from a static target, a blue disc, displayed on the floor of the arena before being submitted to various configurations of virtual floor they had to go through to reach the target. The VR displayed an elevated platform above the floor where the target was positioned, the elevated platform had a hole through which the insect could fly to reach the target below.

In order to create the illusion that the elevated platform is above the floor, the authors controlled two parameters: motion cues and occlusion cues. Motion cues are the apparent movement speed of different objects: objects that appear to move fast are close, while slower objects appear to be further away. A typical way to explain that effect is to describe the experience one has when being a passenger on the highway, when looking out the side window, objects near the car appear to move really fast, while objects further away move slower and really distant objects on the horizon appear almost still. Thus, by making the elevated platform move slower than the virtual floor in response to the bee movements, it would appear to be closer to the insect.

Occlusion cues are easier to describe; the closest object should hide the object situated behind it. This setup was sufficient to produce a believable illusion for the bumblebees as they tried to avoid the virtual elevated platform, and slowed down and extended their leg when getting close in an attempt to land on it. The genius of this setup, and one of the real strength of VR in general, is that it allowed to create impossible realities where motion cues and occlusion cues were conflicting. It means that the floor would move slower than the platform (motion cues), but would mask the platform (occlusion clues). The authors were thus able to evaluate what cues takes precedence for the bumblebee flight navigation by creating this conflicting situation (Fig. 9b). The bumblebees flew through the occluding texture and avoided the regions with higher motion speed to reach the target (Fig. 9b), thereby showing that they prioritize motion cues over occlusion clues for flight navigation (Frasnelli et al., 2018). Many setups have also been made to study walking bees (Schultheiss et al., 2017). They are called locomotion compensators, or running spheres, use either a light weight ball suspended on an airflow or a ball controlled by precise servomotors that compensate the movement of the insect to always keep it at the top of the ball. This thus creates an omnidirectional treadmill on which the bee can run indefinitely (Fig. 10). Ball movements can be tracked accurately by appropriate optical mouse sensors (Fig10a) [START_REF] Taylor | Insects modify their behaviour depending on the feedback sensor used when walking on a trackball in virtual reality[END_REF] or a video camera (Moore et al., 2014). This kind of device has been used for more than half a century to study different aspects of insect behavior, in particular stereotyped responses to environmental stimuli (Kramer, 1976). Recent progress in video tracking and computer controlled systems have allowed to present the insect with a visual environment that is directly updated by its movements walking stationary on the treadmill (closed-loop). Paulk et al. (Paulk et al., 2014a) used a variant of such a closed-loop VR setup for studying attention-like processes in tethered walking honeybees. Bees walking stationary in the middle of an LED arena were presented with one or two competing green vertical bars separated by 90° and flickering at different frequencies. The goal was to confront the tethered bee with two competing percepts, which would induce sharing attentional resources between them given the fact that bees tend to fixate either stimulus. The authors were able to combine the recording of behavioral fixation of these stimuli with extracellular electrophysiological recordings of neural activity in different parts of the bee brain, inspired by earlier work on Drosophila [START_REF] Van Swinderen | Competing visual flicker reveals attention-like rivalry in the fly brain[END_REF]. Using this method, it was shown that attention-like processes had a neural correlate at the level of the optic lobes before the bee displayed a behavioral choice. No such correlate was detected at the level of the mushroom bodies probably because of the sparse coding occurring at this level, which renders difficult detecting electrophysiological signals The use of closed-loop instead of open-loop controlled visual stimuli seems to be an important parameter, as it increases the temporal coordination of neural activity in the insect brain (Paulk et al., 2014b[START_REF] Paulk | Closed-Loop Behavioral Control Increases Coherence in the Fly Brain[END_REF]. Closed loop conditions also seem to modulate neural activity as early as the medulla (Rusch et al., 2021). Indeed, when honeybees had behavioral control over the horizontal displacement of the visual scene, a subset of spiking neurons, in the medulla, exhibited increased responses for the duration of the stimulus and before the onset of behavioral fixation, but not during their replay in open loop (Rusch et al., 2021).

The VR setups used are constantly evolving, and each study so far has used slightly different parameters and materials (Schultheiss et al., 2017). Most importantly, the techniques used for visual stimulus presentation have changed from LCD to LED screens, as LCD screens do not allow for easy control over parameters such as color, brightness, light polarization or flicker frequency [START_REF] D'eath | Can video images imitate real stimuli in animal behaviour experiments?[END_REF]. LED bulbs allow precise control, and arrays of bulbs can be adjusted to match the visual resolution of bee eyes [START_REF] Reiser | A modular display system for insect behavioral neuroscience[END_REF]. Another way to display stimuli that is cheaper than LED arrays and more flexible than LED screens is the use of video projectors, which offer similar control over color and brightness and can display images on screens of different shapes such as spheres or cylinders around the insect (Buatois et al., 2017(Buatois et al., , 2018)). The latest development of VR in Drosophila allowed to build setups as cheap as $300 [START_REF] Loesche | An Inexpensive, High-Precision, Modular Spherical Treadmill Setup Optimized for Drosophila Experiments[END_REF]. This setup could probably be adapted to bees by scaling up the treadmill.

Studying associative learning and memory

The first two studies exploring visual learning and discrimination of honeybees walking stationary on a trackball and facing virtual visual stimuli came out in 2017 (Buatois et al., 2017;Rusch et al., 2017). In both cases, a visual projection system displayed different visual stimuli on a semi-cylindrical screen placed in front of the tethered walking bee. After a pre-test assessing naïve preference for the two visual stimuli to be discriminated, bees were trained by pairing one of them with appetitive sucrose solution and the other with aversive quinine solution. Training was performed under open-loop conditions, presenting one stimulus at a time. Thus, the tethered bee had no control over the stimulus displacements on the screen.

Thereafter, bees were tested with the two visual stimuli displayed simultaneously and without reinforcement, to determine whether learning induced a change in the original preference. Buatois et al. showed that when tested in extinction condition (without reward) after 12 trials in open-loop around 60% of bees chose the CS+ while the rest either chose the CS-(20%) or didn't make a choice (20%) thus showing that the insects were able to learn the correct association under open loop condition in the VR setup. They also showed that when presenting only the CS+ or CS-during the training phase it is not possible to record any discrimination between CS+ and CS-as the spontaneous phototactic response of bees leads them to always orientate towards the light (Buatois et al., 2017). Finally they showed that using distilled water or quinine solution as punishment associated with the CS-was more effective to induce learning as bees submitted to either dry toothpick or NaCl solution associated with CS-did not learn the discrimination (Buatois et al., 2017). Those results are really important to guide future conditioning protocol in choosing appropriate US, and suggest strongly to condition bees under close-loop condition, in order to offer them a choice between CS+ and CS-at every trial to have a chance to measure acquisition performance during the learning phase.

Similarly, Rusch et al showed that about 60% of bees were able to learn the association after 12 trials, going further they showed that 6 trials were sufficient to get more than 50% of bees to choose the correct stimulus. Bees were able to learn when CS+ and CS-differed both in shape and colors or when both CS were circle of different colors but not when they were either of the same color or square differing only in color. The authors conclude that the color and shape are learned in non-additive manner as not all combinations of shape-color variation lead to learning and that they should thus be considered carefully when designing an experiment (Rusch et al., 2017).

As mentioned earlier, one of the problems potentially underlying the poor learning performances observed in visual PER conditioning is the restriction of active vision, which precludedfor instanceextracting the borders of objects to better detect their presence. In a further study, the question of the role of active vision in the VR setup described above was analyzed. Buatois et al. (Buatois et al., 2018) realized two transfer experiments in which the bees either learned the association within a miniature Y-maze (described earlier), where they were free to move and explore the visual stimuli projected on the back walls, or while being tethered and walking stationary in the VR setup, where the control of the visual stimuli was restricted to a 2D plane (bees could only displace the stimuli laterally to bring them in front of them or to move away from them). In either case, the transfer consisted in testing bees after their initial learning in the opposite setup (i.e. from Y-maze to VR or from VR to Y-maze).

Approximately 60% of the bees learned the visual discrimination in both conditions. Transfer from VR to the maze improved significantly the bees' performances: 75% of bees having chosen the rewarded stimulus (CS+) continued doing so and 100% of bees having chosen the punished stimulus (CS-) reverted their choice in favor of the CS+. In contrast, no improvement was seen for these two groups of bees during the reciprocal transfer from the Y-maze to VR. In this case, bees exhibited inconsistent choices in the VR setup. The asymmetric transfer between contexts indicates that the information learned in each environment may be different despite the similar learning success. Moreover, it shows that reducing the possibility of active vision and movement freedom in the passage from the maze to the VR impairs the expression of visual learning while increasing them in the reciprocal transfer improves it (Buatois et al., 2018).

These results underline the active nature of visual processing in bees and suggests that current VR systems require more work to increase immersion, like for example by adding looming and optic flow to the virtual environment.

In 2019 Zwaka et al. published the first results showing electrophysiological recordings from higher order neurons of honeybees submitted to a visual differential conditioning using a VR system that consisted of an air-supported spherical treadmill allowing the stationary walking bee in closed-loop to control a visual environment projected onto a cone-shaped screen from above (Zwaka et al., 2019). They then used this setup to record A3 mushroom body extrinsic neurons that are known to change their response properties during classical olfactory conditioning [START_REF] Haehnel | Sensory representation and learning-related plasticity in mushroom body extrinsic feedback neurons of the protocerebral tract[END_REF][START_REF] Filla | Mushroom body extrinsic neurons in the honeybee (Apis mellifera) brain integrate context and cue values upon attentional stimulus selection[END_REF] and receive input from Kenyon cells. These neurons provide GBAergic inhibitory feedback onto the mushroom body calyces and regulate therefore Kenyon cell activity (Rybak and Menzel, 1993;[START_REF] Grünewald | Physiological properties and response modulations of mushroom body feedback neurons during olfactory learning in the honeybee, Apis mellifera[END_REF]Zwaka et al., 2018). They have been found to play a crucial role for non-elementary discriminations in the olfactory domain (Boitard et al., 2015;Devaud et al., 2015a). After conditioning within the VR, a significant increase in response from the recorded units in reaction to the rewarded color was found. Yet, this increase was observed in animals for which no behavioral readout of learning was available, thus raising the question of the signification of this variation in neural activity.

The interest of honeybees as a model for the study of learning and memory resides in the fact that these animals cannot only solve simple discriminations; they can also solve complex visual tasks relying on categorization, conceptual learning or numerosity [START_REF] Giurfa | The concepts of 'sameness' and 'difference' in an insect[END_REF]Avarguès-Weber et al., 2011b, 2012a;[START_REF] Howard | Numerical cognition in honeybees enables addition and subtraction[END_REF]. A form of higher-order learning is the so-called negative patterning discrimination in which a subject has to learn to respond to single stimuli (A, B) but not to their conjunction (AB) [START_REF] Kehoe | Summation and configuration: Stimulus compounding and negative patterning in the rabbit[END_REF][START_REF] Whitlow | Negative patterning in classical conditioning: Summation of response tendencies to isolable and configurai components[END_REF]. The ambiguity of the task resides in the fact that each element (A and B) is as often reinforced (when presented alone) as non-reinforced (when presented as a compound).

Besides, the problem is difficult given the natural tendency to summation upon compound AB presentation; in other words, if A and B were positively reinforced, the prediction is that AB would be twice as good [START_REF] Whitlow | Negative patterning in classical conditioning: Summation of response tendencies to isolable and configurai components[END_REF]. Yet, in this discrimination, individuals have to inhibit this summation response and respond only to the single elements. A visual version of this protocol was established in the VR environment (Buatois et al., 2020). It was shown that honeybees were able to solve a negative patterning task where A and B were green and blue gratings against a dark background, while AB was a green-blue composite grating.

When conditioned with rewarded green grating (A+) and rewarded blue grating (B+) versus the non-rewarded composite green and blue grating (AB-) (Fig11.B), 25% of the bees were able to solve both A vs AB and B vs AB tests and about 60% were able to solve at least one of the two.

The relative low success is explained by the higher complexity of the task compared to associative learning. Nevertheless, a non-negligible number of bees were able to solve the task thereby proving that honeybees can solve a negative patterning task in VR. and composite green-blue grating (AB). Taken from Buatois et al., 2020 VR thus appears to be an appropriate tool for the study of visual learning as it was successfully used for both elemental association (Buatois et al., 2017;Rusch et al., 2017) and non-elemental learning (Buatois et al., 2020). Moreover, it shows great promises for live electrophysiological recording of learning bees (Rusch et al., 2021;Zwaka et al., 2019).

Benefits and caveats of using virtual reality to study visual learning in bees

One of the most important benefits of VR setups is the opportunity to combine controlled behavioral analyses with invasive analyses of underlying neural performances (Paulk et al., 2014b;Zwaka et al., 2019;Rusch et al., 2021), which is really difficult in freely-flying or moving insects [START_REF] Paffhausen | Neural Correlates of Social Behavior in Mushroom Body Extrinsic Neurons of the Honeybee Apis mellifera[END_REF][START_REF] Paffhausen | A Flying Platform to Investigate Neuronal Correlates of Navigation in the Honey Bee (Apis mellifera)[END_REF].

In VR, parameters of interest can be manipulated with great precision and flexibility, changing simple things like shape and color of stimuli (Rusch et al., 2017), or even creating conflict between properties that are impossible in the natural world (Frasnelli et al., 2018). The complexity of possible stimuli can vary greatly, ranging from very simple open-loop presentations (Buatois et al., 2017;Rusch et al., 2017) to naturalistic, immersive, multimodal scenarios in closed-loop (Fig. 12) [START_REF] Kócsi | The Antarium: A Reconstructed Visual Reality Device for Ant Navigation Research[END_REF]. Contrary to the real world, VR is under the complete control of the experimenter, which allows precise control over both the timing of stimuli, and the bees themselves, including their entire sensory exposure over the course of the experiment (Schultheiss et al., 2017). This is very important for gene expression analysis as it allows to normalize the sensory experience across individuals and thus reduce noise and unspecific response in the results. As tethered bees can be kept for long periods if they are fed regularly and controlled for their motivation, this opens up the possibility of studying the neurobiological processes of long-term memory formation. VR systems still present some limitations. Learning success is reduced under such conditions compared to performances of freely-flying bees, i.e. 60% versus the typical 90% to 100% success rates of freely-flying bees trained to discriminate visual stimuli in Y-mazes (Buatois et al., 2017), this might be caused by the tethering that limits free movement and could induce some stress in the insect. Moreover, transfer experiments showed an asymmetry in learning success between real world and virtual reality conditions suggesting an important role of active vision in visual learning (Buatois et al., 2018). Thus, VR setup might need to offer the possibility for insects to actively scan objects with not only closed loop conditions but also adding a third dimension with looming and optic flow from a virtual background. Additionally, under tethered conditions, bees might be lacking some essential proprioceptive input (such as antennal deflections during flight) for complete multisensory integration. Missing mechanosensory input is known to influence responses of Drosophila to visual stimuli [START_REF] Mureli | Cross-modal influence of mechanosensory input on gaze responses to visual motion in Drosophila[END_REF], and is believed to be a cause of 'cybersickness' in human applications of VR [START_REF] Rebenitsch | Review on cybersickness in applications and visual displays[END_REF]. In some bees, tethering may also induce a decrease or a switch from the appetitive motivation necessary for visual training to an escape motivation, which will interfere with learning. Also, when keeping bees tethered for longer periods, a proper control of appetitive motivation is necessary as well as regular checks of normal motor behavior to avoid fatigue effects (Schultheiss et al., 2017). Recent development in VR setups for ants also underlined the need to use a treadmill of appropriate weight in order to make sure that the force required by the insect to move the ball is similar to the force required to move its own weight on a flat surface (Dahmen et al., 2017). This parameter seems to have been mostly overlooked by previous work on honeybees and could be a crucial first step in developing 3D VR as moving the ball forward would require more force than rotating it around its vertical axis.

Finally, one also needs to account for the specific properties of the bee visual system when designing VR setups [START_REF] D'eath | Can video images imitate real stimuli in animal behaviour experiments?[END_REF][START_REF] Fleishman | Some comments on visual perception and the use of video playback in animal behavior studies[END_REF]. Bees have compound eyes with photoreceptor sensitivities peaking in the green, blue and UV regions of the spectrum [START_REF] Backhaus | Color vision in honeybees[END_REF]. Common technology for creating visual stimuli is, however, designed for human vision, in which, for example, yellow will be a blend of green and red. As bees cannot perceive red light, their color perception of such stimulus will be very different [START_REF] Fleishman | Colour perception and the use of video playback experiments in animal behaviour[END_REF]. It is possible to produce images taking into account the properties of insects visual system [START_REF] Vorobyev | FLOWERS THROUGH INSECT EYES[END_REF], and this has been successfully done already in several VR setups [START_REF] Tedore | Using RGB displays to portray color realistic imagery to animal eyes[END_REF][START_REF] Kócsi | The Antarium: A Reconstructed Visual Reality Device for Ant Navigation Research[END_REF]. In addition, bee vision has a high temporal resolution, almost 200 Hz [START_REF] Srinivasan | Temporal acuity of honeybee vision: behavioural studies using moving stimuli[END_REF], which should therefore be the minimum frequency of any VR display system. Flicker frequency is an important parameter as Poll et al. showed that honeybees payed more attention to LEDs flickering at 20-25 Hz, while they avoided higher or lower frequencies respectively) [START_REF] Poll | Using an abstract geometry in virtual reality to explore choice behaviour: visual flicker preferences in honeybees[END_REF].

Using a very detailed VR that provides as much information as possible like a three-dimensional landscape, with polarized light information and optic flow to produce a virtual world as close as possible to the visual setting in a free-flight experiment, could be a way to overcome the VR limitations and produce a "perfect" virtual setting. But doing so would be particularly expensive, in financial terms, but more importantly in terms of the time and technical skills required to build such a setup (Fig. 12). On the other hand, we could investigate the importance of each parameter like optic flow, depth, polarized light etc. on the bees' learning performances in order to design the minimal VR necessary to produce the coherent behavior required for our purpose. This would be giving us more insight into the insect visual behavior and allow the emergence of a simple VR paradigm that could be easily disseminated. Making VR more accessible is important because it would allow teams with limited resources to also be able to explore underlying mechanisms of visual learning.

While it is showing great promises, there's still work to be done in order for VR systems to realize their full potential. The technical bar of entry can be lowered, for example, through the diffusion of open source VR software. And, as suggested by transfer experiments (Buatois et al., 2018), we need a better understanding of the importance of parameters like optic flow on the ability of bees to learn efficiently in VR. It provides above all a valid approach to uncover the neuronal mechanisms of visual learning in bees What do we know of the underlying mechanism of visual learning?

Now that we have established what tools were at our disposal and what work was needed to improve them we need to identify what questions will benefit from the application of those tools. Despite the difficulties mentioned earlier to access the brain of learning honeybees, their vision has been studied intensively in the past decades. By drawing from those experiments and work done on other insects' models, four main regions appear de be involved in visual learning.

From the periphery to the center: the optic lobes, the ventrolateral neuropils, the central complex and the mushroom bodies (Fig. 13) (Ito et al., 2014b).

In this part we'll review the roles of these different structures for visual learning in order to identify good path of exploration for VR experimentation.

The Optic lobes:

The optic lobes are the first level of integration of visual information. It is a relay point for information which arrives from photoreceptors in the compound eyes [START_REF] Kien | Chromatic properties of interneurons in the optic lobes of the bee[END_REF].

There are three types of photoreceptors, S,M, and L (for short-, mid-, and long-range wavelength), peaking in the UV (344 nm), blue (436 nm), and green (544 nm) regions of the spectrum, respectively, which have been identified in the honeybee retina [START_REF] Menzel | Spectral sensitivity of photoreceptors in insect compound eyes: Comparison of species and methods[END_REF][START_REF] Menzel | Comparative Physiology and Evolution of Vision in Invertebrates: A: Invertebrate Photoreceptors, Handbook of Sensory Physiology[END_REF][START_REF] Peitsch | The spectral input systems of hymenopteran insects and their receptor-based colour vision[END_REF]. Photoreceptors are connected to the lamina, the outermost structure of the optic lobes. It is itself connected to the medulla which is connected to the lobula. These three structures from the three layers of the optic lobes [START_REF] Ribi | The first optic ganglion of the bee[END_REF]Avarguès-Weber et al., 2012b).

The lamina is the first visual neuropil in which the axons of the photoreceptors connect to first order processing interneurons, the lamina monopolar cells (LMC) [START_REF] Menzel | Spectral Sensitivity of Monopolar Cells in the Bee Lamina[END_REF]. In honeybees, the lamina was shown to contain mainly neurons exhibiting relatively little response variation across a wide range of wavelengths [START_REF] Menzel | Spectral Sensitivity of Monopolar Cells in the Bee Lamina[END_REF][START_REF] Ribi | The first optic ganglion of the bee[END_REF][START_REF] Kien | Chromatic properties of interneurons in the optic lobes of the bee[END_REF]. This neuropil is made of thousands of optical cartridges, each receiving an axon bundle (containing the nine photoreceptor cell axons) from the overlying ommatidium, as well as the axons of four different types of monopolar cells. The spatial arrangement of photoreceptor axons and LMCs within a cartridge remains constant throughout the lamina, thus retaining the retinotopic organization. The outer chiasm forms the connection between the lamina and the second visual neuropil, the medulla, a structure that contains most of the bee visual system neurons [START_REF] Ribi | The second and third optic ganglia of the worker bee: Golgi studies of the neuronal elements in the medulla and lobula[END_REF].

Fibers coming from the anterior part of the lamina project to the posterior medulla while posterior fibers from the lamina project to the anterior medulla. Thus, the retinotopic organization is retained but reversed in the medulla, which is also organized into a columnar pattern. Medulla columns are highly connected by horizontal fibers (serotoninergic or GABAergic) in contrast with the lamina that has few horizontal connections [START_REF] Ribi | The first optic ganglion of the bee[END_REF][START_REF] Bicker | Chemical Neuroanatomy of the Honeybee Brain[END_REF]. In addition, the medulla exhibits a distal proximal laminated architecture consisting of eight identified layers, oriented orthogonally to the long axis of the columns [START_REF] Ribi | The second and third optic ganglia of the worker bee: Golgi studies of the neuronal elements in the medulla and lobula[END_REF]. Neurons in the medulla already respond with spectral opponency, i.e., with opponent excitation or inhibition depending on photoreceptor-type input [START_REF] Kien | Chromatic properties of interneurons in the optic lobes of the bee[END_REF]. These color-opponent neurons, which exhibit combination-sensitive excitatory and/or inhibitory interactions between two or three photoreceptor classes, represent the principal basis of color vision in honeybees.

The third visual neuropil is the lobula, where columnar stratification and retinotopic organization are preserved mainly in the outer part (Hertel et al., 1987). The inner chiasm forms the connection between the medulla and the lobula, in which the retinotopic organization is again reversed anteroposteriorly. Chromatic properties of neurons in the medulla are preserved and amplified in the lobula, which was also shown to contain distinct color-opponent neurons [START_REF] Kien | Chromatic properties of interneurons in the optic lobes of the bee[END_REF][START_REF] Hertel | Chromatic properties of identified interneurons in the optic lobes of the bee[END_REF]Hertel and Maronde, 1987). Moreover, different types of spatial opponent neurons (i.e., with opponent excitation or inhibition depending on the visual field region or on direction in which the stimulus is presented) were also described in the lobula (Hertel et al., 1987;Hertel and Maronde, 1987).

Inner-layer lobula and medulla neurons, which are more likely to exhibit color-sensitive responses, send projections to anterior brain areas, particularly to the mushroom bodies and the anterior ventrolateral protocerebrum (Paulk et al., 2008;Paulk and Gronenberg, 2008;Paulk et al., 2009a;Dyer et al., 2011). Thus, some of the major visual afferences to the mushroom bodies are color-sensitive [START_REF] Gronenberg | Physiological and anatomical properties of optical input-fibres to the mushroom body in the bee brain[END_REF][START_REF] Mauelshagen | Neural correlates of olfactory learning paradigms in an identified neuron in the honeybee brain[END_REF]Ehmer and Gronenberg, 2002). On the other hand, outer lobula and both inner and outer medulla neurons, project to the posterior protocerebrum (Paulk et al., 2009b(Paulk et al., , 2009a;;Dyer et al., 2011). It seems therefore that achromatic and chromatic pathways are largely segregated in different steps of visual processing in the bee brain. The optic lobes are thus involved in visual processing with the emergence of color vision and shape perception thanks to color opponent and spatial opponent neurons.

The ventrolateral neuropils

In bees, the ventrolateral neuropils can be divided in at least five main regions: the anterior optic tubercle (AOTu), the ventrolateral protocerebrum, the posteriorlateral protocerebrum, the wedge and the posterior optic tubercle (Ito et al., 2014b). Most ventrolateral neuropils receive visual input from the medulla and/or lobula and participate in visual processing (Paulk et al., 2009b). As mentioned before, anteroposterior segregation of achromatic and chromatic processing was found in the input from the medulla and lobula to the ventrolateral protocerebrum of bees (Paulk et al., 2008(Paulk et al., , 2009b(Paulk et al., , 2009a;;Dyer et al., 2011). Moreover, this same gradient of achromatic/chromatic segregation in the anteroposterior brain axis seems to be retained at the level of ventrolateral protocerebrum neurons (Paulk et al., 2009b). The most prominent optic neuropil in the anterior region of the ventrolateral neuropils is the AOTu. The AOTu of bees is compartmentalized in four distinct units (Fig. 14) (Theo Mota et al., 2011b).

The AOTu receives substantial input from the medulla and lobula via the anterior optic tract and send output to lateral accessory lobe via the tubercle accessory lobe tract (Fig. 14).

Furthermore, two distinct tracts interconnect the AOTus of both brain hemispheres: the ventral inter-tubercle tract and the medial inter-tubercle tract (Fig. 14). In addition to these four tracts, a specific neuron provides input from the vertical lobe of the mushroom bodies to the AOTu (Theo Mota et al., 2011b).

Visual information from the dorsal and ventral parts of the bee eye segregate within different AOTu compartments, both at the level of the visual input via the anterior optic tract and of the visual output to the contralateral AOTu via intertubercle tracts (Theo Mota et al., 2011b).

Therefore, visual processing in the AOTu of bees includes a notable spatial component characterized by this dorsoventral segregation.

In vivo calcium imaging revealed that stimulation with distinct monochromatic lights (ultraviolet [UV], blue, and green) matching the sensitivity of the three photoreceptor types of the bee retina induced different signal amplitudes, temporal dynamics, and spatial activity patterns in the AOTu intertubercle network, thus revealing intricate chromatic processing properties. Green light strongly activated both the dorsal and ventral lobes of the AOTu's major unit; blue light activated the dorsal lobe more while UV light activated the ventral lobe more.

Eye stimulation with mixtures of blue and green light induced suppression phenomena in which responses to the mixture were lower than those to the color components, thus concurring with color-opponent processing. These data reinforce strongly the idea that there is a spatial segregation of color processing in the AOTu, which may serve for navigation purposes (Mota et al., 2013).

The central complex

The central complex (CX) comprises a group of neuropils in the center of the insect brain. One important role of the CX is generation of motor outputs according to processed internal and external stimuli [START_REF] Pfeiffer | Organization and Functional Roles of the Central Complex in the Insect Brain[END_REF][START_REF] Plath | Current progress in understanding the functions of the insect central complex[END_REF]. The CX is essential for the initiation and termination of walking, turning and climbing behavior in fruit flies [START_REF] Triphan | Visual Targeting of Motor Actions in Climbing Drosophila[END_REF], cockroaches [START_REF] Martin | Central-Complex Control of Movement in the Freely Walking Cockroach[END_REF] and crickets [START_REF] Kai | Characterization of Locomotor-Related Spike Activity in Protocerebrum of Freely Walking Cricket[END_REF] and is considered as a site for action selection and goal-directed behavior [START_REF] Barron | What insects can tell us about the origins of consciousness[END_REF][START_REF] Fiore | Evolutionarily conserved mechanisms for the selection and maintenance of behavioural activity[END_REF]. A role of the CX in visual learning of spatial features has been shown in various behavioral assays using fruit flies [START_REF] Neuser | Analysis of a spatial orientation memory in Drosophila[END_REF]Ofstad et al., 2011;[START_REF] Kuntz | The visual orientation memory of Drosophila requires Foraging (PKG) upstream of Ignorant (RSK2) in ring neurons of the central complex[END_REF][START_REF] Kuntz | Visual Working Memory Requires Permissive and Instructive NO/cGMP Signaling at Presynapses in the Drosophila Central Brain[END_REF]. The CX is also important for polarized light processing and navigation [START_REF] Pfeiffer | Organization and Functional Roles of the Central Complex in the Insect Brain[END_REF][START_REF] Heinze | Unraveling the neural basis of insect navigation[END_REF].

Using a genetic approach in Drosophila melanogaster, it was shown that the fan-shaped body (FB), the largest component of the central complex, houses a memory trace for the pattern parameter 'elevation', and a memory trace for 'contour orientation' [START_REF] Liu | Distinct memory traces for two visual features in the Drosophila brain[END_REF]. A following study showed that blocking the ellipsoid body, which is another substructure of the CX connected to the FB, interferes with visual pattern memory [START_REF] Pan | Differential roles of the fan-shaped body and the ellipsoid body in Drosophila visual pattern memory[END_REF].

In Cataglyphis noda ants a comparison of neuroanatomical changes in the central complex before and after a learning walks revealed that, under natural light conditions (UV light together with a naturally changing polarization pattern), the CX undergoes a volume increase. While it is not clear whether or not the neuroanatomical changes found in the CX are triggered by appropriate sensory exposure or following the formation of spatial memory, those results still suggest a potential involvement of the CX in visual learning [START_REF] Grob | The Role of Celestial Compass Information in Cataglyphis Ants during Learning Walks and for Neuroplasticity in the Central Complex and Mushroom Bodies[END_REF]. Honeybees with inactivated CX (Fig. 15) were unable to avoid a shock paired light despite not displaying any motor deficit (Plath et al., 2017).

The CX appears to play a role in visual learning in the context of navigation and spatial orientation as is it involved in processing celestial cues like polarized light, which is crucial for azimuthal orientation, but also in pattern recognition which is important to recognize landmarks.

The mushroom bodies

The mushroom bodies contain 170,000 intrinsic neurons called Kenyon cells. At least three sub-populations can be distinguished within these cells: small-type class I cells, large-type class I cells and class II cells, more recently a third type called middle-type Kenyon cells was identified [START_REF] Kaneko | Gene expression profiles and neural activities of Kenyon cell subtypes in the honeybee brain: identification of novel 'middle-type' Kenyon cells[END_REF].

Quantifying the expression of Immediate Early Genes (IEGs) kakusei, Kiya et al. showed that neural activity of a the small-type Kenyon cells is prominently increased in the brains of dancer and forager honeybees. In contrast, the neural activity of the small-and large-type Kenyon cells is increased in the brains of re-orienting workers, which memorize their hive location during re-orienting flights. These findings demonstrate that the small-type Kenyon cell-preferential activity is associated with foraging behavior, suggesting its involvement in information integration during foraging flight, and thus potentially in visual learning (Kiya et al., 2007).

IEGs are gene that are transcribed transiently and rapidly in response to specific stimulations inducing neural activity without de novo protein synthesis (Bahrami and Drabløs, 2016).

Thanks to those properties they offer a good proxy of neuronal activity, in mammals, c-fos, zif268 and Arc are regularly used as such during learning, memory and other forms of cellular plasticity like as long-term potentiation [START_REF] Minatohara | Role of Immediate-Early Genes in Synaptic Plasticity and Neuronal Ensembles Underlying the Memory Trace[END_REF]Gallo et al., 2018;He et al., 2019). In insects, IEGs are used less often as the number of candidate genes serving this goal is reduced and the reliable detection of their expression is sometimes difficult (Sommerlandt et al., 2019). In their 2007 study, Kiya et al identified such a candidate IEG, kakusei, and established a method to use it as a marker of neural activity. The kakusei transcript is localized in the nuclei of neurons and does not encode an open reading frame, suggesting that it functions as a non-coding nuclear RNA (Kiya et al., 2007).

Mushroom bodies are divided into two types of structures: calyces and lobes. The dendrites of Kenyon cells form the calyces and their axons form the pedunculus made up of two lobes [START_REF] Mobbs | The brain of the honeybee Apis mellifera. I. The connections and spatial organization of the mushroom bodies[END_REF][START_REF] Mobbs | Neural networks in the mushroom bodies of the honeybee[END_REF][START_REF] Strausfeld | The mushroom bodies of Drosophila melanogaster: An immunocytological and golgi study of Kenyon cell organization in the calyces and lobes[END_REF]. Calyces are the input region for various types of sensory information while the lobes act as the output of the system. There are two calyces per mushroom body: a median calyx and a lateral calyx. These two structures have direct neuronal connections with the medulla and the lobula (Paulk et al., 2008). They are made up of three sub-parts: the basal ring, the collar and the lip. The collar receives visual input , the lip receives the olfactory inputs and the basal ring receives both [START_REF] Strube-Bloss | Multimodal integration and stimulus categorization in putative mushroom body output neurons of the honeybee[END_REF]. The collar region of the calyx is segregated into five layers that receive alternating input from the dorsal or ventral medulla, respectively. A sixth, innermost layer of the collar receives input from lobula neurons. In the basal ring region of the calyx, medulla neuron terminals are restricted to a small, distal part. Lobula neurons are more prominent in the basal ring, where they terminate in its outer half (Ehmer and Gronenberg, 2002).

The lobes of mushroom bodies are divided into two parts: vertical lobe and median lobe (Ito et al., 2014a), which are interconnected. Information from the calyces joins other structures of the bee brain by passing through these two lobes (Menzel, 1999).

The mushroom bodies have been shown to be involved in both olfactory and visual learning [START_REF] Komischke | Partial unilateral lesions of the mushroom bodies affect olfactory learning in honeybees Apis mellifera L[END_REF]Devaud et al., 2015b;Plath et al., 2017) although their implication in visual learning is less clear. Using the APIS assay described earlier, Plath et al. studied learning performance of bees in which different lobes of the mushroom bodies had been transiently inactivated by microinjection of the reversible anesthetic procaine. Control bees learned to escape the shock-paired light field and to spend more time in the safe light field after a few trials. When medial lobe neurons of the mushroom bodies were silenced, bees were no longer able to associate one light field with shock. By contrast, silencing of one collar region of the mushroom body calyx (Fig15.C) did not alter behavior in the learning assay in comparison to control treatment (Plath et al., 2017). Those results are coherent with previous olfactory experiments that showed that inactivation of mushroom body lobes via the injection of cholinergic antagonist disrupts memory retrieval [START_REF] Lozano | Memory impairment induced by cholinergic antagonists injected into the mushroom bodies of the honeybee[END_REF]. Taken from Plath et al., 2017. Similarly, Kamhi et al. studied the effect of procaine inaction of the vertical lobes (VL) of the mushroom bodies on view-based navigation in ants (Myrmecia midas). Experienced foragers were collected, treated, and released in their familiar environment where their behavior was documented. Animals with procaine-inactivated VLs had tortuous paths and were unable to find their nest, whereas control ants were well directed and were the most successful at returning home. Untreated animals walked faster when their gaze was directed toward home, and this behavior was eliminated by anesthetizing the VL region (Kamhi et al., 2020), thus showing that the mushroom body vertical lobes are necessary for retrieving visual memories.

Contrary to Plath et al. results on calyces, in an ant species (Formica rufa) lesions of the mushroom body calyces rendered the ants unable to go toward a previously trained feeder location (Buehlmann et al., 2020). The discrepancy between those results might be explained by the fact that Plath et al only silenced one collar region of the calyces which might have been compensated by the other three collars. More results suggest that the calyces play a part in visual learning, Li et al found a correlation between success in a visual discrimination task and microglomeruli density in the collar region of the MB [START_REF] Li | A possible structural correlate of learning performance on a colour discrimination task in the brain of the bumblebee[END_REF].

Moreover, when bees are trained in the ICARUS setup (described earlier) to inhibit their spontaneous phototaxis by pairing the attracting light with an electric shock (Marchal et al., 2019), learning induced an increase in the dopaminergic receptor gene Amdop1 in the calyces of the mushroom bodies, consistently with the role of dopaminergic signaling for electric-shock representation in this region of the brain [START_REF] Unoki | Participation of octopaminergic reward system and dopaminergic punishment system in insect olfactory learning revealed by pharmacological study[END_REF]Vergoz et al., 2007;[START_REF] Mizunami | Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect[END_REF][START_REF] Agarwal | Dopamine and Octopamine Influence Avoidance Learning of Honey Bees in a Place Preference Assay[END_REF]Tedjakumala et al., 2013).

In Drosophila melanogaster it was found that inhibition from a single pair of giant GABAergic neurons, the anterior paired lateral (APL) neurons, onto the mushroom bodies (MBs) selectively facilitates behavioral flexibility during visual reversal learning. Indeed, acute disruption of the APL-MB circuit was sufficient to impair visual reversal learning, while flies with dysfunctional APL-MB circuit performed normally in simple forms of visual learning [START_REF] Ren | A GABAergic inhibitory neural circuit regulates visual reversal learning in Drosophila[END_REF].

In honeybees inhibition of the MBs was also shown to specifically impair olfactory reversal learning (Devaud et al., 2007;Boitard et al., 2015). GABAergic inhibitory feedback on the MBs is provided by A3v and A3d neurons [START_REF] Bicker | Mushroom body feedback interneurones in the honeybee show GABA-like immunoreactivity[END_REF][START_REF] Gronenberg | Anatomical and physiological properties of feedback neurons of the mushroom bodies in the bee brain[END_REF][START_REF] Grünewald | Physiological properties and response modulations of mushroom body feedback neurons during olfactory learning in the honeybee, Apis mellifera[END_REF].

Both innervate the output region of the MBs (the lobes) but A3v neurons also feedback onto the input region (the calyces). A3 neurons have been shown to change their response to rewarded and unrewarded visual stimulus after conditioning (Zwaka et al., 2019). And similarly to APL neuron in Drosophila, it was shown that inhibition of GABAergic signaling into the MB calyces impairs olfactory reversal learning, but leaves intact the capacity to perform elemental olfactory conditioning (Boitard et al., 2015;Devaud et al., 2015b). However, inhibition of GABAergic signaling into the lobes instead of the calyces had no effect on reversal learning (Boitard et al., 2015). Even though these experiments were conducted with olfactory conditioning, it is reasonable to expect similar results for vision as centralization of similar brain functions spares the cost of maintaining similar circuit motifs in different brain areas.

Indeed, in Drosophila it was found that the same subsets of dopaminergic MB neurons drive formation of both olfactory and visual memories (Vogt et al., 2014). Furthermore, distinct yet partially overlapping subsets of mushroom body intrinsic neurons were shown to be required for visual and olfactory memories (Vogt et al., 2014). This convergence of different modality might be an evolutionary conserved design of information processing as such converging inputs of different stimuli into a multisensory area have even been described in humans [START_REF] Beauchamp | Touch, sound and vision in human superior temporal sulcus[END_REF]. GABAergic feedback between the lobes (output) and calyces (input) was also shown to be involved in visual context learning and neural error responses following erroneous behavior [START_REF] Filla | Mushroom body extrinsic neurons in the honeybee (Apis mellifera) brain integrate context and cue values upon attentional stimulus selection[END_REF].

Thus the MBs appear to be major integration centers in the honeybee's brain, involved in multiple forms of learning from simple association to more complex reversal learning.

Whole circuit mechanisms

Despite the consequent body of evidence pointing to the MBs as a center for learning and memory, they are not the only structure of the visual system involved in memory formation. A recent study quantifying gene expression kinetic in the brains of honeybees after aversive visual conditioning showed a parallel activation of the optic lobes and the MBs following a similar time course [START_REF] Avalos | Parallel mechanisms of visual memory formation across distinct regions of the honey bee brain[END_REF]. This suggest that sensory neuropils are also involved in associative learning.

In order to explore the implication of the peripheral processing stages and high-order integration centers of the insect brain in visual learning, Yilmaz et al. quantified the volumetric changes in different neuropils directly after color conditioning and, 3 days later, after the establishment of long-term memory (LTM), in Camponotus blandus ants. They found a volume increase of the OLs, the AOTu, and the fan-shaped body (FB) and protocerebral bridge (PB) which are neuropil of the CX, after color learning and LTM formation. They did not find any specific structural change in the MBs [START_REF] Yilmaz | Distributed plasticity in ant visual pathways following colour learning[END_REF], which is coherent with findings in honeybees where no changes of the number of presynaptic buttons in the collar was found after fine color discrimination [START_REF] Sommerlandt | Does Fine Color Discrimination Learning in Free-Flying Honeybees Change Mushroom-Body Calyx Neuroarchitecture[END_REF]. However, those results could be explained by the absence of memory formation in the tested bees as the authors did not test for long term memory. Yilmaz et al. results are also coherent with results mentioned earlier where a role of the VL but not the collar was suggested for aversive color learning after procaine injections in the respective areas (Plath et al., 2017).

A volume increase in the OLs might affect processing of color information at the level of color opponency, the volume of the OLs increased significantly after LTM formation, which may increase and strengthen the excitatory neuronal connections that are relevant for discrimination behavior [START_REF] Yang | Patterns of chromatic information processing in the lobula of the honeybee, Apis mellifera L[END_REF]. These results are coherent with the parallel activation of the OLs with the MB found in honeybees during visual learning [START_REF] Avalos | Parallel mechanisms of visual memory formation across distinct regions of the honey bee brain[END_REF]. Changes in the volume of the FB, suggest it is a potential region in the CX involved in visual memory formation after associative color learning. This is consistent with previous findings in Drosophila that implicate the FB in visual pattern memory formation [START_REF] Liu | Distinct memory traces for two visual features in the Drosophila brain[END_REF][START_REF] Pan | Differential roles of the fan-shaped body and the ellipsoid body in Drosophila visual pattern memory[END_REF].The volume increase of the AOTu found in ants is coherent with results in bees and locusts, where the upper unit of the AOTu has been implicated in the processing of chromatic information [START_REF] Pfeiffer | Coding of Azimuthal Directions via Time-Compensated Combination of Celestial Compass Cues[END_REF]Mota et al., 2013[START_REF] Mota | Synaptic Organization of Microglomerular Clusters in the Lateral and Medial Bulbs of the Honeybee Brain[END_REF].

Thus the MBs appear to be the best candidate for further exploration of the neural basis of visual learning as they have been shown to play a role in both simple associative conditioning (Plath et al., 2017) and more complex reversal learning (Devaud et al., 2007;[START_REF] Ren | A GABAergic inhibitory neural circuit regulates visual reversal learning in Drosophila[END_REF].

However, we can also see that learning actually involves the whole system, from volume modification measured in the optic lobes [START_REF] Yilmaz | Distributed plasticity in ant visual pathways following colour learning[END_REF] to impaired response to learn stimulus by silencing the CX (Plath et al., 2017). As such whole brain analysis of the effect of visual learning on the activation of those different structure, using for example immediate early gene like kakusei (Kiya et al., 2007) as markers, appears like a promising path.

Chapter 1 Motion cues from the background influence associative color learning of honey bees in a virtual-reality scenario

We developed a fully 3D virtual environment, in which tethered bees walking stationary can explore a virtual arena and investigate and learn 3D objects. The presence of the third dimension thus created a more complex and immersive VR in which we studied the incidence of motions cues, induced either ventrally by displacements of the treadmill or frontally by virtual movements in the VR itself, on visual discrimination in the VR landscape.

We expected that the addition of motion cues would increase learning performances either by creating a more immersive experience or simply by enhancing attention by increasing the amount of movement on screen. However, we found that frontal background motion cues impaired color discrimination. Ventral motion cues did not affect color discrimination but influenced walking parameters. In this chapter, we present the various effects of motion cues on visual learning and motor behavior, andprovide potential explanations for their negative impact on color discrimination.

Understanding the spatiotemporal processes that guide decision-making in animals and humans is essential in cognitive research and may be facilitated by virtual reality (VR) 1,2 , which allows generating of immersive spatial environments in well-controlled laboratory settings. In such environments, experiences are simulated based on changes of perceived landscapes or images, which are updated based on the subject's own movements and decisions 1,2 .

Insects have pioneered the implementation of VR paradigms aimed at studying perceptual and cognitive capacities. A predecessor of current VR systems is the flight simulator conceived for the fruit fly Drosophila melanogaster. In this setup, which was first used to study how optical properties of compound eyes influence optomotor reactions 3 , a tethered fly flies stationary in the middle of a cylindrical arena and experiences surrounding visual stimuli that can be updated by the fly's movements. Newer versions of this apparatus are still used for numerous studies on visual learning and memory in flies 4-7 . 'Locomotion compensators' were also developed to study decision-making by walking insects on two-dimensional surfaces. In silk moths and honey bees, for instance, a 'servosphere' -a form of spherical treadmill that compensates every locomotive movement of a walking insectwas first used to study olfactory orientation towards controlled odor stimuli such as pheromone components and odor gradients 8,9 . Spherical treadmills have been used to study multiple behaviors in different insect species. In these setups, the walking movements of the insect under study are constantly monitored and translated into displacements of surrounding visual cues (closed-loop conditions). The insect can be either free 10,11 or immobilized 12-16 by a tether glued onto its body surface (typically on the thorax). In both cases, the insect walks stationary on a treadmill whose movements are recorded by captors placed lateral or ventral to the treadmill.

VR setups are particularly useful for the presentation of visual cues and the study of visual performances. Screens consisting of LED bulb arrays are commonly employed to provide simple forms of visual stimulation ( 17-19 ). In addition, stimuli projected onto screens by highrate video-projectors have also been used on walking arthropods (e.g. 12-14,16,20 ). Furthermore, treadmills holding a tethered animal can also be set in natural visual surroundings to study the influence of landscape features on navigation performances 10 .

Owing to their status of classic models for the study of visual cognition 21-23 , the visual performances of honeybees (Apis mellifera) have recently started to be studied in VR setups.

The main drive to develop these studies was the impossibility to access the neural underpinnings of visual performances in free-flying bees, which have been traditionally used to study basic properties of visual learning and perception 24 . Immobilized bees have been traditionally required for population recordings of neural activity in the bee brain 25,26 , thus precluding the possibility of recording active visually-driven behaviors. VR setups in which a tethered animal makes decision based on visual cues represent a suitable solution to overcome these limitations as they provide access both to behavioral output and to the nervous system of a behaving bee with restricted mobility 27,28 . This perspective is supported by recent developments allowing to record from specific neurons in the brain of walking bees 16,27,29-31 . Yet, the development of VR environments requires considerable work in order to adapt visual displays to the subjective perception of an insect and determine optimal parameters for immersive sensations from an insect's perspective.

Prior work allowed the development of virtual-reality (VR) systems in which a tethered honey bee walks stationary on a spherical treadmill (a Styrofoam ball floating on an air cushion) while perceiving a virtual environment displayed by a video projector onto a semicircular screen 12-16,27 . In most cases, however, the visual stimulation provided consisted of a 2D virtual environment in which only translational image movement (left-right) was coupled to the bees' movements, thus providing an imperfect immersive environment. Despite the absence of depth components, bees learned both elemental (e.g. discrimination between blue vs. green discs or squares) 12-14 and non-elemental discriminations (e.g. the negative patterning problem in which responding to a visual compound, but not to its components, has to be suppressed) 15 , thus showing the suitability of VR for the study of visual learning.

Here we introduce an improved version of our prior VR setup in which a custom-made software allowed us to create a 3D virtual landscape in which bees move and learn to discriminate visual stimuli. This modification introduced depth perception estimated via the optic flow generated by the bee's own movements as a new variable, whose influence on the visual discrimination needs to be considered. In this new scenario, motion cues were not only derived from the targets themselves, but also from the background presented either 'behind' the vertically displayed targets or ventrally, on the walking surface. We therefore studied if and how the addition of these motion cues to our VR setup affects learning and discrimination in tethered bees.

Materials and methods

Study species and collection

Honey bee foragers (Apis mellifera) were obtained from the CRCA apiary located in the campus of the University Paul Sabatier. Foragers were captured at gravity feeders providing 0.88 M sucrose solution upon landing and before they began feeding. This step is important as it ensures that only bees with the appropriate appetitive motivation were brought to the laboratory for the visual learning experiments. Captured bees were enclosed in individual glass vials and then transferred to small cages housing ten bees in average; where they had access to ad libitum water and 300 µl of 1.5 M sucrose solution. They were then kept overnight in an incubator at 28°C and 80% humidity. On the next day, each bee was cooled on ice for 5 minutes to anesthetize it and attach it to its tether. Bees were handled under red light, which ensured a dark environment to the insects.

Tethering procedure

Each bee was tethered by means of a 0.06 g steel needle, 0.5 mm in diameter and 40 mm in length, which was fixed to the thorax by melted beeswax. The needle was placed within a glass cannula, 1 mm in diameter, which was held within a black plastic cylinder, 1 cm in diameter and 55 mm in length, which was fixed on a holding frame placed above the treadmill (Fig. 1A,B). This system allowed the bee to adjust its position in the vertical axis once set on the ball, but did not allow rotational movements. The holding frame consisted of a vertical black, plastic half frame made of two vertical rectangular supports, 105 mm in length, connected to an upper, horizontal rectangular support, 120 mm in length. The latter held the black cylinder in the middle (Fig. 1B). After being attached to its tether, each bee was placed on a small (49 mm diameter) Styrofoam ball for familiarization to a provisory set-up and provided with 5 μl of 1.5 M sucrose solution. Each bee was held for 3 h in this provisory setup, which was kept in the dark and without visual stimulations.

Virtual reality set-up

The bee was then moved to the VR setup to be trained and tested in a 3D visual environment.

To establish this environment, we used a custom software developed using the Unity engine (version 2018.3.11f1), open-source code available at https://github.com/G-Lafon/BeeVR. The software updated the position of the bee within the VR every 0.017 s.

The VR apparatus consisted of a spherical Styrofoam ball, which acted as a treadmill onto which a stationary bee walked while perceiving an artificial visual landscape displayed in front of it on a semi-circular screen (Fig. 1A). The ball was 50 mm in diameter and weighted 1.07 g (Fig. 1B). The ideal weight (M) for spheres holding insects walking on locomotion compensators was suggested to be 10 Msphere = 2.5*Manimal , which in case of a honeybee weighting in average 0.09 g yields a sphere weight of 0.23 g. Despite the fact that our sphere was about 5 times heavier, the bees used in our experiments walked on it without noticeable problems. The ball was positioned within a 3D-printed, hollow, cylindrical support (cylinder: 50 mm high, 59 mm diameter). The cylinder allowed distributing an upwards air flow of 33 L.min -1 produced by an AquaOxy 2000 aquarium pump, and released through a small hole at the base of the cylindrical support. The Styrofoam ball floated on the resulting air cushion and the tethered bee walked on it while remaining stationary. If the bee moves forward, the ball moved backwards and if it intended to turn to the right or the left, the ball moved to the left or the right, respectively. The ball was white and unmarked (Fig. 1B,C) except in the experiment where the influence of the ventral optic flow was tested. In this case, we compared the bees' performance using a white ball (Fig. 1C) and a ball displaying a black and white checkered pattern made of 7 mm 2 squares (Fig. 1D). The movements of the ball, and thus the walking behavior of the bee (i.e. speed, orientation and location in the virtual environment), were recorded by two infrared optic-mouse sensors (Logitech M500, 1000 dpi) placed at a distance of 7 mm from the sphere and forming an angle of 90° angle relative to each other (i.e. 45° from the bee body axis; see Fig. 1A). E) but in this case, the vertical background of the VR arena was covered by a vertical grating made of black and reddish bars. Depending on its movements, the background gave origin to three different conditions: the 'Vertical Grating -Optic Flow Condition', in which the grating was set in closed loop conditions with respect to the bee movements; the 'Vertical Grating -No Optic Flow Condition', in which the grating was moved in synchrony with the bee's gaze so that no motion cues could be derived from the background; and the 'Rotating Vertical Grating Condition', in which the grating was displaced in the anti-clockwise direction across the screen at a constant speed, thus generating a constant optic flow that was independent of the bee's movements.

The ball was positioned in front of a half-cylindrical vertical screen, 268 mm in diameter and 200 mm height, which was placed at 9 cm from the bee. The screen was made of semitransparent tracing paper, which allowed presentation of a 180° visual environment to the bee (Fig. 1A). The visual environment was projected from behind the screen using a video projector connected to a laptop (Fig. 1A). The video projector was an Acer K135 (Lamp: LED, Maximum Vertical Sync: 120 Hz, Definition: 1280 x 800, Minimum Vertical Sync: 50 Hz, Brightness: 600 lumens, Maximum Horizontal Sync: 100.10 3 Hz, Contrast ratio: 10 000:1, Minimum Horizontal Sync: 30.10 3 Hz). The lag between the motion of the bee and the update of the visual surrounding was measured by a high-speed camera at 1000 fps (Canon RX10 mkIII). The VR display started as usual and the hovering motionless ball was quickly moved by hand. A highspeed video containing the ball, the hand and the VR was shot. The number of frames until the background illumination changed were counted by two researchers independently. This procedure yielded a lag value of 18.00 ± 2.53 ms (mean ± S.E.; n =10).

Experiment 1: choosing the red intensity for the achromatic black-red background gratings Honey bees can perceive a red target in achromatic terms and can discriminate it from black based on its achromatic L-receptor contrast 32,33 . In order to present a vertical, red and black striped background against which a color discrimination had to be achieved, we first performed an experiment to choose the intensity of red that was most appropriate for our background grating. We thus determined the spontaneous phototactic responses of bees towards a vertical red cuboid, which varied in intensity. A red intensity that was high enough to be perceived should induce phototactic attraction.

The cuboid had a 5×5 cm base and 1 m height so that it occupied the entire vertical extent of the screen irrespective of the bee's position (Fig. 2A,left). At the beginning of each trial, it subtended a horizontal visual angle of 6.5° and was positioned either to the left (-50°) or the right (+50°) of the tethered bee. Approaching the cuboid resulted in an expansion of its horizontal extent (1.7°/cm). A choice was recorded when the bee approached the cuboid within an area of 3 cm surrounding its virtual surface and directly faced its center (Fig. 2A, middle and right). Three different groups of bees were tested, each one with a different red intensity (see 20). Table 1 summarizes the conditions of this Experiment as well as those of the subsequent experiments.

Within each group, each bee was subjected to four consecutive tests in extinction conditions. During a test, the bee faced the red cuboid on one side, and no stimulus on the alternative side. We recorded whether the bee chose the red cuboid or the equivalent empty space on the other side (to account for possible stimulus choice from random locomotion paths).

Each test lasted 60 s and the inter-test interval was 10 s.

Experiment 2: the influence of motion cues from a vertical background on color discrimination Having chosen a red intensity for the red and black striped vertical background (Red 100; see above), we trained bees to discriminate between two vertical colored cuboids, one rewarded and the other not (see below). Both cuboids had the same dimensions of the red cuboid employed in the previous experiment. One was blue (RGB: 0, 0, 255, with a dominant wavelength of 446 nm) and the other green (RGB: 0, 51, 0, with a dominant wavelength at 528 nm) (Fig. 1E, Fig. 2B left) (see Fig. S1A). Their intensity, measured at the level of the bee eye, was 161,000 μW/cm² (blue cuboid) and 24 370 μW/cm² (green cuboid). These values were shown to elicit the same level of spontaneous attraction 12,15 . The cuboids were positioned respectively at -50° and +50° from the bee's body axis at the beginning of each trial. As in the previous experiment, approaching a cuboid within an area of 3 cm surrounding its virtual surface followed by direct fixation of its center was recorded as a choice (Fig. 2B middle and right).

The background on which the color cuboids were visible was varied to assess the effect of background motion cues on visual discrimination learning. Four experimental conditions were defined (see Table 1). In the 'Transparent Condition' (N=24), no background was provided and the VR display contained only the two cuboids on an empty dark background (Fig. 1E). The residual light from the empty background had a dominant wavelength of 449 nm and an irradiance of 38 µW/cm². In the 'Vertical Grating -Optic Flow Condition' (N=17), the walls of the virtual arena were covered by a vertical grating made of black (RGB: 0, 0, 0; irradiance: 45 μW/cm²; dominant wavelength 628 nm) and red bars (RGB: 255, 0, 0; irradiance: 1130 μW/cm²; dominant wavelength 628 nm), each subtending a visual angle of 6° (Fig. 1F).

Moving forward increased this visual angle by 0.18°/cm. In the 'Vertical Grating -No Optic Flow Condition' (N=17), the same grating made of black and red bars was used but the VR software moved it in synchrony with the bee's gaze so that no motion cues could be derived from the background. Finally, in the 'Rotating Vertical Grating Condition' (N=17), the same black and red grating was displaced in the anti-clockwise direction across the wall at a constant speed (12 m/s), thus generating a constant optic flow that was independent of the bee's movements.

Experiment 3: the influence of motion cues from a ventral background on color discrimination

In order to test the potential impact of ventral motion cues, we trained bees to discriminate the same two vertical colored cuboids used in the previous experiment under two different conditions. While the vertical frontal background remained the same as in the Transparent Condition of Experiment 2 (Fig. 1E), the treadmill texture was varied between two groups of bees: in one case it was a plain white surface (Fig. 1C; N=29) while in the other case, it was a black and white checkered pattern made of 7 mm 2 squares (Fig. 1D; N=38). While the first condition did not provide ventral optic flow, the second condition provided it (see Table 1).

Training and testing procedure for the Experiments 2 and3 Bees were trained during 10 trials using a differential conditioning procedure (Fig. 2C) in which one of the cuboids (i.e. one of the two colors, green or blue) was rewarded with 1.5 M sucrose solution (the appetitive conditioned stimulus or CS+) while the other cuboid displaying the alternative color (the aversive conditioned stimulus or CS-) was associated with either 60 mM quinine (Experiment 2) 34 or 3 M NaCl solution 35,36 (Experiment 3). The latter was used to increase the penalty for incorrect choices 37 .

At the beginning of the experiment, bees were presented with a dark screen for 60 s.

During training trials, each bee faced the virtual environment with the two cuboids in front of it. The bee had to learn to choose the CS+ cuboid by walking towards it and centering it on the screen. Training was balanced in terms of color contingencies (i.e. blue and green equally rewarded across bees) based on a random assignment by the VR software. If the bee reached the CS+ within an area of 3 cm in the virtual environment (i.e. the chosen cuboid subtended a horizontal visual angle of 53°) and centered it in its front, the screen was locked on that image for 8 s (Fig. 2B). This allowed the delivery of sucrose solution in case of a correct choice, or of quinine or NaCl in case of an incorrect choice. Solutions were delivered for 3 s by the experimenter who sat behind the bee and used a toothpick to this end. The toothpick contacted first the antennae and then the mouthparts while the screen was locked on the visual image fixated by the bee.

Each training trial lasted until the bee chose one of both stimuli or until a maximum of 60 s (no choice). Thus, a single choice (or a no choice) was recorded during each training trial.

Trials were separated by an inter-trial interval of 60 s during which the dark screen was presented. The bees that were unable to choose a stimulus in at least 5 trials were excluded from the analysis. From 216 bees trained in the Second Experiment, 75 were kept for analysis (~35%). From 272 bees trained in the Third Experiment, 67 bees were kept for analysis (~25%).

After the last training trial, each bee was subjected to a non-reinforced test (Fig. 2C) that contrary to training trials had a fixed duration of 60 s. During this test, two variables were recorded: the first choice (as defined above) and the time spent fixating the rewarded and the non-rewarded stimulus. Both variables have been used in prior works performed in our VR setup to characterize test performances as they may reveal different aspects of behavioral performances 12,13,15 . Fixation time (s) was defined as the time spent by each cuboid at the center of the screen (± 2.5 mm) where it was brought by the bee's motor actions. We used a ray-casting approach to determine if the object was there and recorded collisions between a ray following the forward vector of the bee and the center of the object.

Statistical analysis

Statistical analyses were performed using R software 38 . In Experiment 1 (red perception), the first choice of the bees in each test was categorized according to three mutually exclusive categories: Red Stimulus (Red), No Stimulus (NS: choice of the area symmetric to the stimulus position) and no choice (NC). Individual choices were translated into a binomial format (0 or 1) within each category. For instance, a bee choosing the red cuboid was recorded as (1,0,0) for a choice of the red stimulus, choice of the no stimulus and NC, respectively. In Experiments 2 and 3, the first choice in each trial and test was categorized as choice of the CS+, choice of the CS-or no choice (NC). Thus, a bee choosing the CS+ was recorded as (1, 0, 0) for choice of the CS+, choice of the CS-and NC, respectively. Data were bootstrapped to plot the proportion of bees in each category with their corresponding 95 % confidence interval.

Performances were analysed using generalized mixed linear models (GLMM) with a binomial error structure-logit-link function (glmer function of R package lme4) 39 . The independent variables (fixed factors) were the experimental group (Condition), the trial number (Trial; Experiments 2 and 3), the choice category (Choice) and the color of the CS+ when applicable (Color: Blue or Green). Bee ID was included as a random factor to account for the repeatedmeasure design. Several models were run by testing interactions between factors and by dropping each factor subsequently to select the model with the highest explanatory power (i.e. the lowest AIC value). P-values for each factor or interaction were obtained by comparing models. The Tukey method was used for multiple comparisons within the selected model; z values are reported for these analyses. For all experiments, the modeling results are reported in Tables S1 to S3 in Supplementary Information. During the tests of Experiments 2 and 3, we also recorded the time spent fixating the test alternatives (CS+ vs. CS-). Time values were compared using a Wilcoxon signed rank test.

For the acquisition trials, we recorded motor variables such as the total distance walked during a trial, the walking speed, and the tortuosity of the trajectories. Tortuosity was calculated as the ratio between the total distance walked and the distance between the first and the last point of the trajectory connected by an imaginary straight line. When the ratio was 1, or close to 1, trajectories were straightforward while higher values corresponded to sinuous trajectories.

In addition, we analyzed the latency to make a choice starting from the beginning of a trial to the moment in which a choice (either for the CS+ or the CS-) was recorded. NC data were excluded from the latency analysis. The analysis of these continuous variables was done using a linear mixed model (lmer function) in which the individual identity (Bee ID) was a random factor and the experimental condition (Condition) and trial number (Trial) were fixed factors.

For each experimental condition, we represented the bees' cumulative trajectories (CS+ choosing and CS-choosing bees) in terms of heat maps, which show the cumulative coordinates occupied by the bees either during the ten training trials or during the non-reinforced test to which they were subjected. Coordinates were binned into 1 cm². Warmer colors depict locations more frequently occupied (see color bar). The highest frequency is cut down to 10 % of the maximum on the color bar. This was done to decrease the excessive occupancy frequency of the starting point at the expense of other locations, given that it was the same for all bees. While the side of the rewarded stimulus was randomized, it was placed arbitrarily on the left in the heat maps. that was opposite to the red cuboid (see Fig. 2A). NC: no choice. Both the Red-50 and the Red-100 intensities were sufficient to render the red stimulus detectable for honeybees. For subsequent experiments, the Red 100 intensity was chosen.

Results

Experiment 1: choosing the red intensity for the achromatic black and red background In a first experiment, we determined the spontaneous phototactic responses of bees towards a red cuboid (dominant wavelength 628 nm) varying in intensity. Using different groups of bees, we tested three different intensities to define the one that would be sufficient to induce phototactic attraction: Red 10 (RGB: 26, 0, 0; irradiance: 13 μW/cm²), Red 50 (RGB: 128, 0, 0; irradiance: 140 μW/cm²) and Red 100 (RGB: 255, 0, 0; irradiance: 1130 μW/cm²). Each bee was tested along four consecutive tests with the same intensity. The model that best fitted the data included an interaction between the red intensity and the bees' choice (Choice*Intensity: χ 2 = 65.48, df: 4, p < 0.001). There was no significant effect of the test sequence on the bees' choices (Test: χ 2 = 0.002, df:1, p = 0.97). Thus, we pooled the data of the four tests and represented for each intensity the percentage of bees within each category (Red, No Stimulus and No Choice; Fig. 3).

In the Red 10 condition (Fig. 3A; N = 19 bees), the bees did not prefer the red cuboid to the alternative symmetrical area displaying no stimulus (Red 10 vs. NS: 11.8% vs. 6.8% of choices, z = 1.11, p = 0.27). Most of the bees did not choose in this condition (81.6% of the cases: NC vs. Red 10: z = 7.56, p < 0.0001; NC vs. NS: z = 7.54, p = <0.0001). By contrast, both in the Red 50 (Fig. 3B; N = 19) and in the Red 100 condition (Fig. 3C; N = 20), bees preferred the red stimulus to the equivalent area displaying no stimulus (Red 50 vs. NS: 48.7% vs. 2.6%, z = 4.73, p < 0.0001; Red 100 vs. NS: 52.5% vs. 1.3%, z= 4.34, p < 0.0001). The percentage of bees not choosing remained high and similar to that of bees choosing the red cuboid (Red 50 vs. NC: z = 0.000, p = 1.00; Red 100 vs. NC: z = 0.58; p = 0.53). For both intensities, the proportion of non-choosing bees was significantly higher than the choice of the absence of stimulus (Red 50 vs. NS: 48.7% vs. 48.6%, z = 4.73 p < 0.0001; Red 100 vs. NS: 52.5% vs. 47.5%, z = 4.14 p < 0.0001). These results indicate that both the Red-50 and the Red-100 intensities were sufficient to render the red stimulus detectable for honeybees. We therefore chose the Red-100 intensity for the red-and-black gratings used in the subsequent experiment as it was the more salient stimulus from the two that were detectable by the bees. We were confident that Red 100 would not induce higher phototaxis than Red 50 as no differences in attraction existed between the cuboids displaying these two lights (compare Fig. 3 B and 3 C).

Experiment 2: the influence of motion cues from a vertical frontal background on color discrimination Four different frontal background conditions were used to assess the effect of motion cues from the background during color discrimination learning. In the 'Transparent Condition' (N = 24 bees), the blue and green cuboids were displayed against a uniform dark background. In the 'Vertical Grating -Optic Flow Condition' (N = 17 bees), the cuboids were presented against a red-and-black vertical grating, which was coupled to the bee's movements (closed-loop conditions). In the 'Vertical Grating -No Optic Flow Condition' (N = 17 bees), the cuboids were displayed against the same red-and-black grating but motion cues from the background were suppressed by keeping it constantly fixed to the bee's gaze. Finally, in the 'Rotating Vertical Grating Condition' (N = 17 bees), the cuboids were shown against the same red-andblack grating, which was rotated counterclockwise around the virtual arena at a constant speed, thus generating a constant optic flow even when the bee did not move. Learning curves were obtained by recording the percentage of bees choosing correctly the CS+ or the CS-in their first choice, or not choosing any stimulus (NC) during each trial. The best explanatory model of the acquisition performance included a three-way interaction between the condition, the trial number and the bees' choice (χ 2 = 50.11, df:15, p < 0.001) but no effect of the nature of the CS+ was found (blue or green: χ 2 = 0.000, df:1, p = 1). For each background condition, data were thus represented as a CS+ vs. a CS-discrimination irrespective of color identity. In the heat maps, the rewarded cuboid is represented on the left side although its side was randomized along the training sequence.

Discrimination learning during training

When no grating was present in the background and the colored cuboids were displayed against a dark homogeneous background ('Transparent Condition'; Fig. 4A), bees learned to respond more to the CS+ than to the CS-. The interaction between trial number and bee choices was significant (χ 2 = 7.99, df:2, p = 0.02). In the course of the 10 conditioning trials, the percentages of bees responding to the CS+ and that of bees responding to the CS-evolved differently (z = 2.51, p = 0.01), thus showing successful discrimination learning. Moreover, the dynamic of CS+ responding bees was also significantly different from that of the nonresponding (NC) bees (z = 2.17, p = 0.03) while the difference between the dynamic of the CSresponding bees and the NC bees was not different (z = 0.13, p = 0.9). In the corresponding cumulative heat map, a clear V shape is visible, indicating that the bees did interact equally with both sides in the VR and walked towards the cuboids. 17), the cuboids were displayed against the same red-and-black grating but motion cues from the background were suppressed by keeping it constantly fixed to the bee's gaze. C) In the 'Vertical Grating -Optic Flow Condition' (N = 17), the cuboids were presented against a red-and-black vertical grating, which was coupled to the bee's movements (closed-loop conditions). D) In the 'Rotating Vertical Grating Condition' (N = 17), the cuboids were shown against the same red-and-black grating, which was rotated counterclockwise around the virtual arena at a constant speed, thus generating a constant optic flow even when the bee did not move.

When the red-and-black grating was moved in synchrony with the bee's gaze so that no motion cues could be derived from the background ('Vertical Grating -No Optic Flow Condition'; Fig. 4B), bees did not modify significantly their stimulus choice along trials. There was a significant interaction between trial number and bee choices (χ 2 = 13.6, df:2, p = 0.001) but only because of a difference in the dynamic of the NC bees compared to other two categories (NC vs. CS+: z = 3.44, p < 0.001; NC vs. CS-: z = 2.60, p < 0.001). Although the CS+ and the CS-curves seem to indicate color discrimination, no differences between the dynamics of the percentages CS+ and CS-choosing bees could be detected (z =1.20, p = 0.23), probably because of the high overlap in confidence intervals of these curves. The cumulative heat map representing the locations of the bees during their training trajectories shows that, as in the previous two conditions, bees walked and interacted equally with both sides in the VR.

In the 'Vertical Grating -Optic Flow Condition' (Fig. 4C), the closed loop conditions included both the cuboids and the background grating, i.e. the bees' movements translated and expanded not only the cuboids but also the background grating accordingly. The interaction between trial number and bee choices was not significant in this case (χ 2 = 5.16, df:2, p = 0.08).

Contrarily to the previous condition, bees were unable to learn the difference between the CS+ and the CS-as no improvement could be detected along the 10 training trials (z = 0.33, p = 0.74). Only the dynamics of the non-responding bees was significantly lower than that of bees selecting either the CS+ (z = 4.63, p < 0.001) or the CS-(z = 4.33, p < 0.001). The cumulative heat map representing the locations of the bees during their training trajectories shows that, as in the previous condition, bees walked towards the cuboids. This result indicates that despite interacting with the cuboids, bees had their color learning impaired by the addition of motion cues from the background.

Finally, in the 'Rotating Vertical Grating Condition' (Fig. 4D) in which the black-andred grating was displaced at a constant speed irrespective of the bee movements and gaze, a similar pattern than for the 'No Optic Flow Condition' was observed. A significant interaction between trial number and bee choices was found (χ 2 = 11.21, df:2, p = 0.004). Yet, it was again due to differences in the dynamic of the percentage of NC bees vs. the percentages of CS+ and CS-bees (NC vs. CS+: z = 3.11, p = 0.002; NC vs. CS-z =2.71, p = 0.007). The percentage of bees choosing the CS+ and that of bees choosing the CS-did not evolve differently (z = 0.44; p = 0.66). In this condition, the cumulative heat map shows that bees also walked and interacted equally with the two cuboid sides along trials.

Motor and temporal components of bee trajectories during training

We analyzed if and how motion cues from the background affected the displacement of bees during the training trials in our four background conditions (Fig. 5). To this end, we quantified the distance walked, the walking speed and the tortuosity of the trajectories (ratio between the total distance walked and the straight line connecting the first and the last point of the trajectory). We also measured the choice latency in each trial, i.e. the time required to choose a cuboid within a trial.

The distance walked (Fig. 5A) increased slightly, yet significantly, over trials (Trial: χ² = 6.86, df:1, p = 0.009) but was not significantly affected by the background condition (Condition: χ² = 5.34, df:3, p = 0.15). The walking speed (Fig. 5B) also increased during successive trials (Trial: χ² = 172.9, df:1, p < 0.001) and revealed a significant interaction with the background condition (Trial*Condition: χ² = 19.3, df:3, p < 0.001), which was introduced by the Optic Flow Condition. In this case, bees decreased their speed at the end of the training, so that a significant difference was detected against the other background conditions (Trial*Condition: 'Optic Flow' vs. 'Transparent': t = 3.64, p < 0.001, Optic Flow vs. No Optic Flow: t = 3.79, p < 0.001 and 'Optic Flow' vs. 'Rotating Grating': t = 3.47, p < 0.001). This decrease was concomitant with an increase in the proportion of bees not choosing (Fig. 4B) so that it may reveal a reduction in motivation at the end of training in this background condition.

The tortuosity of the trajectories (Fig. 5C) was neither affected by the succession of trials nor by the background condition (Trial: χ² = 0.17, df:1, p = 0.68; Condition: χ² = 3.62, df:3, p = 0.31), thus confirming that the structure of motor patterns was similar across the background conditions. Finally, the analysis of choice latency (Fig. 5D) showed a significant decrease along trials (Trial: χ² = 21.85, df:1, p < 0.001; Fig. 5D), suggesting an improvement in the bee's capacity to navigate in the VR environment. This evolution was independent of the background displayed (Condition: χ² = 1.67, df:3, p = 0.65) but a tendency towards larger latencies was observed for the 'Optic Flow Condition'.

Test Performance

After the end of training, each bee was subjected to a test in which the green and the blue cuboids were presented in extinction conditions (no reinforcement provided). We recorded the percentage of bees choosing correctly the CS+ or the CS-in their first choice, or not choosing (NC) and the time spent fixating the CS+ and the CS-(Fig. 6).

The rewarded color did not affect the first choice during the test (Color: χ² = 0, df:1, p = 1), so that performances could be analyzed irrespective of color identity within each background condition. Only under the 'Transparent Condition' (Fig. 6A), the difference between the percentages of CS+ and CS-responding bees was significant (CS+ vs CS-, z = 2.33, p = 0.02). The difference between the CS+ responding bees and the NC bees was also significant (CS+ vs. NC: z = 2.83, p = 0.005). On the contrary, no difference was detected between the CS-responding bees and the NC bees (CS-vs. NC: z = 0.71, p = 0.48). For the other three background conditions, no significant differences were detected between the percentage of bees choosing the CS+ or the CS-(Fig. 6B Overall, the first-choice data show that a significant discrimination between the CS+ and the CS-occurred in the 'Transparent Condition', i.e. in the total absence of background information.

The analysis of the fixation time confirmed and extended this conclusion (Fig. 6 E-H).

Again, no discrimination learning was observed for the conditions in which motion cues were available from the background; bees spent the same amount of time fixating the CS+ and the CS-both in the 'Optic Flow Condition' (Fig. 6F; Wilcoxon U rank Test: V = 69, p = 0.63) and in the 'Rotating Grating Condition' (Fig. 6H; V = 83, p = 0.78). On the contrary, and consistent with the analysis based on the 1 st choice, bees in the 'Transparent Condition' learned the discrimination between the CS+ and the CS-as they spent more time fixating the rewarded color than the non-rewarded one (Fig. 6E; V = 203, p = 0.049). Interestingly, a significant discrimination was also observed for the 'No Optic Flow Condition' (Fig. 6G; V = 128, p = 0.012), a condition for which the 1 st choice did not reveal significant differences. This result indicates that the reduction of motion cues inherent to the 'No Optic Flow Condition' also favored the occurrence of color learning, in agreement with what was observed for the 'Transparent Condition'.

The heat maps displaying the cumulative locations occupied by the bees' trajectories during the entire test are shown in the bottom of Fig. 6. In these maps, the CS+ is displayed on the right by convention. In the 'Transparent Condition' (Fig. 6A), besides choosing significantly more the correct cuboid upon their first choice and spending more time fixating it, bees consistently walked towards the cuboids and inspected them. This tendency was not visible in the conditions in which motion cues were available from the background (Fig. 6B: 'Optic Flow Condition' and Fig. 6D: 'Rotating Grating Condition'), thus showing the impairment of performances induced by these cues. In the 'No Optic Flow Condition' (Fig. 6C), bees walked towards the cuboids and their choice was slightly biased towards the correct color, in accordance with the longer fixation time elicited by this color. Overall, these results reveal a negative influence of motion cues from the vertical background on visual-discrimination learning under VR conditions. 17), the cuboids were presented against a red-and-black vertical grating, which was coupled to the bee's movements (closed-loop conditions). C) In the 'Vertical Grating -No Optic Flow Condition' (N = 17), the cuboids were displayed against the same red-and-black grating but motion cues from the background were suppressed by keeping it constantly fixed to the bee's gaze. D) In the 'Rotating Vertical Grating Condition' (N = 17), the cuboids were shown against the same red-and-black grating, which was rotated counterclockwise around the virtual arena at a constant speed, thus generating a constant optic flow even when the bee did not move. For one group, the ball was homogenously white (Fig. 1C) so that no ventral motion cues were available to the walking bees ('No Ventral Optic Flow Condition', N = 29 bees). For the other group, the ball presented a black-and-white checkered pattern made of 7 mm 2 squares (Fig. 1D; 'Ventral Optic Flow Condition', N = 38 bees) so that ventral optic flow was available to the walking bees.

Discrimination learning during training

Learning curves were again obtained by recording the percentage of bees correctly choosing the CS+ or the CS-in their first choice, or not choosing any stimulus (NC) during each trial.

Figure 7A,B shows the learning curves obtained under the two ventral optic flow conditions and the cumulative heat map showing equal interaction with the two cuboid sides along trials.

Yet, in this case the model that provided the best fit to the data included a three-way interaction between choices, trial number and color (Color*Trial*Choice: χ² = 64.30, df:7, p < 0.001) but with no significant effect of the type of ball used (Condition: χ² = 0, df:1, p = 1). This shows that the availability of ventral optic flow did not influence the bees' performance when the variable quantified was the stimulus choice and that, on the contrary, a color effect existed. To analyze this effect, we pooled acquisition performances irrespective of the ventral optic flow condition, and represented them in terms of a green vs. blue discrimination (Fig. 7C: blue+ vs. green-; Fig. 7D: blue-vs. green+). In both color conditions, the percentages of bees choosing the stimuli varied along trials (Choice*Trial: Blue: χ² = 48.86, df:2, p < 0.001; Green: χ² = 15.47, df:2, p < 0.001). However, the dynamic of the percentage of bees choosing the CS+ and that of bees choosing the CSdiffered significantly only when blue was the rewarded color. In this case, the percentages of bees responding to blue (CS+) and to green (CS-) along trials differed significantly (Fig. 7C;

Choice*Trial: CS+ vs. CS-: z = 4.88, p < 0.001). In addition, both the percentages of bees responding to the rewarded blue and to the punished green differed significantly from the nonresponding bees along trials (CS+ vs. NC: z = 5.93, p < 0.001; CS-vs. NC: z = 2.62, p = 0.009).

Segregating these data between the blue-rewarded bees that experienced the ventral optic flow condition and those that did not experience ventral optic flow yielded the same result. In both cases, the dynamic of the percentage of bees choosing the blue+ and that of bees choosing green-differed significantly ('Ventral Optic Flow': CS+ vs. CS-: z = 3.62, p < 0.001; 'No Ventral Optic Flow': CS+ vs. CS-: z = 3.31, p < 0.001). When green was the rewarded color, no significant differences in the percentages of bees responding to green+ and that of bees responding to bluewas detected along trials, even if the former tended to be higher than the latter (Fig. 7D; Choice*Trial: CS+ vs. CS-: z = 0.60, p = 0.55). Both percentages were significantly higher than that of bees not responding to any stimulus (CS+ vs. NC: z = 3.58, p < 0.001; CS-vs. NC: z = 3.23, p = 0.001). The same pattern of responses with respect to bees responding to green+ and to blue-was found when analyzing separately the two optic-flow conditions ('Ventral Optic Flow': CS+ vs. CS-: z = 0.23, p = 0.82; 'No Ventral Optic Flow': CS+ vs. CS-: z = 1.16, p = 0.25).

Motor and temporal components of bee trajectories during training

We analyzed the motor performance of bees in the two conditions described above to determine if and how ventral motion cues affected the displacement of bees in the VR setup during the training trials (Fig. 8A-D). The distance walked during the acquisition phase (Fig. 8A) was affected by the presence of ventral optic flow (Condition: χ² = 7.45, df:1, p = 0.006). With the checkered ball, the bees walked less. The walking speed during the acquisition phase (Fig. 8B) was also significantly slower when ventral optic flow was available (Condition: χ² = 6.03, df:1, p = 0. 01) although it increased significantly over trials for both conditions (Trial: χ²=85.20; df: 1, p < 0.0001). The tortuosity of the walking paths (Fig. 8C) decreased over trials (Trial: χ²=7.95, df: 1, p = 0.005) but was unaffected by the ventral optic flow (Condition: χ² = 0.56, df:1, p = 0.45). Finally, the latency before making a choice (Fig. 8D) was stable over trials even if an apparent decrease was observed in the first trials (Trial: χ²=1.97; df: 1, p = 0.16), and was not influenced by the ventral optic flow (Condition: χ² = 0.19, df:1, p = 0.66). Overall, the significant variation in distance walked and walking speed detected between the two conditions shows that bees were not insensitive to the presence of ventral motion cues. They perceived them and in consequence walked slower and less.

Test Performance

After the end of training, each bee was subjected to a test in which the green and the blue cuboids were presented in extinction conditions (no reinforcement provided) in the presence or absence of ventral optic flow. We recorded the percentage of bees correctly choosing the CS+, the CS-or not choosing (NC). There was no significant effect of the ventral optic flow on test performances when the variable considered was the choice made by the bees (Condition: χ² = 0, df:1, p = 1). Thus, the results of both groups of bees were pooled (N = 67) and shown as a single graph (Fig. 9A). In this case, the color of the CS+ did not affect the performance (Color: χ² = 0, df:1, p = 1), thus showing that the color effect detected during training was not consistent.

In the test, bees preferred the CS+ over all conditions (CS+ vs. CS-: z = 2.41, p = 0.02; CS+ vs. NC: z = 5.03, p < 0.0001; CS-vs. NC: z = 3.16, p = 0.002), thus confirming that they had learned the color discrimination during acquisition. The first condition is identical to the 'Transparent Condition' previously studied, in which a white ball was used as a treadmill. The results were, therefore, consistent between the two experiments: bees preferred the CS+ in their first choice and spent more time fixating it. The second condition shows that ventral motion cues played a distractive role as only when they were absent, did the fixation time correlate with the bees' choice.

The heat maps displaying the cumulative locations occupied by the bees' trajectories during the entire test are shown in Figs. 9D,E. In the 'No Ventral Optic Flow' condition (Fig. 9D), bees consistently walked towards the CS+ cuboid and inspected it during the test besides choosing it more frequently upon their first choice. In the 'Ventral Optic Flow' condition (Fig. 9E), bees still walked towards the CS+ cuboid but in a less clear way.

Discussion

We studied the impact of motion cues provided by the background on visual-discrimination learning by honey bees in virtual-reality conditions. Bees had to learn the difference between two virtual color cuboids, one of which was rewarded while the other was punished. We focused both on motion cues derived from a background placed frontally 'behind' the color stimuli, and from a ventral ground, which was perceived in the ventral visual field while the bee walked on a Styrofoam ball. In the latter case, the perceived optic flow had no direct relation with the cuboids perceived in the frontal field. The color discrimination task was set under closed loop conditions so that in the case of the grating displayed frontally, both the cuboids and the background could vary (translation and expansion) according to the bees' movements. Our results indicate that in VR conditions, frontal but no ventral motion cues from the background interfered with the learning of colors. Although ventral motion cues did not affect color learning, they were well perceived as they affected walking distance and speed and impaired fixation time of the rewarded stimulus during the test.

Optic flow and visual performances in insects

Optic flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer and a scene 40,41 . It can be seen as a vector field that gives the retinal slip speed of each contrasting object encountered in the environment when the observer moves and/or when features in the environment move relative to the observer 42 . Optic flow processing is crucial for navigation as it allows assessing the distance to objects encountered. Objects closer to an observer move faster in the retinal field than distant objects, so that approaching a target induces higher optic flow while moving away from it decreases it. This information is crucial for moving insects as it allows estimating distances in translational segments 43-46 and avoiding collisions with circumventing obstacles and flying equidistantly from parallel landmarks. For instance, when flying along narrow corridors, insects use the magnitude of visual motion experienced in each eye to control their position, height and speed 47-49 .

Motion cues can be extracted at the edge of objects through parallax and allow evaluating the distance of targets with respect to their background based on differences in their relative retinal speed 50-54 . Edges are therefore contrasting regions in terms of motion-parallax cues and are privileged by flying insects in their detection and landing strategies 51 . Numerous experiments have documented this fact in honey bees 50-54 . An interesting example is provided by experiments in which bees were trained to solve a discrimination between a plain black disk and a black ring positioned a few centimeters in front of a white background. The targets provided a good contrast to the background both in terms of intensity as well as in terms of the motion cues provided at their edges so that bees had no problems in learning this shape discrimination 55 . However, when bees were trained on the same shapes, yet cut from a textured paper and placed in front of a similarly textured background, the task was impossible for them 55 .

This result shows that motion cues alone, which existed because the textured targets were placed in front of the textured background, are not always helpful to appreciate shape differences between targets. Interestingly, this impossible discrimination became possible after the bees were primed by pre-training them with the easy discrimination involving plain stimuli against the white background. This improvement shows that attentional mechanisms boosted by the priming procedure are crucial for achieving target/background segmentation. The role of attentional mechanisms will be discussed below.

Ventral motion cues did not influence the color discrimination performance of bees in VR but affected walking parameters and fixation time

Multiple layers of neurons within visual circuits in the bee brain are devoted to the segregated processing of motion cues, which are essential to estimate distances traveled in translational pathways 56 . Ventral optic flow is particularly important for insects flying in open spaces. In consequence, flying above surfaces providing strong optic flow cues is preferred by bumblebees over flying above featureless backgrounds 57 . Experiments on bumblebees trained to fly along textured tunnels showed that in tunnels of 60 and 120 cm width, control of the lateral position was achieved by balancing the magnitude of translational optic flow experienced in the lateral visual field of each eye; yet, in wider tunnels, bumblebees used translational optic flow perceived in the ventral visual field to control their lateral position and to steer along straight tracks 57 . Ventral optic flow can be used to keep a constant height above the ground using a feedback control loop in which a set point value of perceived ventral optic flow is maintained constant by varying the lift, a solution that was shown experimentally in flying bees 58,59 and that proved to be efficient when implemented in flying robots that needed to keep a constant fly height 60 .

These and other findings 59,61-63 clearly show the importance of ventral motion cues for the translational displacements of flying insects. Although less information is available for walking insects, experiments performed on desert ants Cataglyphis fortis walking in narrow tunnels showed that both the lateral and the ventral optic flow were dispensable for distance estimation 64 . In these insects, the use of a 'pedometer' was proposed, i.e. a stride integrator that accounts for stride number and the respective stride length 65,66 . Although optic flow can be computed by these ants, as shown by the case of ants transported by nestmates, which rely on the optic flow perceived during their transport 67 , the primary mechanism to gauge distances is based on idiothetic cues.

In our experiments, bees walking on a Styrofoam ball were partially affected by the presence or absence of ventral optic-flow cues (Figs. 789). During the training, these cues did not affect the learning performance measured in terms of color choice (Fig. 7). Yet, we found an effect of color, suggesting that discrimination learning was better when blue was the rewarded color. However, this effect disappeared during the test (Fig. 9), as the first choice of the bees revealed that they preferred significantly the CS+, irrespective of its color. Ventral motion cues affected the other variable recorded during the test, the time spent fixating the cuboids (Figs. 9 B,C). When these cues were absent, bees fixated more the CS+, consistently with their first color choice; however, when ventral motion cues were available, they fixated both the CS+ and the CS-to similar extents, even if they preferred the CS+ in their first color choice. Thus, ventral motion cues interfered with the time spent fixating the CS+ during the test.

The absence of effect of ventral motion cues during the training did not mean that bees did not pay attention to them or that they were unable to perceive the difference between the two walking surfaces. Fig. 8A,B shows that both the distance walked and the walking speed decreased significantly when ventral motion cues were available, thus showing that bees perceived them. Their impact on these motor variables indicates, in addition, that such cues are relevant for estimating walking distances. This conclusion goes against the possibility that in a walking context, bees, like desert ants, rely on a mechanism for estimating distances different from that employed during flight. In fact, it is difficult to conceive how the relevance of optic flow could be switched off during walking, given its fundamental role for bee navigation.

Alternatively, our findings may indicate that ventral optic-flow cues play a fundamental role en route to the goal for distance estimation and completion of an intended translational vector, but not in the immediate vicinity of the goal, when the insect faces the task of close-up object recognition. In the latter situation, translational ventral optic flow may be irrelevant as the goal has been reached. Last but not least, it is worth considering that our experiments did not create a ventral optic flow in the virtual arena, i.e. below the targets to be discriminated, but only on the walking treadmill. Including ventral motion cues in the floor of the virtual arena could affect the choice of the color cuboids in a way similar to that induced by the frontal motion cues from the background.

Frontal motion cues from the background interfered with the color discrimination performance of bees in VR

Motion cues perceived frontally at the edges of vertically displayed targets allows segregating them from their respective background based on motion parallax cues. This feature extraction improves therefore object identification and landing on targets. Yet, in our experiments, whenever motion cues from the background were available ('Vertical Grating -Optic Flow Condition' and 'Rotating Vertical Grating Condition'; see Fig. 4), color discrimination of objects located in the virtual foreground was impaired. This result is in contradiction with the hypothesis that animals should behave better in more realistic environments and challenges a priori efforts towards enriching our VR environment with additional cues besides those to be learned and discriminated. Indeed, learning was only possible in the total absence of a frontal background ('Transparent Condition', Fig. 4), suggesting that background cues interfered with the learning of foreground objects. Interestingly, in the 'Vertical Grating -No Optic Flow Condition', optic flow from the background was artificially suppressed and yet learning was not apparent even if a tendency towards a segregation of CS+ and CS-curves was observed (see Fig. 4B). However, when test performances were analyzed in terms of the time spent fixating the CS+ and the CS-, a significant difference in favor of the former was found, which is consistent with a learning effect. The fact that performances were not as clear as in the 'Transparent Condition' suggests that the mere presence of the background may have been distractive for the bees. Thus, both the motion cues emanating from the background, and its illumination conditions, may have interfered with color learning in the VR arena.

A first explanation for this interference could rely on the role of irradiance cues used to establish the background in our VR environment. The background was projected onto the semicircular screen of our setup by a videoprojector and therefore provided irradiance cues that could have attracted the bees based on positive phototaxis, thus interfering with color discrimination. As bees are tested in the dark, a situation inherent to the use of a videoprojector, phototaxis may have indeed influenced the behavior of the bees in our VR setup as shown in experiments performed in open-loop conditions using the same kind of videoprojector-based display 12 . Admittedly, the green and the blue lights used for training the bees had 22 times and 143 times more irradiance than the red used for the background (Red 100: RGB: 255, 0, 0; irradiance: 1130 μW/cm²). It could be interesting to determine if a similar interference with color learning would take place when using the other red light that the bees could see (Red 50: RGB: 128, 0, 0; irradiance: 140 μW/cm²; see Fig. 3). For this light, the difference of irradiance between colors and background decreases in one order of magnitude with respect to the Red 100 used in our experiments. In theory, using the Red 50 light should not change the main findings reported because the phototactic attraction exerted by this stimulus was identical to that induced by the Red 100 light (see Figs. 3 B andC).

Another reason for the negative influence of motion cues emanating from the frontal background could be an excessive salience of these cues with respect to those from the foreground objects that had to be discriminated. In natural conditions, background objects provide motion cues that are less salient than those of foreground objects. Although we attempted to reproduce this situation in our VR environment (the expansion of the cuboids during forward motion was 1.7 °/cm while that of a red bar from the background was 0.18 °/cm), the optic flow generated by the background might still have been too high and detracted bees from efficiently learning the discrimination task.

Finally, in the 'Rotating Vertical Grating Condition', optomotor responses triggered by the background rotating regularly in front of the bees may have interfered with the color learning task. To determine if this was the case, we analyzed the cumulative turning exhibited by the bees in this condition and in the 'Transparent Condition', where no background was available. Figure S2 shows that the cumulative turning in the direction of the rotating background (to the left) was significantly higher in the 'Rotating Vertical Grating Condition', which indicates the presence of optomotor responses. These responses may have interfered with color learning and may be one of the causes of the impaired performance observed.

Chromatic and achromatic vision in the VR setup

Bees were trained to discriminate two different colors against an achromatic background. Blue and green colors differing in intensity were used to this end (Fig. S1A), which may have resulted in bees using differences in intensity rather than chromatic differences to solve the task when motion cues from the background did not interfere (i.e. in the 'Transparent Condition'; see Fig. 4A). Yet, this possibility is ruled out by the performance of the bees itself. In this condition, no asymmetries in color learning were observed depending on which color was rewarded. Had the bees been guided by achromatic intensity, then significant learning asymmetries should have emerged: bees trained to the less intense green should show impaired learning, detracted by the highly intense blue displayed by the alternative non-rewarded stimulus. On the contrary bees trained to the highly intense blue should have their performances amplified by the attraction induced by the blue light. This was not the case and no color effect was observed in this experiment. The situation presented in the 'Transparent Condition' was reproduced in the experiments studying the effect of the ventral optic flow, when the surface of the treadmill was plain white (see Fig. 7, 'No Optic Flow Condition'). In this case, a color effect consistent with the use of intensity was visible during the training, as performance was better when blue was rewarded than when green was rewarded (see Fig. 7C,D). However, this effect disappeared during the test (see Fig. 9), showing that it was inconsistent and that even the bees rewarded on the green color learned the task. These results indicate that in the absence of distractive motion cues from the background, the bees were mainly guided by chromatic cues from the blue and green colors although we cannot definitely rule out an incidence of stimulus intensity in these experiments. Note that the same colors were used in previous studies performed in our VR setup and that no color asymmetry was found, which goes against the use of color intensity by the bees 12-16,27 .

An additional issue that requires consideration is the possible interference of the red light (Red 100) used for the background with the color vision system of the bees involved in the blue-green discrimination. Fig. S1 shows that Red 100 could only be perceived via the L (Green) receptor type, i.e. via an achromatic visual mechanism involving a single receptor type.

Bees can see red (see Fig. 3), not as a color, but as an achromatic stimulus, perceived in terms of its intensity by the L receptor 32,33 . Whether Red 100 affected negatively chromatic discrimination, interfering with the L-receptor type involved in this discrimination is unknown.

VR experiments with freely flying bumblebees trained to land on a virtual horizontal blue target located on a projected achromatic checkerboard made of random pink (RGB: 255,127,127) and white (RGB: 255, 255, 255) squares showed that the background had no incidence on the bees' performance 61 . In this case, the pink light used could potentially stimulate all receptor types and thus truly affect the color vision system, contrarily to our red light (RGB: 255, 0, 0). The fact that this was not the case suggests a minor effect, if any, of the red light in our color discrimination experiments.

Conclusion

Our results point towards deficits in attentional processes underlying color discrimination whenever motion cues from the background were frontally available in our VR setup. In the case of ventral motion cues, no interference of color learning was observed, yet, a distractive effect on the time spent fixating the stimuli was detected during the test. Attention plays a fundamental role in visual discrimination tasks achieved by bees and other insects 68-71 .

Attention is defined as the "ability to focus our perception on one stimulus (or group of related stimuli), while filtering out other simultaneous stimuli that are less relevant at any moment" 72 .

Several studies focusing on color discrimination by bees have underlined the importance of attention in this context. In particular, differential conditioning protocols -as the one used in this workare said to require more attention than absolute conditioning, the simple training of a single stimulus 73 , in particular when the stimuli to be discriminated are similar 34 . The role of attention in visual object recognition was studied by training bees to choose a colored target disc among a variable number of differently colored distractor discs 74 . Accuracy and decision time were measured as a function of distractor number and color. For all color combinations, decision time increased and accuracy decreased with increasing distractor number, whereas performance improved when more targets were present.

From this perspective, highly salient irradiance or motion cues from the background may have interfered with attentional processes required to achieve the color cuboid discrimination. Further experiments may explore strategies to reduce their salience and thus enable their perceptual filtering in our VR landscape.

In the previous chapter, we established a protocol that allowed conditioning tethered bees walking stationary to discriminate 3D colored stimuli. With this tool, we can train bees to solve a visual task while providing to every insect the same visual experience during conditioning.

We decided to use our new setup to investigate the brain regions and visual pathways underlying visual learning. To do so, we quantified the expression of immediate early genes (IEGs) in different brain regions after conditioning. IEGs are considered as useful markers of neural activity so that we compared their levels of expression between learners and nonlearners. Since every bee received the same visual experience in the VR, the main source of variability should come from whether or not an individual learned the task. By focusing on kakusei, Hr38 and Egr1, three IEGs that have been related to bee foraging and orientation, we found that, compared to non-learners, learners exhibited Egr1 upregulation in the calyces of the mushroom bodies. This indicates an involvement of mushroom body calices in associative color learning. and the usefulness of Egr1 as a marker of neural activity induced by this phenomenon.

INTRODUCTION

Invertebrate models of learning and memory have proved to be extremely influential to determine where and when such experience-dependent plasticity occurs in the nervous system 1- 6 . One of these models is the domestic honey bee Apis mellifera, which has been intensively investigated for its visual and olfactory learning capacities 5,7,8 . Yet, the knowledge gained on the mechanisms of these abilities is disparate. While an extensive body of research has accumulated on the neural bases of olfactory learning and memory in bees 9 , practically nothing is known about the neural and molecular underpinnings of their visual learning and memory 10,11 . This asymmetry is due to the fact that olfactory learning protocols use harnessed bees that learn to extend their proboscis to an odorant that has been forward-paired with sucrose water, while visual learning protocols use free-flying bees trained to choose a visual target where they collect sucrose reward 5,10 . Whilst the harnessing situation of olfactory-learning protocols facilitates the use of invasive techniques to record neural activity, the use of bees that commute freely between the hive and the experimental site precludes an equivalent access to visual neural circuits.

Virtual-reality (VR) environments constitute a valuable tool to overcome this limitation.

In such environments, tethered bees walking stationary on a treadmill are exposed to a controlled visual environment that allows studying decision making based on visual cues 12-17 .

Under these conditions, bees learn and memorize simple and higher-order visual discrimination problems, which enables coupling the study of this visual learning with mechanistic analyses of brain activity 16,17 . VR setups may differ according to the degree of variation introduced by the bee movement into the visual environment. In closed-loop conditions, this variation is contingent with the movements of a tethered bee, thus creating a more immersive environment.

In prior works, we introduced a 2D VR environment in which a tethered bee could displace laterally (from left to right and vice versa) a color stimulus on a frontal screen according to its association with sucrose reward of absence of reward 12,14,18 . Here we moved towards a more realistic 3D VR environment which allowed, in addition, for stimulus expansions and retractions depending on forward or backward movements, respectively. In this arena, bees may therefore learn to discriminate colors but can also explore in a less restricted way the virtual world proposed to them.

One way to detect brain regions and pathways activated in this scenario is the quantification of immediate early genes (IEGs) in neural tissues 19 . IEGs are transcribed transiently and rapidly in response to specific stimulations inducing neural activity without de novo protein synthesis 20 . In mammals, IEGs such as c-fos, zif268 and Arc are regularly used as markers of neural activity during learning, memory and other forms of cellular plasticity such as long-term potentiation 21-23 . In insects, the use of IEGs as neural markers is less expanded as the number of candidate genes serving this goal is still reduced and the reliable detection of their expression is sometimes difficult 24 . Three of the IEGs reported for the honey bee are interesting as they have been related to a foraging context in which learning plays a fundamental role. The first one, termed kakusei (which means 'awakening' in Japanese) is a nuclear noncoding RNA transiently and strongly induced in the brain of European workers by seizures that can be induced by awakening them from anesthesia 25 . It is also activated after the experience of dancing in the hive following a foraging flight and in pollen foragers so that it seems related to the neural excitation resulting from foraging activities 26 . This IEG is activated within a subtype of Kenyon cells, the constitutive neurons of the mushroom bodies, which are a higherorder center in the insect brain 27 . A second IEG is the hormone receptor 38 gene (Hr38), which is a transcription factor conserved among insects and other species including humans 28 , and which has been indirectly related to learning and memory in honey bees and other insects 29,30 .

Hr38 is also upregulated by foraging experiences in honey bees 29 and bumblebees 30 and by orientation activities upon hive displacement 31 . The third gene is the early growth response gene-1 (Egr1), whose expression is induced in the brain of honey bees and bumble bees upon foraging 29,30 and orientation flights 32 , and which seems to be controlled by circadian timing of foraging 33 . None of these IEGs has been studied so far in the context of associative learning and memory formation in the honey bee.

We thus focused on these IEGs to characterize neural activation induced by visual learning in the brain of bees under 3D VR conditions. Bees had to learn to discriminate a rewarded color from a punished color 34-37 and should retain this information in a short-term retention test. Our goal was to determine if successful learning and retention activate specifically certain regions in the brain, in particular the mushroom bodies, whose importance for olfactory learning and memory has been repeatedly stressed 5,38 , yet with a dramatic lack of equivalent evidence in the visual domain. Our results show that successful learners exhibited Egr1 upregulation only in the calyces of the mushroom bodies, thus uncovering a privileged involvement of these brain regions in associative color learning.

RESULTS

Color learning under 3D VR conditions

Honey bee foragers were captured at an artificial feeder to which they were previously trained and brought to the laboratory where a tether was glued on their thorax. (Fig. 1A,B). They could be then attached to a holder that allowed adjusting their position on a treadmill, a polystyrene ball floating on a constant airflow produced by an air pump (see Methods for details). The VR setup consisted of this treadmill placed in front of a semi-cylindrical semi-transparent screen made of tracing paper (Fig. 1A). The movements of the walking bee on the treadmill were recorded by two infrared optic-mouse sensors placed on the ball support perpendicular to each other. Cuboids were green and blue on a dark background. Color intensities were adjusted to avoid phototactic biases independent of learning.

Bees were trained to discriminate a green from a blue vertical cuboid against a black background during ten conditioning trials (Fig. 1C; see Supplementary Fig. 1 for color characteristics). Training consisted in pairing one of the cuboids (CS+) with a rewarding 1 M sucrose solution and the other (CS-) with an aversive 3M NaCl solution 39,40 (Fig. 2). Bees performed equally irrespective of the color trained (z= -0.97, p=0.33). They were subdivided according to their test performance to distinguish those which showed successful discrimination (i.e. choice of the CS+; "learners") from those which did not ("non-learners"). This distinction allowed subsequent brain gene analyses according to learning success. Bees that were unable to choose a stimulus in at least 5 trials were excluded from the analysis. Acquisition was significant for learners (n=17) during conditioning trials (Fig. 3A; CS*Trial effect: χ 2 =33.68, df:2, p<0.0001), confirming the occurrence of learning. Indeed, the percentages of bees responding to the CS+ and to the CS-differed significantly along trials (CS+ vs. CS-: CS*Trial; z=-5.46, p<0.0001). Significant differences were also found when comparing the percentages of non-responding bees against the CS+ responding bees and against the CS-responding bees (NC vs. CS+: CS*Trial; z=8.14, p<0.0001; NC vs. CS-: CS*Trial; z=4.59, p<0.0001). Nonlearners (n=18) did also show a significant interaction (Fig. 3B; CS*Trial effect: χ 2 =7.66, df:2, p=0.02), but this was introduced by the percentage of non-responding bees. These bees differed significantly along trials both from the bees responding to the CS+ (NC vs. CS+: CS*Trial; z=6.10, p<0.0001) and from the bees responding to the CS-(NC vs. CS-: CS*Trial; z=6.07, p<0.0001). On the contrary, the percentages of bees responding to the CS+ and to the CS-did not vary along trials (CS+ vs. CS-: CS*Trial; z=-0.07, p=1), consistently with the absence of learning.

Figure 2. Choice criterion and conditioning protocol for color discrimination learning. A)

Choice criterion. Left: A bee facing the two virtual cuboids. Center: A bee approaching a target cuboid; the cuboid has not yet been centered by the bee (gray area). Right: A bee having centered the target cuboid (gray area). A choice was recorded when the bee reached an area of a radius of 3 cm centered on the cuboid and fixed it frontally. The cuboid image was then frozen during 8 s and the corresponding reinforcement (US) was delivered. B) Conditioning protocol.

Bees were trained along 10 conditioning trials that lasted a maximum of 1 min and that were spaced by 1 min (intertrial interval). After the end of conditioning, and following an additional interval of 1 min, bees were tested in extinction conditions with the two colored cuboids during 1 min.

We next asked if differences between learners and non-learners could be due to differences in motor components. To answer this question, we analyzed for each conditioning trial the total distance walked, the walking speed, and the tortuosity of the trajectories.

Tortuosity was calculated as the ratio between the total distance walked and the distance between the first and the last point of the trajectory connected by an imaginary straight line.

When the ratio was 1, or close to 1, trajectories were straightforward while higher values corresponded to sinuous trajectories. The distance travelled (Fig. 4A) did neither vary along trials (Trial: χ 2 =0. 24, df:1, p=0.62) nor between learners and non-learners (Condition: χ 2 =1.10, df:1, p=0.30; Condition*Trial: χ 2 =0.71, df:1, p=0.40). Tortuosity (Fig. 4B) varied along trials (Trial: χ 2 =14.53, df:1, p<0.001) but not between learners and non-learners (Condition: χ 2 =0.08, df:1, p=0.80; Condition*Trial: χ 2 =0.42, df:1, p=0.52). Finally, the walking speed (Fig. 4C) increased significantly along trials (Trial: χ 2 =30.49, df:1, p<0.0001) but did not vary between learners and non-learners (Condition: χ 2 =1. 43, df:1, p=0.23); in this case, however, the interaction between Trial and Condition was significant (χ 2 =4.68, df:1, p<0.05). This suggests that learners were slower than non-learners, which is reminiscent of a speed-accuracy trade off reported in numerous experiments in bees 41-43 . Finally, in the non-reinforced test, per definition learners (n=17; Fig. 3C) chose correctly the CS+ (100% of the bees) while non-learners (n=18; Fig. 3D) did either chose the CS-(72.22%) or did not perform any choice (27.78%). We thus focused on differences between learners and non-learners in the subsequent IEG analyses to uncover possible changes in neural activity induced by learning. Table 1. Primer sequences used to quantify RNA expression of genes of interest and reference genes by RT-qPCR. Amplicon length (bp), efficiency (E, %) and the coefficient of correlation obtained for the standard curve (R 2 ) are also shown. Hr38: Hormone receptor 38 gene; Egr1:

Early growth response gene-1; Ef1α: Elongation factor 1 α gene.

IEG analyses in the honey bee brain following color learning under 3D VR conditions

We aimed at determining if visual learning in VR induces post learning transcriptional changes, which might participate in amplifying neural activity reflecting associative color learning. To this end, we performed RT-qPCR in individual brains of learners and non-learners, which were collected 1h after the retention test and placed in liquid nitrogen until brain dissection. We analyzed relative expression levels of kakusei, Hr38 and Egr1 (see Table 1) in three main brain regions 44 (Fig. 5A): the optical lobes (OL), the upper part of the mushroom bodies (i.e. the mushroom-body calyces or MB Ca) and the remaining central brain (CB), which included mainly the central complex, the subesophageal zone and the peduncula and lobes (and  lobes) of the mushroom bodies. Two reference genes were used for the normalization, Ef1 (E=106%) and Actin (E=110%), which proved to be the best choice for the normalization (see groups (learners and non-learners) for the MB and the CB. In the case of the OL, Ef1 varied significantly between groups. Thus, normalization used the product of the two reference genes for MB and CB while only actin could be used to normalize OL data. No cross-comparisons between brain regions or genes were performed. between learners and non-learners were neither found in the OL (Fig. 5H, t29=1.48, p=0.15) nor in the CB (Fig. 5J, t29=0.17, p=0.86), thus showing that learning-dependent variation in IEG expression was circumscribed to the calyces of the mushroom bodies and that Egr1 was more sensitive than both Hr38 and kakusei to detect changes in neural activity induced by associative learning.

DISCUSSION

Our work shows that visual discrimination learning under virtual-reality conditions leads to an enhancement of IEG expression in the case of Egr1 in the calyces of the mushroom bodies in successful honey bee learners. Learning success did not correlate with differences in distance travelled or tortuosity of trajectories, i.e. with differences in exploratory drive (Fig. 4), but was correlated with differences in walking speed as learners tended to be slower than non-learners.

Although strictly speaking the two categories did not differ with respect to this parameter, the significant interaction between Trial and Condition suggests a speed-accuracy trade off in which individuals taking more time to decide can improve the accuracy of their decisions 41-43 .

Differences in Egr1 expression were thus related to learning success and not to differences in exploratory components. For the other two IEGs analyzed, kakusei and Hr38, no learningdependent changes could be detected in the different brain regions considered, even if prior reports indicated similar levels of expression for the three IEGs in the brain of bees engaged in foraging 29,30,33,45 and orienting around the hive 29-31 . Our work demonstrates therefore that this similarity does not necessarily reflect a relationship with associative learning and memory as only Egr1 acted as bona fide marker of learning success in the bee brain under our experimental conditions and revealed the implication of the calyces of the mushroom bodies in associative visual learning and memory in honey bees.

Differential expression of IEGs in the honey bee brain as related to visual learning

Kakusei did not vary in the brain regions considered, under the experimental conditions defined in our work. This IEG does not have orthologous genes in other taxa and its role in honey bees is unclear. It is induced by seizures following anesthesia 25,27,45,46 and thermal stimulation 46 , but also by foraging and reorientation activity following hive displacement 25,31,45 . These experiences increase kakusei expression in the mushroom bodies 25 but also in the optic lobes 25,27,45 and the dorsal lobe 27 . Our results suggest that its enhanced expression in foragers or in orienting bees is not necessarily related to learning occurring in these contexts.

Differential expression of kakusei with respect to an inducing treatment (typically, an induced seizure) starts around 15 min post treatment 25,31,46 but continues during longer periods which may go beyond 60 min 46 . Thus, the waiting time of 60 min between test and brain freezing in our experiments was appropriate to detect changes in kakusei as a result of associative visual learning. However, as other temporal analyses of kakusei expression reported a decay in expression beyond 30 min 25 , the possibility that our sampling period was too long to capture changes in kakusei expression cannot be excluded.

This concern does not apply to Hr38 and Egr1, for which temporal expression analyses showed a systematical increase at the time chosen for our experiments 30 . As in the case of kakusei, no learning-related changes were detected in Hr38 expression across the brain regions considered. This hormone receptor gene has been indirectly related to learning and memory in honey bees and other insects 29,30 and is also upregulated by foraging experiences in honey bees 29 and bumblebees 30 and by orientation activities upon hive displacement 31 . Despite its involvement in these activities, it did not reveal learning-dependent changes in neural activity in the experimental context defined by our setup and training protocol.

Only Egr1 reported a significant variation in the mushroom body calyces of learners in relation to non-learners (Fig. 5). As for the two other IEGs, the expression of this early growth response gene is enhanced in the brain of honey bees and bumblebees upon foraging 29,30 and orientation flights 32 . Yet, in this case, Egr1 was sensitive enough to report differences in neural activity related to learning success in our experimental conditions. Learners and non-learners were identical in their experience and handling all along the experiment and they only differed in learning success. Thus, differences in Egr1 expression demonstrate that associative color learning is accompanied by increased neural activity in the calyces of the mushroom bodies.

The role of mushroom bodies for visual learning and memory

Although the crucial role of mushroom bodies for the acquisition, storage and retrieval of olfactory memories has been extensively documented in bees 7,38,47 and other insect species 2,3,48 , less is known about their implication in visual learning and memory. In the honey bee, the fact that visual learning was mainly studied using free-flying bees trained to choose visual targets precluded its study at the cellular level 13 . The neural circuits for color processing are known in the bee brain 49-52 but evidence about plasticity-dependent changes in these circuits remains scarce. Such changes could occur at multiple stages, as is the case in olfactory circuits mediating olfactory learning 9 . Upstream the mushroom bodies, inner-layer lobula and inner medulla neurons project to both the mushroom bodies and the lateral protocerebrum 49,50,53 and exhibit color sensitivity, color opponency and temporally complex patterns including adaptation and entrainment 49,53,54 . These patterns are important for color coding and discrimination and could be subjected to experience-dependent changes in activity 55 .

The implication of mushroom bodies in visual learning and memory in the bee is expected given the parallels between visual and olfactory inputs at the level of the calyces.

Whilst afferent projection neurons convey olfactory information to a subdivision of the calyces, the lip 56 , afferent neurons from the lobula and the medulla, which are part of the optic lobes, convey visual information to other calyx subdivisions, the collar and the basal ring 50,57 . In spite of this similarity, studies addressing the role of mushroom bodies in honey bee visual learning and memory remain rare. The recent development of protocols for the study of aversive visual learning (association between a color light and an electric shock delivered to walking bees enclosed in a box compartment) 44,58 has showed the possible implication of mushroom bodies in this form of learning. In a pharmacological study, in which one half of a chamber was illuminated with one color and paired with shock while the other half was illuminated with a different color not paired with shock, bees learned to escape the shock-paired light and spent more time in the safe light after a few trials 59 . When ventral lobe neurons of the mushroom bodies were silenced by procaine injection, bees were no longer able to associate one light with shock. By contrast, silencing one collar region of the mushroom body calyx did not alter behavior in comparison with that of controls 59 . The latter result does not exclude a role for the calyces in visual learning, as blocking one of four collar regions may not have a significant impact on learning. In a different study, bees were trained to inhibit their spontaneous phototaxis by pairing the attracting light with an electric shock 44 . In this case, learning induced an increase in the dopaminergic receptor gene Amdop1 in the calyces of the mushroom bodies, consistently with the role of dopaminergic signaling for electric-shock representation in the bee brain 60,61 .

In the fruit fly, the study of the role of mushroom bodies for visual learning and memory has yielded contradictory results. Flies suspended within a flight simulator learn to fly towards unpunished visual landmarks to avoid heat punishment delivered to their thorax; mushroom body deficits do not affect learning so that these structures were considered dispensable for visual learning and memory 62 . Similarly, learning to discriminate colors in a cylindrical container made of a blue-lit and a yellow-lit compartment, one of which was associated with aversive shaking, was not affected in mushroom body mutants 63 . Visual place learning by flies walking within a cylindrical arena displaying landmarks can also take place in the absence of functional mushroom bodies but requires the central complex 64 . Yet, the dispensability of mushroom bodies for visual learning and memory in fruit flies has been questioned by experiments in which appetitive and aversive color learning and discrimination were studied in an arena in which blue and green colors were presented from below. Walking flies learned both the appetitive (based on pairing one color with sugar) and the aversive discrimination (based on pairing one color with electric shock) but failed if mushroom body function was blocked using neurogenetic tools 65 . Thus, the role of mushroom bodies for visual learning and memory in fruit flies may be both task-and learning specific. In addition, the dominance of olfactory inputs to the mushroom bodies may overshadow their role for visual learning in Drosophila.

IEG expression within the mushroom bodies in relation to visual learning

Kenyon cells are the constitutive neurons of the mushroom bodies. Their somata are located both within the mushroom-body calyces and adjacent to them. Thus, our brain sectioning (see Fig. 4A) collected them massively. Detecting IEG activation in the mushroom bodies upon visual learning may be particularly difficult as learning-dependent changes in neural activity may be subtle due to the characteristic sparse neural activity observed at the level of the calyces.

This reduced activity, which has been revealed in studies on olfactory coding 66-68 and odorrelated learning 69 , can also be a hallmark of visual processing and visual learning. Sparse neural coding of odorants is in part due to GABAergic inhibition by feedback extrinsic mushroombody neurons acting on Kenyon cells 70,71 , the constitutive neurons of the mushroom bodies.

These GABAergic neurons, present both in bees and flies 70,72,73 , suppress Kenyon cell activity to maintain sparse, neural coding, and may render difficult detecting variations of IEG expression in the calyces. Yet, we were able to find differences that were dependent on the experience of the animals analyzed. Such differences might vary according to the difficulty of the learning problem considered. For instance, higher GABAergic input is required in the calyces to solve non-linear discriminations, in which subjects have to inhibit response summation to the simultaneous presentation of stimuli A and B, which are rewarded when presented alone but non-rewarded when presented together. Bees that learn to solve this discrimination in the olfactory domain require inhibitory GABAergic feedback in the calyces to this end 47 . Such a requirement could translate into a different form of IEG expression in this brain region as a consequence of a more complex discrimination learning.

Recent work on gene expression in the Kenyon-cells of honey bees revealed the existence of various cell subtypes/populations with unique gene expression profiles and cell body morphology 74 . Among these populations, small Kenyon cells (sKC) 75 , formerly called inner Kenyon cells 76 , are found in the central, inner core of the MB calyces and express preferentially three genes, EcR, E74 and Hr38, the latter being higher in the brain of foragers than in nurses 74 . Unfortunately, no information on Egr1 was reported in this analysis. Yet, another study that did not distinguished between Kenyon-cell subtypes reported that the expression of Egr1is enriched in Kenyon cells compared to the rest of the brain 32 and that this enrichment might be related to learning and memory given its association with the orientation flights of bees 32 and with foraging activities 29,30,77 . However, the sensory cues and behavioral programs participating in both foraging and orientation are multiple so that it is difficult to sustain such a claim in the absence of a controlled learning experiment. For instance, Egr1 is also upregulated in the brain of honey bees upon seizure induction 78 , with no relation to foraging or orientation. Only specific experiments like the one performed in this work can reveal whether increases in this and other IEGs reflect neural activity induced by associative learning.

Consistently with the notion that sKCs may be particularly relevant for learning and memory formation, phosphorylated (activated) cAMP-response element binding protein (pCREB) is enriched in these sKCs in the honey bee 79 . CREB is a nuclear protein that modulates the transcription of genes required for the cellular events underlying long-term memory (LTM) formation in both invertebrates and vertebrates 80-83 and its activation leads also to the expression of IEGs. It is thus possible that the increased expression of Egr1 induced by visual learning and memory formation is localized within sKCs, and that this increase results from CREB activation. In our experiments, the reinforced tests were done shortly after the last conditioning trial and only one hour elapsed since the end of the test and the collection of brains for IEG analysis (a time necessary for the expression of the IEGs selected). This period does not correspond with the temporal requirements for olfactory LTM formation in the standard view of memory dynamics in the honey bee, where a protein-synthesis dependent LTM is expected after 24-h post conditioning 84 . However, recent work on olfactory memory formation has shown that protein-synthesis dependent memories arise much earlier and with less conditioning trials than previously thought 85 . Whether our visual conditioning leads to protein-synthesis dependent LTM, mediated by CREB activation, remains to be determined.

Taken together, our results show both the implication of mushroom bodies in appetitive visual learning in honey bees and the usefulness of Egr1 as a marker of neural activity induced by these phenomena under our experimental conditions. The learning success in our VR setup was 50%, which contrasts with the higher learning rates observable for similar color discriminations in the case of free-flying bees. This decrease may be due to several reasons such as the impossibility to return to the hive between rewarded experiences, the tethering conditions and the resulting reduction in active vision. As the tethering impedes, in part, free movements, it may affect the possibility of actively scanning the images perceived, impairing thereby the possibility of extracting target information and learning. In spite of these restrictions, our setup allowed to segregate between learners and non-learners and achieve

relevant analyses to answer questions on the neural and molecular underpinnings of associative visual learning. It constitutes therefore a valuable tool for further studies on the mechanisms of visual cognition in bees.

The protocol used to train the bees in our work consisted in an elemental discrimination between a rewarded and non-rewarded color. Yet, bees are well known for remarkable visual performances, which include the non-elemental learning of concepts and relational rules 86-88 . It is therefore possible that different forms of learning, which recruit different brain regions 47 , may reveal experience-dependent neural activation through different IEGs and with different temporal dynamics. Moreover, IEG upregulation may not always be the hallmark of successful learning as in some cases inhibition of neural activity may be crucial for plastic changes in behavior. Thus, addressing if IEG expression varies qualitatively and quantitatively according to learning type and complexity is of fundamental importance. Furthermore, including different intervals post conditioning is important to characterize possible activity changes related to the formation of different memory phases in different regions of the bee brain. Last, but not least, our results highlight the value of virtual-reality conditions for further explorations of the neural and molecular underpinnings of visual learning and memory in bees.

METHODS

Honey bee foragers (Apis mellifera) were obtained from colonies located in our apiary at the University Paul Sabatier. Only foragers caught upon landing on a gravity feeder filled with a 0.9 M sucrose solution were used in our experiments to ensure high appetitive motivation.

Captured bees were brought to the laboratory where they were placed on ice for five minutes to anesthetize them and facilitate the fixation of a tether glued to their thorax by means of melted wax (Fig. 1A). After being attached to the tether, each bee was placed on a small (49 mm diameter) Styrofoam ball for familiarization with the treadmill situation. Bees were provided with 5 μl of 1.5 M sucrose solution and kept for 3 h in this provisory setup in the dark. They were then moved to the VR arena and used for the experiments.

Once in the VR setup, the bee was attached to a holder that allowed adjusting its position on the treadmill (Fig. 1B), a polystyrene ball (diameter: 5 cm, weight: 1.07 g) held by a 3Dprinted support and floating on a constant airflow produced by an air pump (airflow: 555ml/s; Aqua Oxy CWS 2000, Oase, Wasquehal, France).

VR setup

The VR setup consisted of the treadmill and of a half-cylindrical vertical screen made of semitransparent tracing paper, which allowed presentation of a 180° visual environment to the bee (diameter: 268 mm, height: 200 mm, distance to the bee: 9 cm Fig. 1AB) and which was placed in front of the treadmill. The visual environment was projected from behind the screen using a video projector connected to a laptop (Fig. 1A). The video projector was an Acer K135 (Lamp:

LED, Definition: 1280 x 800, Brightness: 600 lumens, Contrast ratio: 10 000:1, Minimum Vertical Sync: 50 Hz, Maximum Vertical Sync: 120 Hz, Minimum Horizontal Sync: 30.10 3 Hz, Maximum Horizontal Sync: 100.10 3 Hz) 14 . The movements of the walking bee on the treadmill were recorded by two infrared optic-mouse sensors (Logitech M500, 1000 dpi, Logitech, Lausanne, Switzerland) placed on the ball support perpendicular to each other.

Experiments were conducted under 3D closed-loop conditions, i.e. rotations of the ball displaced the visual stimuli not only laterally but also towards the bee. To generate these conditions, we developed a custom software by means of the Unity engine (version 2018.3.11f1). The open-source code is available at https://github.com/G-Lafon/BeeVR. The software updated the position of the bee within the VR every 0.017 s. A displacement of 1 cm on the ball corresponds to an equivalent displacement in the VR landscape. Moving 1 cm on the ball towards an object increased the visual angle of the object by ca. 1.7°. Based on the ball movements, our software calculated the position of the walking bee and its heading, and determined which object was centered on the screen.

Visual stimuli

Bees had to discriminate two vertical cuboids (Fig. 1C) based on their different colors and association with reward and punishment. The colors of the cuboids (see Supplementary Fig. 1)

were blue (RGB: 0, 0, 255, with a dominant wavelength of 450 nm and an irradiance of 161000 μW) and green (RGB: 0, 100, 0, with a dominant wavelength of 530 nm and an irradiance of 24370 μW/cm2). They were displayed on a black background (RGB: 0, 0, 0). These colors were chosen based on previous work showing their successful learning in the VR setup 14 .

Each cuboid had a 5×5 cm base and 1 m height so that it occupied the entire vertical extent of the screen irrespective of the bee's position. The cuboids were positioned at -50° and +50° from the bee's body axis at the beginning of each trial. Approaching a cuboid within an area of 3 cm surrounding its virtual surface followed by direct fixation of its center was recorded as a choice (Fig. 2A).

Conditioning and testing at the treadmill

Bees were trained using a differential conditioning, which promotes better learning performances owing to the presence of penalized incorrect color choice that result in an enhancement of visual attention 36 .

Bees were trained during 10 consecutive trials using a differential conditioning procedure (Fig. 2B) in which one of the cuboids (i.e. one of the two colors, green or blue) was rewarded with 1.5 M sucrose solution (the appetitive conditioned stimulus or CS+) while the other cuboid displaying the alternative color (the aversive conditioned stimulus or CS-) was associated with 3 M NaCl solution. The latter was used to increase the penalty of incorrect choices 40,89,90 . To avoid directional biases, the rewarded and the punished color cuboids were swapped between the left and the right side of the virtual arena in a pseudo random manner along trials. Moreover, a reconstruction of the trajectories of the bees analyzed did not show side biases.

A dark screen was shown initially to the bees. During training trials, each bee faced the two cuboids. The bee had to choose the CS+ cuboid by walking towards it and centering it on the screen. Colors were equally and randomly assigned to the CS+ and the CS-category during training. If the bee reached the CS+ within an area of 3 cm in the virtual environment (i.e. if the cuboid chosen by the bee subtended 53° in its horizontal extent) and centered it, the screen was locked during 8 s to ensure fixation. This allowed the delivery of sucrose solution in case of a correct choice, or of NaCl in case of an incorrect choice. Solutions were delivered for 3 s by the experimenter who sat behind the bee and used a toothpick to this end. The toothpick touched first the antennae and then the mouthparts during the 8 s in which the screen was locked on the cuboid fixated by the bee. Each training trial lasted until the bee chose one of both stimuli or for a maximum of 60 s (no choice). Trials were separated by an inter-trial interval of 60 s during which the dark screen was presented. Bees that were unable to choose a stimulus (i.e. that did not fulfill the criterion of a choice defined above) in at least 5 trials were excluded from the analysis. From 216 bees trained, 75 were kept for analysis (~35%).

After the last training trial, each bee was subjected to a non-reinforced test that lasted 60 s (Fig. 2B). Test performance allowed distinguishing learners (i.e. bees that chose the CS+ as their first choice in the test) from non-learners (i.e. bees that either chose the CS-in their first test choice or that did not make any choice during the test). IEG expression was compared between these two groups, which had the same sensory experience in the VR setup and which differed only in their learning success.

Brain dissection

One hour after the test, bees were decapitated, and the head was instantly frozen in a nitrogen solution. The period between post-test and brain collection was chosen to allow induction of the three IEGS studied (typically, 15 or more min in the case of kakusei 25,46 and 30-60 min in the case of Hr38 31 and Egr1 30 ). The frozen bee head was dissected on dry ice under a microscope. First, the antennae were removed and a window was cut in the upper part of the head capsule, removing the cuticle between the compound eyes and the ocelli. Second, the glands and tracheae around the brain were removed. Third, the retinas of the compound eyes were also removed.

The frozen brain was cut in three main parts for IEG analyses (Fig. 4A): the optic lobes (OL), the upper part of the mushroom bodies (the mushroom-body calyces, MB Ca) and the remaining central brain (CB), which included mainly the central complex (CC), the subesophageal zone (SEZ) and the peduncula of the mushroom-bodies (and  lobes). Samples were stored at -80 °C before RNA extraction. During the dissection process, one of these three regions was lost in 4 non-learners brains As only bees for which all regions were available were kept in the analyses, the sample sizes of the non-learners differ between the behavioral (n=18)

and the molecular analyses (n=14).

RNA extraction and reverse transcription

The RNAs from the three sections mentioned above (OL, MB Ca and CB) were extracted and purified using the RNeasy Micro Kit (Qiagen). The final RNA concentration obtained was measured by spectrophotometry (NanoDrop™ One, Thermo Scientific). A volume of 10 µl containing 100 ng of the RNA obtained was used for reverse transcription following the procedure recommended in the Maxima H Minus First Strand cDNA Synthesis kit (Thermoscientific, 0.25 µl of random hexamer primer, 1 µl of 10 mM dNTP mix, 3.75 µl of nuclease free H2O, 4 µl 5X RT Buffer and 1 µl Maxima H Minus Enzyme Mix).

Quantitative Polymerase Chain Reaction (RT-qPCR)

All the primers used for target and reference genes generated amplification products of approximately 150 pb. The efficiencies of all reactions with the different primers used were between 95 and 110 % (Table 1). Their specificity was verified by analyzing melting curves of the qRT-PCR products (see Supplementary Fig. 2). Two reference genes (Ef1 and Actin) were used for normalization.

Expression was quantified using a SYBR Green real-time PCR method. Each sample was run in triplicates. If the triplicates showed too much variability (SD > 0.3), the furthest triplicate was discarded. If the two remaining triplicates still showed too much variability (SD > 0.3) the sample was discarded. The samples were subjected to a relative quantification and normalization. First for each sample and for each reference gene per brain region, the relative quantity (Qr) was computed using the difference between the mean Ct value of each sample and the highest mean Ct value (ΔCt), using the following formula: Qr= (1+E) ΔCt (with E= efficiency of the reaction). Then a normalization factor for each sample was obtained computing the geometric mean of the relative quantities obtained for the reference genes in the corresponding samples (ΔΔCt).

Data analysis and statistics

Behavioral data

The first choice of the bees was recorded during the conditioning trials and the non-reinforced test. In this way, we established for each trial and test the percentages of bees choosing first each of the stimuli displayed or not choosing a stimulus (± 95% confidence interval).

Test percentages were analyzed within groups by means of a generalized linear mixed model (GLMM) for binomial family in which the individual identity (Bee) was considered as a random factor (individual effect) while the choice category (CS+, CS-, NC) was fitted as a fixed effect; z values with corresponding degrees of freedom are reported throughout for this kind of analysis.

For each acquisition trial, we recorded motor variables such as the total distance walked, the walking speed, and the tortuosity of the trajectories 91 . Tortuosity was calculated as the ratio between the total distance walked and the distance between the first and the last point of the trajectory connected by an imaginary straight line. When the ratio was 1, or close to 1, trajectories were straightforward while higher values corresponded to sinuous trajectories 91 .

The analysis of these continuous variables was done using a linear mixed model (lmer function)

in which the individual identity (Bee ID) was a random factor and the experimental condition (Condition) and trial number (Trial) were fixed factors 91 . Statistical analyses were performed using with R 3.5.1 92 . The package lme4 was used for GLMMs and LMMs.

In the second chapter, we found that visual learning under 3D VR conditions led to an upregulation of Egr1 in the calyces of the mushroom bodies. We thus asked a different question, namely whether the constraints of the VR environmental impact on neural activation in a color discrimination task. Specifically, we asked if learning the same color discrimination as in the previous chapter, but this time in a 2D VR, which sets higher movement constraints, results in similar IEG expression patters as those detected in the 3D environment. Surprisingly, the comparison between non-learners and learners revealed a completely different pattern of activation from the one found in the second chapter. Here, Egr1 was downregulated in the optic lobes, while Hr38 and kakusei were coincidently downregulated in the mushroom body calyces.

In this chapter, we present these results and offer some possible explanations as to why 2D conditioning leads to a different pattern of IEG expression. In particular, we interpret this downregulation as a reflect of a higher neural inhibition in the 2D VR discrimination.

Introduction

Learning relies on changes in neural activity and/or connectivity in the nervous system, which underlie the acquisition of new, durable information based on individual experience.

Invertebrate models have proved to be extremely influential to characterize learning and memory at multiple levels, not only because they allow determining where and when such changes occur in the nervous system 1-7 but also because their behavioral performances can be studied in standardized laboratory protocols that allow full control over the sensory variables that animals should learn and memorize. A paradigmatic example is provided by the honey bee

Apis mellifera, where the study of olfactory learning and memory experienced significant progresses thanks to the availability of a Pavlovian conditioning protocol that offers the possibility of acquiring consistent behavioral data coupled with the simultaneous use of invasive methods to record neural activity 5, 8-10 . In this protocol, termed the olfactory conditioning of the proboscis extension reflex (PER), harnessed bees learn to associate an odorant with a reward of sucrose solution 10, 11 . The immobility imposed to the trained bees and the Pavlovian nature of the association learned (the odorant acts as the conditioned stimulus and the sucrose reward as the unconditioned stimulus) allows a full control over the stimulations provided and thus a fine characterization of behavioral changes due to learning and memory.

In the case of visual learning by honey bees, this possibility is reduced as performances are mostly studied in free-flying foragers 5, 12 under semi-natural conditions. Yet, virtual-reality (VR) environments have been recently developed to overcome this limitation 13 as they provide not only a full control over the visual surrounding of an experimental subject, be it tethered or not, but also the delivery of physically impossible ambiguous stimuli, which give conflicting visual information 14 . In one type of VR that we developed in the last years, a tethered bee walks stationary on a treadmill while being exposed to a controlled visual environment displayed by a video projector. Bees can then be trained with virtual targets that are paired with gustatory reward or punishment 13, 15-19 . To create an immersive environment, closed-loop visual conditions are used in which the variations of the visual panorama are determined by the walking movements of the bee on the treadmill. Under these conditions, bees learn and memorize simple 15, 19 and higher-order 20 visual discriminations, which offers the potential for mechanistic analyses of visually-oriented performances 17, 18 .

We have used two different types of closed loop situation so far: a restrictive 2D situation, in which bees can displace conditioned targets only frontally (i.e. from left to right and vice versa) 15, 19, 20 , and a more realistic 3D situation which includes a depth dimension so that targets expand upon approach and retract upon distancing 21 . Although bees learn to discriminate color stimuli in both conditions, the processes underlying such learning may differ given the different conditions imposed to the bees in terms of stimulus control. Indeed, while in 3D scenarios movement translates into a displacement and a recognizable change in the visual scene, which can then be computed against the available internal information about the displacement, 2D scenarios are restricted to the execution of actions that are dependent on reinforcement contingency. These two conditions may give rise to different mechanisms of information acquisition.

In a recent work, we studied color learning in the 3D scenario and quantified immediate early genes (IEGs) in the brain of learners and non-learners to uncover the regions that are involved in this discrimination learning 22 . IEGs are efficient markers of neural activity as they are transcribed transiently and rapidly in response to specific stimulations inducing neural activity without de novo protein synthesis 23-25 . Three IEGs were quantified on the basis of numerous reports that associated them with foraging and orientation activities 26-30 : kakusei, a nuclear non-coding RNA 31 , the hormone receptor 38 gene (Hr38), a component of the ecdysteroid signalling pathway 32 , and the early growth response gene-1 (Egr1), which is a major mediator and regulator of synaptic plasticity and neuronal activity 33 . We found that color learning in the 3D VR environment was associated with an upregulation of Egr1 in the calyces of the mushroom bodies 22 , a main structure of the insect brain repeatedly associated with the storage and retrieval of olfactory memories 2, 9 . No other changes of IEG expression were detected in other regions of the brain, thus underlining the relevance of mushroom bodies for color learning and retention 22 .

Here we asked if color learning in the more restrictive 2D VR environment induces changes in IEG expression, both at the gene level and at the brain region level, similar to those detected in the 3D VR system. Asking this question is important to determine if changes in IEG expression differ according to the degrees of freedom of the VR system and the distinct motor patterns that are engaged in either case. Despite the similarity in behavioral performance (bees learn to discriminate colors in both scenarios), we reasoned that the processes underlying learning may be different given the restrictive conditions of the 2D VR, which demand a tight stimulus control while the 3D VR enables exploration of the virtual environment. We thus studied color learning in the 2D VR environment and performed ex vivo analyses to map IEG expression in brain areas of learners and non-learners, which had the same sensory experience and only differed in terms of learning success.

Results

Behavioral analyses

Honey bee foragers from a hive located in our apiary were captured at an artificial feeder to which they were previously trained. They were enclosed in individual glass vials and brought to the laboratory where they were prepared for the VR experiments. A tether was glued on their thorax (Fig. 1A,B), which allowed to attach them to a holder to adjust their position on a treadmill. The treadmill was a polystyrene ball that was suspended on an air cushion produced by an air pumping system (see Methods for details). The bee suspended from its tether could walk stationary on the treadmill; its movements were recorded by two infrared optic-mouse sensors placed on the ball support perpendicular to each other, which allowed to reconstruct the trajectories and quantify motor parameters. A semi-cylindrical screen made of semitransparent tracing paper was placed in front of the treadmill (i.e. of the walking bee; Fig. 1A). Images were projected onto this screen by a video projector placed behind it.

Bees were trained to discriminate a green from a blue vertical bar against a black background during ten conditioning trials (Fig. 1C; see Supplementary Fig. 1 for color characteristics). Experiments were performed under 2D closed-loop conditions so that the movements of the walking bee displaced the bars laterally on the screen to bring them towards or away from front of the bee. During training, one of the bars (CS+) was rewarded with 1 M sucrose solution while the other bar (CS-) was punished with an aversive 3M NaCl solution 34- 36 . A choice was recorded when the bee moved one rectangle to the center of the screen (i.e., between -12.5° and +12.5° of the bee's central axis; see Fig. 1D, right).

We segregated learners and non-learners according to the bees' performance in a dedicated unrewarded test at the end of the training. Learners (n=23) were those bees that showed successful discrimination in the test (i.e. which chose the CS+). Non-learners (n=17), were those bees that did not succeed in the test (i.e. they either chose the CS-or did not make a choice). Importantly, these bees have the same sensory experience in terms of exposure to the color stimuli and reinforcements as our training procedure froze the CS+ or the CS-stimuli in front of the bee during 8 s upon a choice and delivered the reinforcements accordingly. Bees that did neither choose the CS+ nor the CS-in at least 5 trials were excluded from the analysis.

Acquisition was significant for learners during conditioning trials (Fig. 2A; CS*Trial effect: χ2=47.746, df:2, p<0.0001), thus showing that the categorization made based on test performance reflected well learning success. The percentages of bees responding to the CS+ and to the CS-differed significantly along trials (CS+ vs. CS-: CS*Trial; z=6.845, p<0.0001).

Significant differences were also found between the bees responding to the CS-and the nonresponders (CS-vs NC: CS*Trial; z=3.541, p=0.0004) but not between bees responding to the CS+ and non-responders (CS+ vs. NC: CS*Trial; z=- 1.201, p=0.23). Non-learners (n=17) did also show a significant CS*Trial effect (Fig. 2B; χ2=9.8383, df:2, p=0.007), but this effect was introduced by the non-responders. These bees differed significantly along trials both from the bees responding to the CS+ (CS+ vs. NC: CS*Trial; z=2.356, p=0.019) and from the bees responding to the CS-(CS-vs. NC: CS*Trial; z=3.068, p=0.002). On the contrary, the percentages of bees responding to the CS+ and to the CS-did not vary along trials (CS+ vs.

CS-: CS*Trial; z=1.437, p=0.2), consistently with the absence of learning. Learners and non-learners did not differ in their motor activity during training, thus excluding this factor as determinant of possible changes in neural activity. When walking speeds and the distances travelled were compared between groups, no significant differences were detected (Distance: Group; χ2=1.93, df:1, p=0.16; Speed: Group; χ2=1.78, df:1, p=0.18).

In the non-reinforced test, per definition learners (Fig. 2C) chose correctly the CS+ (100% of the bees) while non-learners (Fig. 2D) did either chose the CS-(35%) or did not perform any choice (65%). Learners spent more time fixating the CS+ than the CS-consistently with the choice made during the test (Wilcoxon signed rank exact test: V=17, p<0.0001) while non-learners did not differ in their fixation time for both stimuli in spite of a tendency to fixate more the CS-(V=26, p= 0.05).

Molecular analyses

We aimed at determining if visual learning in the 2D VR induces transcriptional changes revealing the neural trace of the associative learning described in the previous section. To this end, we performed RT-qPCR in individual brains of learners (n=22; one learner brain was lost during the dissection process) and non-learners (n=17), focusing on three main brain sections (Fig. 3A): the optic lobes (OLs), the calyces of the mushroom bodies (MB) and the remaining central brain (CB), which included mainly the central complex, the subesophageal zone and the peduncula of the mushroom-bodies (and  lobes). Brains were collected one hour after the retention test, which ensures that expression of all three genes was already induced (typically, from 15 to 90 min in the case of kakusei 31, 37 and 30-60 min in the case of Hr38 and Egr1 28, 29 ).

Two reference genes were used for the normalization (see Table 1): Ef1a (E=106%) and Actin (E=110%) 38 . Within-brain structure analyses showed that reference genes did not vary between learners and non-learners (t test; all comparisons NS; see Suppl Fig. 3) thus enabling further comparisons between these two categories with respect to the three target IEGs. To this end, the normalization procedure used the geometric mean of the two reference genes. No crosscomparisons between brain regions or genes were performed.

Figures 3 B-D, E-G, and H-J show the relative normalized expression of kakusei, Hr38

and Egr1, respectively, for the three brain regions considered in the case of learners and nonlearners. Significant variations of normalized expression between learners and non-learners were found for the three IEGs: in the case of kakusei and Hr38, these differences were restricted to the MBs (kakusei: Fig. 3C; two-sample t test; t = -2.23; df:37; p=0.03; Hr38: Fig. 3F; t = -2.39; df:37; p=0.02) while in the case of Egr1, they were observed in the optic lobes (Egr1: Fig. 3H; t = -2.32; df:37; p=0.03). All other within-structure comparisons between learners and non-learners were not significant (p˃0.05). Notably, in the three cases in which significant variations of IEG expression were found, learners exhibited a downregulation of IEG expression with respect of non-learners. In addition, from the three cases, two referred to the MB calyces, which indicates the important role of this region for visual learning and memory.

Discussion

The present work studied visual learning under a restrictive 2D VR environment and confirmed that bees can learn to discriminate visual stimuli based on their color under these artificial conditions. Walking parameters did not differ between learners and non-learners so that changes in IEG expression could be ascribed to learning success. We showed that associative color learning leads to a downregulation of the three IEGs considered in different areas of the visual circuit. While Egr1 was downregulated in the optic lobes, Hr38 and kakusei were coincidently downregulated in the MB calyces. Our work thus reveals that the neural trace of associative color learning in the bee brain is distributed along the sequential pathway of color processing and highlights the importance of MBs for color learning in bees.

IEG downregulation in the bee brain

We observed an IEG downregulation both in the optic lobes and the calyces of the MBs. This phenomenon is interesting as increased neural activity resulting from experience-dependent phenomena is usually reflected by an upregulation of IEG expression 24 . Typically, upon neural responses, a relatively rapid and transient pulse of gene expression may occur, which corresponds to an experience-dependent activation of the underlying synaptic circuitry 23, 39 . In our case, however, the downregulation observed indicates that a different form of experiencedependent change in neural activity occurred as a consequence of learning. A possible explanation for this phenomenon may put the accent on an inhibition of neural activity in key visual areasoptic lobes and mushroom bodies -of the learner group.

In the optic lobes, Erg1 downregulation may correspond to an increased GABAergic inhibitory activity associated with learning. The optic lobes exhibit multiple GABAergic fibers distributed principally in the medulla and the lobula 40 so that neural activity in these regions is subjected to intense GABAergic inhibitory signaling. Higher GABAergic activity has been reported in the optic lobes of forager bees via quantification of Amgad, the honey bee homolog of the gene responsible for synthesizing the enzyme GAD 41 , which catalyzes the decarboxylation of glutamate to GABA. This increase was accompanied by an increase in kakusei 41 , which we did not observe. Yet, we did not study foraging behavior in a natural context, but associative learning in a controlled laboratory context. Natural foraging may involve multiple behavioral components and stimulations that may be responsible for the increase of kakusei that was absent in our study. The interesting finding is, however, that Amgad expression revealed higher GABAergic neuron activity in the optic lobes of foragers, confirming the importance of inhibitory feedback for sustaining experience-dependent visual responses. This conclusion is supported by observed increases of GABA titers in the honey bee optic lobes upon restart of foraging activities 42 and by findings in fruit flies indicating that GABA-ergic neurons in the optic lobes are involved in tuning the sensitivity and selectivity of different visual channels 43, 44 .

In the calyces of the MBs, where coincident downregulation of kakusei and Hr38 was found, neural inhibition is provided by GABAergic feedback neurons (the so-called Av3 neurons) 45 , which are responsible for the sparse coding responses exhibited by Kenyon cells, the constitutive neurons of the MBs. Similar GABAergic neurons exist in fruit flies, which provide inhibitory feedback to the MBs. These neurons, termed APL (anterior paired lateral) neurons, are necessary for discrimination learning of similar odorants. When flies are trained to discriminate odorants in a simple differential conditioning, disrupting the Kenyon cell-APL feedback loop decreases the sparseness of Kenyon cell odor responses, increases inter-odor correlations and prevents flies from learning to discriminate similar, but not dissimilar, odors 46 .

Inhibitory feedback onto the calyces of honey bees is needed for solving patterning tasks in which insects have to suppress summation of responses to single elements previously rewarded when they are presented in an unrewarded compound 47 (i.e. animals have to learn to respond to the elements and not to the compound) or for reversal learning 48 . A similar conclusion applies to fruit flies as GABAergic input to the MBs provided by APL neurons also mediates the capacity to solve patterning tasks 49 . Increased feedback inhibition at the level of the MBs may therefore appear as a hallmark of certain learning phenomena, which require enhanced neural sparseness to decorrelate stimulus representations and thus memory specificity. In our experiments, both kakusei and Hr38 were subjected to downregulation in the MBs as a consequence of learning, a phenomenon that may be due to plastic changes in GABAergic signaling in the calyces of the MBs. Importantly, other visual areas such as the central complex 50 or the anterior optic tuberculum 51, 52 , among others, could exhibit similar variations undetectable for our methods as sectioning the frozen bee brain for molecular analyses does not allow a fine dissection of these areas.

IEG downregulation is not a common phenomenon as upregulation is usually reported to indicate the presence of neural activation 22 . Our hypothesis on neural inhibition being the cause for this downregulation requires, therefore, to be considered with caution. Further experiments are necessary to validate it, usingfor instanceelectrophysiological recordings in key areas of the visual circuits of learners to verify that neural activity is indeed sparser therein compared to non-learners. In addition, quantifying IEG expression in preparations in which neural inhibition has been characterized extensively at the cellular level such as in the case of hippocampal and cerebellum slices exhibiting long-term depression (LTD) 53 could be also important.

The neural signature of associative learning differs between different forms of VR

While the main finding in our experiments refer to a downregulation of IEG genes in key regions of the visual circuit, our previous work using a different 3D VR system yielded a different result 22 . In this 3D VR, bees could explore the virtual surroundings around the stimuli to be learned (not bars, but cuboids that expanded upon forward movements of the bee) and could displace these stimuli laterally and in depth. They explored and learned to discriminate the color stimuli proposed to them and their learning success was comparable, yet slightly lower than that observed in the 2D VR arena (50% vs. 55%, respectively). IEG analyses comparing learners and non-learners in the 3D VR reported an upregulation of Egr1 expression in the MB calyces of learners but not of non-learners. No other change was detected for kakusei and Hr38

in the same three brain regions considered in the present work 22 .

These differences are difficult to interpret as the 2D and the 3D VR experiments were not done simultaneously but in different years, though in similar seasons. In both cases, motivated foragers captured at a feeder were used for the experiments. The previous visual experience of these foragers may have differed across individual and between years, thus leading to differences in performances. This explanation seems, however, rather implausible

given that in bees rely on the most recent appetitive learning as the one guiding predominantly actual choices. In addition, irrespective of differences in the VR environments and the resulting difference in VR immersivity, the experiments were done under similar handling conditions and using strictly the same behavioral criteria. Gene analyses were also performed under the same conditions and using the same materials and methods. Thus, the contrasting results obtained in the two VR scenarios may be due to the distinct constraints they imposed to achieve discrimination learning and to the fact that the two scenarios may engage different acquisition mechanisms for learning visual information. In the 3D scenario, bees explored both the stimuli the vertical color cuboidsand the imaginary empty surroundings; they could return to the stimuli if they missed them and walk around them, which added an important exploratory component that was absent in the 2D arena. In the latter case, although bees could also bring back the stimuli if they missed them by walking too fast, such a control was restricted to the frontal plane and did not allow for three-dimensional inspection. Erg1 upregulation in the 3D VR upon learning may thus reflect the convergent effects of an exploratory drive and learning in a non-constrained environment. It cannot be due to a pure exploration of the environment as non-learners exhibited the same motor performances and did not show Egr1 upregulation.

In the 2D VR, bees were forced to control tightly the lateral displacements of the stimuli the color rectangleswithout any further change allowed. This environment and task may thus impose a higher stimulus and movement control and force the bee to focus exclusively and artificially on lateral stimulus movements to gain access to sucrose reward and avoid aversive saline solution. Although in both VR scenarios the background was empty and only the training stimuli were visible, the 2D VR missed the expansion of the images upon approach and thus lacked of immersivity. In this context, GABA-mediated inhibition may act as a gain control mechanism that enhances response efficiency and stimulus control. In the primary visual cortex (V1) of vertebrates, GABA inhibition has been proposed to play a fundamental role in establishing selectivity for stimulus orientation and direction of motion 54-56 . As the latter is particularly important in the 2D VR, enhanced GABA inhibition could be associated with learning to master the visual discrimination in this context.

In addition, a different, yet compatible explanation for the different patterns of IEG expression found in the 3D and the 2D VR refers to a possible difference in the visual acquisition mechanisms recruited by these two scenarios. In a navigation task, body movement translates into a displacement and a recognizable change in the visual scene, which can then be computed against the available internal information about the displacement 57 . These pathfinding, closed-loop actions can be viewed as different from motor actions that are contingent on reinforcement such as operant behaviors produced when a visual discriminative stimulus is present 58 . In the latter case, vision is also engaged in discrimination learning but in a context that is not navigational. Visual learning in the 2D VR could be seen as a form of operant learning in which colors define the action to be produced to obtain the appropriate reinforcement. Thus, the observed difference in IEG expression between the two types of VR may reflect a difference in the mechanisms used to reach the rewarded stimulus.

The role of mushroom bodies for visual learning and memory

Our work highlights the participation of mushroom bodies in visual learning and short-term visual retention. Numerous works have demonstrated the necessity of these brain structures for the acquisition, storage and retrieval of olfactory memories in bees 8, 59, 60 and other insects 2, 3, 61 . Yet, less is known about their implication in visually-driven behavioral and neural plasticity 62, 63 . In our study, the full control over sensory stimulation offered by the VR system allowed a sound comparison between the brain of learners and non-learners, which revealed a neural signature for visual learning that included the mushroom bodies.

The implication of mushroom bodies in visual learning and memory in the bee is expected given the parallels between visual and olfactory inputs at the level of the calyces.

While afferent projection neurons convey olfactory information to the lip, a subdivision of the calyces 64 , afferent neurons from the lobula and the medulla, which are part of the optic lobes, convey visual information to other calyx subdivisions, the collar and the basal ring 65, 66 . In spite of this similarity, studies addressing the role of mushroom bodies in honey bee visual learning and memory remain rare.

Bees learning to associate color lights with the presence or absence of an electric shock in a double-compartment box 38, 67 require the ventral lobe of the mushroom bodies to learn to avoid the punished color and spend more time in the safe color 68 . In the same study, pharmacological blockade of one of the four collars (two per MB) had no effect on discrimination learning 68 , which does not exclude a participation of this MB region in this visual learning given that the remaining three calyces could compensate for the loss of the blocked one. In a different study, upregulation of the dopamine receptor Amdop1 was found in the calyces of the MBs when bees were trained to inhibit positive phototaxis towards a colored light 38 .

More recently, the implication of MBs in visual navigation was shown in wood ants Formica rufa, which are innately attracted to large visual cues (i.e. a large vertical black rectangle) and which can nevertheless be trained to locate and travel to a food source placed at a specific angle away from the attractive black rectangle 69 . When their MB calyces were blocked by injection of procaine 70, 71 , ants reverted their trajectories towards the attractive rectangle, which suggests a role for mushroom bodies in the dissociation between innate and learned visual responses, and in navigational memory 69 . In another study involving the ant Myrmecia midas, procaine was again used to block MB function via delivery into the vertical lobes and evaluate the impact of this blockade in orientation in a familiar environment 72 .

Experienced forager with procaine-inactivated VLs had tortuous paths and were unable to find their nest, whereas control ants were well directed and successful at returning home 72 . Overall, these two studies on ant navigation indicate that the vertical lobes of MBs are necessary for retrieving visual memories for successful view-based navigation.

Studies on the role of MBs for visual learning and memory in fruit flies have yielded contradictory findings. Mushroom body deficits do not affect learning success in the flight simulator, a setup in which tethered flies in stationary flight learn to avoid quadrants associated with specific visual landmarks based on the presence of an aversive heat beam pointed towards their thorax 73 . Similarly, learning to discriminate colors in a cylindrical container made of a blue-lit and a yellow-lit compartment, one of which was associated with aversive shaking of the flies, was not affected in mushroom body mutants 74 . Spatial learning of a non-heated spot in an otherwise heated cylindrical arena displaying surrounding visual landmarks is possible in the absence of functional mushroom bodies but not of the central complex 75 . Although these various results points toward a dispensability of MBs for visual learning in fruit flies 73 , experiments comparing appetitive and aversive color learning and discrimination question this view 76 . When blue and green colors were presented from below in an arena, walking flies learned both the appetitive (based on pairing one color with sugar) and the aversive discrimination (based on pairing one color with electric shock) but failed if MB function was blocked using neurogenetic tools 76 . Furthermore, MBs are required for visual context generalization (e.g. generalizing landmark discrimination in a flight simulator in which contextual light was switched from blue to green between training and test) 77-79 . Thus, MBs participate in different forms of visual learning in fruit flies, although their involvement in these phenomena seems to be less clear than in other insects.

Taken together, our results revealed that learning a visual discrimination under a 2D

VR, in which closed-loop conditions restricted stimulus control to lateral displacements, induced a neural signature that spanned the optic lobes and MB calyces and that was characterized by IEG downregulation, consistent with an inhibitory trace. This trace may vary and become excitatory in more permissive VR conditions in which closed-loop conditions allow for 3D exploration during discrimination learning 22 .

Materials and Methods

Honey bees (Apis mellifera) were obtained from our apiary located at the campus of the University Paul Sabatier -Toulouse III during September 2021. Only foragers caught upon landing on a gravity feeder filled with a 0.9 M sucrose solution were used in our experiments to ensure high appetitive motivation. Captured bees were enclosed in individual glass vials and then transferred to small cages housing ten bees in average; caged bees had access to ad libitum water and to 300 μl of 1.5 M sucrose solution. They were kept overnight in an incubator at 28 °C and 80% humidity. On the next day, they were placed on ice for five minutes to anesthetize them and facilitate the gluing of a tether to their thorax by means of melted wax (Fig. 1A). After being attached to the tether, each bee was placed on a small (5 cm diameter) Styrofoam ball for familiarization with the treadmill situation. Bees were provided with 5 μl of 1.5 M sucrose solution and kept for 3 h in this provisory setup in the dark. They were then moved to the VR arena and used for the experiments.

Once in the VR setup, the bee was attached to a holder that allowed adjusting its position on the treadmill (Fig. 1B), a polystyrene ball (diameter: 5 cm, weight: 1.07 g) held by a 3Dprinted support and floating on a constant airflow produced by an air pump (airflow: 555ml/s; Aqua Oxy CWS 2000, Oase, Wasquehal, France).

VR setup

The VR setup consisted of the treadmill and of a half-cylindrical vertical screen made of semitransparent tracing paper, which allowed presentation of a 180° visual environment to the bee (diameter: 268 mm, height: 200 mm, distance to the bee: 9 cm Fig. 1ABC) and which was placed in front of the treadmill. The visual environment was projected from behind the screen using a video projector connected to a laptop (Fig. 1A). The video projector was an Acer K135 (Lamp: LED, Maximum Vertical Sync: 120 Hz, Definition: 1280 x 800, Minimum Vertical Sync: 50 Hz, Brightness: 600 lumens, Maximum Horizontal Sync: 100.10 3 Hz, Contrast ratio: 10 000:1, Minimum Horizontal Sync: 30.10 3 Hz) 15 . The movements of the walking bee on the treadmill were recorded by two infrared optic-mouse sensors (Logitech M500, 1000 dpi, Logitech, Lausanne, Switzerland) placed on the ball support perpendicular to each other.

Experiments were conducted under 2D closed-loop conditions, i.e. rotations of the ball displaced the visual stimuli only laterally. To this end, we used a custom software developed using the Unity engine (version 2018.3.11f1), open-source code available at https://github.com/G-Lafon/BeeVR 21 . The software updated the position of the bee within the VR every 0.017 s.

Visual stimuli

Bees had to discriminate two vertical rectangles (Fig. 1C) based on their different colors and association with reward and punishment. The colors of the rectangles (see supplementary Fig. S1) were blue (RGB: 0, 0, 255, with a dominant wavelength of 450 nm and an irradiance of 161000 μW) and green (RGB: 0, 100, 0, with a dominant wavelength of 530 nm and an irradiance of 24370 μW/cm2). They were displayed on a black background (RGB: 0, 0, 0).

These colors were chosen based on previous work showing their successful learning in the VR setup 15, 21 .

Each rectangle had a 5 cm base and occupied the entire vertical extent of the screen. The rectangles were positioned at -50° and +50° from the bee's body axis at the beginning of each trial (Fig. 1D, left). Keeping the object within -12.5° and +12.5° in front of the central axis of the bee continuously for 1 s was recorded as a choice (Fig. 1D, right).

Conditioning and testing at the treadmill

Bees were trained using a differential conditioning, which promotes better learning performances owing to the presence of penalized incorrect color choice that result in an enhancement of visual attention 80 .

Bees were trained during 10 consecutive trials using a differential conditioning procedure (Fig. 1E) in which one of the rectangles (i.e. one of the two colors, green or blue)

was rewarded with 1.5 M sucrose solution (the appetitive conditioned stimulus or CS+) while the other rectangle displaying the alternative color (the aversive conditioned stimulus or CS-)

was associated with 3 M NaCl solution. The latter was used to increase the penalty of incorrect choices 34-36, 81 . To avoid directional biases, the rewarded and the punished color rectangles were swapped between the left and the right side of the virtual arena in a pseudo random manner along trials.

At the beginning of the experiment, bees were presented with a dark screen. During training trials, each bee faced the two rectangles (Fig. 1D, left). Choice of the CS+ rectangle was recorded if the bee kept it at the center of the screen (between -12.5° and +12.5° of the bee's central axis) during 1 s (Fig. 1D, right). Training was balanced in terms of color contingencies (i.e. blue and green equally rewarded across bees) based on a random assignment by the VR software. If the bee kept the CS+ in the center of the screen continuously during 1 s (i.e. if it chose it), the screen was locked on that image for 8 s. This allowed the delivery of sucrose solution in case of a correct choice, or of NaCl in case of an incorrect choice. Solutions were delivered for 3 s by the experimenter who sat behind the bee and used a toothpick to this end. The toothpick contacted first the antennae and then the mouthparts while the screen was locked on the visual image fixated by the bee. A different toothpick was used for each tastant.

Each training trial lasted until the bee chose one of the two stimuli or until a maximum of 60 s (no choice). Trials were separated by an inter-trial interval of 60 s during which the dark screen was presented. Bees that were unable to choose a stimulus (i.e. that did not fulfill the criterion of a choice defined above) in at least 5 trials were excluded from the analysis. From 50 bees trained, 40 were kept for analysis (~80%).

After the last training trial, each bee was subjected to a non-reinforced test that lasted 60 s (Fig. 1E). Test performance allowed distinguishing learners (i.e. bees that chose the CS+ as their first choice in the test) from non-learners (i.e. bees that either chose the CS-in their first test choice or that did not make any choice during the test). IEG expression was compared between these two groups, which had the same sensory experience in the VR setup and which differed only in their learning success.

Brain dissection

One hour after the test, the bee was sacrificed and its head was instantly frozen in a nitrogen solution. The frozen head was dissected on dry ice under a binocular microscope. First, the antennae were removed and a window was cut in the upper part of the head capsule, removing the cuticle between the compound eyes and the ocelli. Second, the glands and tracheae around the brain were removed. Third, the retinas of the compound eyes were also removed.

The frozen brain was cut in three main parts for IEG analyses (Fig. 3A): the optic lobes (OL), the upper part of the mushroom bodies (the mushroom-body calyces, MB Ca) and the remaining central brain (CB), which included mainly the peduncula of the mushroom-bodies (and  lobes), the central complex (CC), the antennal lobes (AL) and the subesophageal zone (SEZ). Samples were stored at -80 °C before RNA extraction. During the dissection process, one learner brain was lost so that learner sample sizes differ between the behavioral (n=23) and the molecular analyses (n=22).

RNA extraction and reverse transcription

The RNAs from the three sections mentioned above (OL, MB Ca and CB) were extracted using the RNeasy Micro Kit (Qiagen). The final RNA concentration obtained was measured by spectrophotometry (NanoDrop™ One, Thermo Scientific). A volume of 10 µl containing 100 ng of the RNA obtained was used for reverse transcription following the procedure recommended in the Maxima H Minus First Strand cDNA Synthesis kit (Thermo Scientific, 0.25 µl of random hexamer primer, 1 µl of 10 mM dNTP mix, 3.75 µl of nuclease free H2O, 4 µl 5X RT Buffer and 1 µl Maxima H Minus Enzyme Mix).

Quantitative Polymerase Chain Reaction (RT-qPCR)

All the primers used for target and reference genes generated amplification products of approximately 150 bp. The efficiencies of all reactions with the different primers used were between 95% and 110 % (Table 1). Their specificity was verified by analyzing melting curves of the RT-qPCR products (see Supplementary Fig. S2). Two reference genes (Ef1 and Actin)

were used for normalization.

Expression was quantified using a SYBR Green real-time PCR method. (with E= efficiency of the reaction). Then a normalization factor for each sample was obtained computing the geometric mean of the relative quantities obtained for the reference genes in the corresponding samples (ΔΔCt).

Data analysis and statistics

Behavioral data

The first choice of the bees was recorded during the conditioning trials and the non-reinforced test. In this way, we established for each trial and test the percentages of bees choosing first each of the stimuli displayed or not choosing a stimulus (± 95% confidence interval).

Test percentages were analyzed within groups by means of a generalized linear mixed model (GLMM) for binomial family in which the individual identity (Bee) was considered as a random factor (individual effect) while the choice category (CS+, CS-, NC) was fitted as a fixed effect; z values with corresponding degrees of freedom are reported throughout for this kind of analysis.

For the acquisition trials, we recorded motor variables such as the total distance walked during a trial, and the walking speed. The analysis of these continuous variables was done using a linear mixed model (lmer function) in which the individual identity (Bee ID) was a random factor and the factors Group (i.e. learner or non-learner) and Trial were fixed.

Statistical analyses were performed using with R 3.5.1 82 . The package lme4 was used for GLMMs and LMMs.

Gene expression data

Statistical differences in gene expression were assessed for reference genes to check for stability and for target genes within a given brain region using One-Factor ANOVA for independent groups in the case of multiple comparisons or two-sample T test in the case of dual comparisons.

Pots hoc comparisons between groups were performed by means of a Tukey test following ANOVA. No cross-comparisons between brain regions or genes were performed due to within-area normalization procedures. Statistical analyses were done either with R 3.5.1 software 82 or with Statistica 13 Software (TIBCO® Data Science).

Introduction

Among insects, honeybees represent a valuable model system for cognitive research. Thanks to a rich behavioral repertoire supported by a small brain, they allow easy access to the neural correlate of cognitive processes like learning and memory (Giurfa, 2007). The last decades have also seen the emergence of bumblebees as a model in cognition [START_REF] Real | Information Processing and the Evolutionary Ecology of Cognitive Architecture[END_REF]Dyer and Chittka, 2004;Worden et al., 2005;[START_REF] Kulahci | Multimodal signals enhance decision making in foraging bumble-bees[END_REF] Bumblebees are eusocial central place forager that despite being closely related to honeybees differ in a few important points (A. J. Riveros and Gronenberg, 2009). Labor division in bumblebees is strongly influenced by size, with larger bees assuming the role of foragers while smaller worker stay in the hives (Goulson, 2003). Bumblebees' colonies are smaller than honeybees' with only between 200-400 worker instead of the tens of thousands typically found in honeybees' hives. This contributes to bumblebees being more easily reared under laboratory conditions (A. J. Riveros and Gronenberg, 2009). Bumblebees are solitary foragers, unlike honeybees they do not share intentionally information about location and quality of potential resources with their nest mates (Leadbeater and Chittka, 2005;Worden et al., 2005;Leadbeater and Chittka, 2007) which makes them an interesting model to study foraging strategies [START_REF] Lihoreau | Unravelling the mechanisms of trapline foraging in bees[END_REF].

Related to their central place forager ecology, vision plays a central role in major aspects of bumblebees' life histories, from navigation [START_REF] Church | Spatial encoding by bumblebees (Bombus impatiens) of a reward within an artificial flower array[END_REF][START_REF] Saleh | Traplining in bumblebees (Bombus impatiens): a foraging strategy's ontogeny and the importance of spatial reference memory in short-range foraging[END_REF] to floral selection [START_REF] Dukas | Categorization of food types enhances foraging performance of bumblebees[END_REF]Laverty, 1994;[START_REF] Cnaani | Flower Choice and Learning in Foraging Bumblebees: Effects of Variation in Nectar Volume and Concentration[END_REF]. In the past decades, bumblebees have been extensively used to investigate cognitive processes under free flight conditions [START_REF] Heinrich | Laboratory Analysis of Flower Constancy in Foraging Bumblebees: Bombus ternarius and B. terricola[END_REF][START_REF] Real | Information Processing and the Evolutionary Ecology of Cognitive Architecture[END_REF][START_REF] Keasar | Overnight memory retention of foraging skills by bumblebees is imperfect[END_REF][START_REF] Chittka | Sensori-motor learning and its relevance for task specialization in bumble bees[END_REF]Worden et al., 2005;[START_REF] Cnaani | Flower Choice and Learning in Foraging Bumblebees: Effects of Variation in Nectar Volume and Concentration[END_REF][START_REF] Kulahci | Multimodal signals enhance decision making in foraging bumble-bees[END_REF]A. J. Riveros and Gronenberg, 2009;[START_REF] Leonard | Flowers help bees cope with uncertainty: signal detection and the function of floral complexity[END_REF][START_REF] Mertes | Visual motion-sensitive neurons in the bumblebee brain convey information about landmarks during a navigational task[END_REF][START_REF] Foster | Bumblebees learn polarization patterns[END_REF][START_REF] Robert | Variations on a theme: bumblebee learning flights from the nest and from flowers[END_REF][START_REF] Li | Large-scale transcriptome changes in the process of long-term visual memory formation in the bumblebee, Bombus terrestris[END_REF][START_REF] Frasnelli | Small and Large Bumblebees Invest Differently when Learning about Flowers[END_REF]. Bumblebees were also tested in harnessed conditions with olfactory conditioning of the PER [START_REF] Riddell | Insect psychoneuroimmunology: Immune response reduces learning in protein starved bumblebees (Bombus terrestris)[END_REF][START_REF] Riveros | Olfactory learning and memory in the bumblebee Bombus occidentalis[END_REF]A. J. Riveros and Gronenberg, 2009;[START_REF] Toda | Bumblebees exhibit the memory spacing effect[END_REF][START_REF] Anfora | Lateralization in the Invertebrate Brain: Left-Right Asymmetry of Olfaction in Bumble Bee, Bombus terrestris[END_REF]. Initially developed for honeybees (Bitterman et al., 1983), it allows a deep control of the different experimental parameters and makes it possible to couple learning paradigms with invasive neurobiological techniques (Giurfa and Sandoz, 2012). More recently, visual versions of the PER conditioning have also been proposed [START_REF] Riveros | Decision-making and associative color learning in harnessed bumblebees (Bombus impatiens)[END_REF][START_REF] Lichtenstein | Dumb and Lazy? A Comparison of Color Learning and Memory Retrieval in Drones and Workers of the Buff-Tailed Bumblebee, Bombus terrestris, by Means of PER Conditioning[END_REF][START_REF] Muth | A pollen fatty acid enhances learning and survival in bumblebees[END_REF][START_REF] Riveros | Learning of bimodal versus unimodal signals in restrained bumble bees[END_REF]. Not only can bumblebees successfully learn in harnessed condition but they are also robust and reliable during electrophysiology recordings [START_REF] Skorupski | Photoreceptor spectral sensitivity in island and mainland populations of the bumblebee, Bombus terrestris[END_REF][START_REF] Spaethe | Size determines antennal sensitivity and behavioral threshold to odors in bumblebee workers[END_REF]Paulk et al., 2008;Paulk and Gronenberg, 2008;Skorupski andChittka, 2010a, 2010b;Vähäkainu et al., 2013;Rusanen et al., 2017) or calcium brain imaging [START_REF] Mertes | Olfactory coding in the antennal lobe of the bumble bee Bombus terrestris[END_REF].

We have recently developed a Virtual reality (VR) setup that allows to successfully train restrained honeybees in a visual differential conditioning task (Buatois et al., 2017;Lafon et al., 2021). Such a setup is intended to open up further possibilities to explore the underlying mechanisms of visual learning by facilitating a live access to the brain of a behaving insect. In that context, bumblebees appear like a very interesting model since they display some of the rich behavioral repertoire of the honeybees but with a bigger brain and a more robust body which make them more compatible with invasive protocols like electrophysiology and calcium imaging.

Here we assessed if and how the bumblebee Bombus terrestris can learn a visual discrimination task under VR conditions. We measured bumblebee's performances during the learning phase and a subsequent non-reinforced test and compared them with honeybees conditioned with the same protocol. We also performed two conditioning experiments using either NaCl or Quinine solutions as a punishment, to find the best way to negatively reinforce a stimulus. Since size plays an important role in bumblebees' labor division (Goulson, 2003), we could expect larger bumblebees, that are usually foragers, to perform better in the color discrimination task than smaller workers. Moreover, our 3D VR setup involves a significant motor component as bees have to move in the virtual environment to reach the rewarded stimuli, so we hypothesize that the insect's strength could impact bee's performances.

Materials and methods

Study species and collection

Bumblebees were collected each morning around 9 am from twenty commercial colonies of B.

terrestris, sixteen for experiment 2, during July 2021, and four for experiment 3, during November 2021 (Koppert, Cavaillon, France) by placing a glass vial at the entrance of the box and collecting the workers that came out into the vial. Each colony contained about 200 workers, brood, and 1 queen. Bumblebees were maintained and tested in the laboratory at 25°C and 30-40% relative humidity, under a 12:12 h light:dark photocycle.

Honey bee foragers (Apis mellifera) were obtained from the CRCA apiary located in the campus of the University Paul Sabatier during July 2021. Foragers were captured the day before the experiment at gravity feeders providing 0.88 M sucrose solution upon landing and before they began feeding. Captured bees were enclosed in individual glass vials and then transferred to small cages housing ten bees in average; where they had access to ad libitum water and 300 µl of 1.5 M sucrose solution. They were then kept overnight in an incubator at 28°C and 80% humidity.

Bees, bumblebees and honeybees, were cooled on ice for 5 minutes to anesthetize them and attach them to their tether. Bees were handled under red light, which ensured a dark environment to the insects. Virtual reality set-up

The bee was then moved to the VR setup (Lafon et al., 2021). The VR relayed on a custom software developed using the Unity engine (version 2020. The ball was positioned in front of a half-cylindrical vertical screen, 268 mm in diameter and 200 mm height, which was placed at 9 cm from the bee. The screen was made of semitransparent tracing paper, which allowed presentation of a 180° visual environment to the bee.

The visual environment was projected from behind the screen using a video projector connected to a laptop. The video projector was an Acer K135 (Lamp: LED, Maximum Vertical Sync: 120

Hz, Definition: 1280 x 800, Minimum Vertical Sync: 50 Hz, Brightness: 600 lumens, Maximum Horizontal Sync: 100.10 3 Hz, Contrast ratio: 10 000:1, Minimum Horizontal Sync: 30.10 3 Hz).

The lag between the motion of the bee and the update of the visual surrounding was of 18.00 ± 2.53 ms (mean ± S.E.; n =10) (Lafon et al., 2021).

Experiment 1: Establishing a balanced pair of green and blue for bumblebee conditioning

In order to find a pair of colors that elicited the same amount of attraction we recorded the spontaneous choice of bumblebee presented with two pairs of green and blue stimulus. One pair called Green (R: 0, G:100, B:1.0; irradiance 24 370 μW/cm²) versus Bright Blue (R: 0, G:80, B:254) and a second pair called Dark Green (R: 0, G:51,B:1.0) versus Blue (R: 0, G:0, B:255; irradiance 161 000 μW/cm²).

Each cylinder had a 5 cm diameter base and 1 m height so that it occupied the entire vertical extent of the screen irrespective of the bumblebee's position. At the beginning of each test, it subtended a horizontal visual angle of 6.5° and was positioned either to the left (-50°) or the right (+50°) of the tethered insect. Approaching the cylinder resulted in an expansion of its horizontal extent (1.7°/cm). A choice was recorded when the bumblebee approached the cylinder within an area of 3 cm surrounding its virtual surface and directly faced its center (Fig. 2A).

Each insect was subjected to two consecutive tests in non-reinforced conditions. One test for each pair of colors, the order of presentation of the pairs was random. We recorded what color the bumblebee chose first. Each test lasted 180 s and the inter-test interval was 10 s. other cylinder displaying the alternative color (the aversive conditioned stimulus or CS-) was associated with either 3 M NaCl solution (experiment 2) or 1.2 g.L -1 quinine solution (experiment 3).

At the beginning of the experiment, bees were presented with a dark screen for 60 s. During training trials, each bee faced the virtual environment with the two cylinders in front of it. The bee had to learn to choose the CS+ cylinder by walking towards it and centering it on the screen.

Training was balanced in terms of color contingencies (i.e. blue and green equally rewarded across bees) based on a random assignment by the VR software. If the bee reached the CS+ within an area of 3 cm in the virtual environment (i.e. the chosen cylinder subtended a horizontal visual angle of 53°) and centered, the screen was locked on that image for 8s for honeybees (Lafon et al., 2021) or 14 s in case of bumblebees. The screen freezing was longer for bumblebees as we found in pre-experiments that they need more time to take the reward from the toothpick. This allowed the delivery of sucrose solution in case of a correct choice, or of NaCl (experiment 2) or quinine (experiment 3) in case of an incorrect choice. Solutions were delivered for 3 s by the experimenter who sat behind the bee and used a toothpick to this end.

The toothpick contacted first the antennae and then the mouthparts while the screen was locked on the visual image fixated by the bee.

Each training trial lasted until the bee chose one stimuli or until a maximum of 180 s (no choice).

Thus, a single choice (or a no choice) was recorded during each training trial. Trials were separated by an inter-trial interval of 60 s during which the dark screen was presented. The bees that were unable to choose a stimulus in at least 3 trials were excluded from the analysis. From 138 bumblebees trained in experiment 2, 123 were kept for analysis (~89%). From 77 honeybees trained, 31 were kept for analysis (~40%). In the experiment 3, out of 315 trained bumblebees, 235 were kept for analysis (~75%). Every animal was frozen at -20°C after the experiment to be later weighed and measured.

After the last training trial, each bee was subjected to a non-reinforced test that contrary to training trials had a fixed duration of 180 s. During this test, two variables were recorded: the first choice (as defined above) and the time spent fixating the rewarded and the non-rewarded stimulus. Both variables have been used in prior works performed in our VR setup to characterize test performances as they may reveal different aspects of behavioral performances (Buatois et al., 2020(Buatois et al., , 2018(Buatois et al., , 2017;;Lafon et al., 2021). Fixation time (s) was defined as the time spent by each cylinder at the center of the screen (± 2.5 mm) where it was brought by the bee's motor actions. We used the same ray-casting method as in Lafon et al. 2021(Lafon et al., 2021).

Weight and size measuring

Size was assessed by measuring the distance between the two wing joints using the Toupview software (ToupTek Photonics, Zhejiang, China). After size measurements insects were placed in an oven at 70°C for 4 hours in order to evaporate all water from their bodies. Dry weigh was measured with a precision scale (OHAUS, Nänikon, Switzerland).

Statistical analysis

Statistical analyses were performed using R software (R Core Team, 2020). In Experiment 1 (Color balancing), we counted the number of green and blue choices for each pair. We then used a chi² test to compare the distribution of green and blue of each pair with a theoretical distribution of 50% green and 50% blue. We report ꭓ², degrees of freedom and p-values for this analysis.

In Experiments 2 and 3, the first choice in each trial and test was categorized as choice of the CS+, choice of the CS-or no choice (NC). Thus, a bee choosing the CS+ was recorded as (1, 0, 0) for choice of the CS+, choice of the CS-and NC, respectively. Data were bootstrapped to plot the proportion of bees in each category with their corresponding 95% confidence interval.

Performances were analyzed using generalized mixed linear models (GLMM) with a binomial error structure-logit-link function (glmer function of R package lme4) [START_REF] Bates | Fitting Linear Mixed-Effects Models using lme4[END_REF]. The independent variables (fixed factors) were the species of bee (Species; Experiment 2), the trial number (Trial), the choice category (Choice) and the color of the CS+ when applicable (Color:

Blue or Green). Bee ID was included as a random factor to account for the repeated-measure design; z values with corresponding degrees of freedom are reported throughout for this kind of analysis. Post-hoc ANOVA were performed on those models to assess the impact of each factors. We report ꭓ² with corresponding degrees of freedom throughout for this kind of analysis.

During the tests of Experiments 2 and 3, we also recorded the time spent fixating the test alternatives (CS+ vs. CS-). Time values were compared using a Wilcoxon signed rank test.

For the acquisition trials, we recorded motor variables such as the total distance walked during a trial, and walking speed. In addition, we analyzed the latency to make a choice starting from the beginning of a trial to the moment in which a choice (either for the CS+ or the CS-) was recorded. NC data were excluded from the latency analysis. The analysis of these continuous variables was done using a linear mixed model (lmer function of R package lme4) in which the individual identity (Bee ID) was a random factor and the experimental condition (Condition)

and trial number (Trial) were fixed factors. Statistical analyses were performed using R version 4.0.2 (R Core Team, 2020).

Results

Experiment 1: Establishing a balanced pair of green and blue stimuli

In a first experiment, we recorded the spontaneous choice of bumblebee presented with two pairs of green and blue stimulus. One pair called Green (R: 0, G:100, B:1.0; irradiance 24 370 μW/cm²) versus Bright Blue (R: 0, G:80, B:254) and a second pair called Dark Green (R: 0, G:51, B:1.0) versus Blue (R: 0, G:0, B:254; irradiance 161 000 μW/cm²). Each bee was presented once with both pair in a random order. There was no significant effect of the order on the bees' choices (z = 1.094, p = 0.27). Thus we pooled the data of the two test sequences and represented for both pair the number of bee choosing green or blue (Fig. 3).

When presented with a choice between Dark green and Blue, the distribution of choices was not significantly different from 50% (Green: 19, Blue: 23, ꭓ² = 0.38095, df = 1, p = 0.54). By contrast, with the Green versus Bright Blue pair, the distribution was significantly different from a random choice (Green: 28, Blue: 11, ꭓ² = 7.41, df:1, p = 0.0065) as the bees preferred the green option. The results indicate that only the Dark Green/Blue pair elicited a balanced spontaneous choice. We therefore chose to use this pair of colors in the subsequent experiments.

Bumblebee learning performances

The color of the CS+ had no effect on the learning of the bumblebees (χ 2 = 0.48, df:2, p = 0.79).

Data were thus represented as a CS+ vs. a CS-discrimination irrespective of color identity (Fig. 5 A). Bumblebees learned to respond more to the CS+ than to the CS-(Fig. 5A). The interaction between trial number and bee choices was significant (χ2 = 7.51, df:2, p = 0.023). In the course of the 6 conditioning trials, the percentages of bees responding to the CS+ and that of bees responding to the CS-evolved differently (z = 2.06, p = 0.04), thus showing successful discrimination learning. Moreover, the dynamic of CS+ responding bees was also significantly different from that of the non-responding (NC) bees (z = 2.31, p = 0.021) while the difference between the dynamic of the CS-responding bees and the NC bees was not different (z = 1.083, p = 0.29). In the corresponding cumulative heat map (Fig. 5C), a clear V shape is visible, indicating that the bees did interact with both sides in the VR and walked towards the cylinders.

Honeybee learning performances

The color of the rewarded stimulus had no significant effect on honeybees 'ability to solve the discrimination task (χ2 = 1.903, df:2, p = 0.39). Data were thus pooled irrespective of the color of CS+ (Fig. 5B).

We found a significant interaction between the number of trials and the nature of the honeybees' choice (χ2 = 8.89, df:2, p = 0.012). Throughout the 6 trials the proportion of CS+ followed a different trajectory from that of CS-(z = 2.903, p = 0.0037), showing that the bees were able to successfully learn the discrimination (Fig. 5B). However, the dynamic of NC was neither different from CS-(z = -1.504, p = 0.133) nor from CS+ (z = 0.467, p = 0.641). The cumulative heat map representing the locations of the bees during their training trajectories (Fig. 5D) shows that, as with bumblebees, honeybees walked and interacted with both sides in the VR.

Test Performance

After the last training trial, each bee was subjected to a test in which the green and the blue cylinders were presented in extinction conditions (no reinforcement provided). We recorded the percentage of bees choosing correctly the CS+ or the CS-in their first choice, or not choosing (NC) and the time spent fixating the CS+ and the CS-(Fig. 7). Similarly, to the learning phase bee species had no effect on the choices during the non-reinforced test (χ2 = 0.407, df:2, p = 0.816).

Bumblebee test performances

Contrary to the training phase, bumblebees' choice was affected by the color of the reward during the test (Color: χ² = 9.532, df:2, p = 0.0085). We thus analyzed test results according to the color of reward (Fig. 7A C).

Insect dimensions and performances during the test

After training we measure the inter wing distance and the dry weight of every bee. When then explored if and how did the dimensions of the insects affected their performances during the test (Fig. 9,10,11). Larger bumblebees walked further than smaller ones (Fig. 9A), as distance walked was positively correlated with the size of the insect (F = 6.4756, df:1, p = 0.012) but not with weight (Fig. 10A) (F = 0.3171, df:1, p = 0.57). On the other hand, distance walked correlated neither with size (Fig. 9B; F = 1.0977, df:1, p = 0.30) nor with weight (Fig. 10B; F = 1.1189, df:1, p = 0.30) in honeybees. Larger bumblebees also walked faster than smaller ones (Fig. 9C) as speed was positively correlated with size in bumblebees (F = 6.4179, df:1, p = 0.013) but not with weight (Fig. 10C; F = 0.2985, df:1, p = 0.59) and neither with size (Fig. 9D; F = 1.0942, df:1, p = 0.30) nor with weight (Fig. 10D; F = 1.1113, df:1, p = 0.3) in honeybees.

Coherent with the effect observed on speed, bumblebees' latency to choose was negatively correlated with both size (Fig. 9E; F = 19.496, df:1, p < 0.0001) and weight (Fig. 10E; F = 6.8365, df:1, p = 0.01). Meaning that bigger bumblebees made their first choice faster, which makes sense considering they also walked faster (Fig. 9 A). For honeybees however, we found no clear effect of size (Fig. 9F) or weight (Fig. 10F) on the latency to choose (Size: F = 0.8738, df:1, p = 0.36; Weight: F = 0.1025, df:1, p = 0.75). It is likely that the weight and size variations in honeybees are too small to significantly impact motor performances. In order to measure bees learning performance during the conditioning we established a learning score. The learning score is computed as the number of CS+ choices minus the number of CS-choices from the second trial to the test. The size of the bee didn't not significantly affect learning success (Fig. 11A,B; Bumblebee: F = 8.10 -4 , df:1, p = 0.98; Honeybee: F = 2.0229, df:1, p = 0.17). Heavier honeybees had significantly lower learning scores than lighter ones (Fig. 11D; F = 5.3484, df:1, p = 0.03). However, no effect of weight was visible in bumblebees (Fig. 11C; F = 1.125, df:1, p = 0.291). Experiment 3: Conditioning bumblebees with Quinine as punishment

In a following experiment we conditioned bumblebees using 1.2 g.L -1 quinine solution as punishment (Dyer and Chittka, 2004). The experiment followed the same protocol from experiment 2, 6 trials of maximum 180 s spaced by 60 s inter-trials followed by one nonreinforced test. The bumblebees had to discriminate between a blue and a green cylinder, we used the colors found in experiment 1, the color of the reward was decided pseudo-randomly, green was rewarded as often as blue.

Discrimination learning during training

The evolution of choices across trials was not affected by the color of the reward (ꭓ² = 5.566, df:2, p = 0.62). Therefore, we pooled both rewarded condition together and only shows the choices in term of CS+ and CS-irrespective of the color of the cylinders (Fig. 12).

CS+ choices evolved differently from both CS-choices (z = -1.458, p < 0.00001) and nonchoices (NC) (Fig. 12A; z = 5.223, p <0.00001) as bees learned to respond more to CS+. On the other hand, CS-choices did no evolved differently from NCs (z = 1.458, p = 0.14). In the corresponding cumulative heat map (Fig. 12D), a clear V shape is visible, indicating that the bees did interact with both sides in the VR and walked towards the cylinders.

Motor and temporal components of bee trajectories during training

We also analyzed how the trajectory parameters of the bumblebees evolved over trials (Fig. 13). The distanced walked decreased significantly across trials (Fig. 13A; ꭓ² = 271.62, df:1, p < 0.00001) as bumblebees took more direct routes toward the cylinders. Speed increased significantly with trials (Fig. 13B; ꭓ² = 10.076, df:1, p = 0.0015). This increase was concomitant with an increase in the proportion of bees choosing CS+ (Fig. 12A) and may thus reveal an augmentation of motivation to reach the reward. Finally, coherent with an increase in speed and a decrease in distance the latency to make a choice decreased significantly across trials (Fig. 13C; ꭓ² = 90.245, df:1, p < 0.00001).

Test Performance

As in experiment 2, bumblebees were submitted to a test where cylinders were not reinforced.

We recorded the percentage of bees choosing correctly the CS+ or the CS-in their first choice, or not choosing (NC) and the time spent fixating the CS+ and the CS-(Fig. 12B).

Bumblebee successfully learned the discrimination as they chose CS+ significantly more than both CS-and NC (Fig. 12B; CS-: z = 7.585, p < 0.00001; NC: z = 10.962, p < 0.00001). The number of NC was also lower than CS-(z = 5.621, p < 0.00001), suggesting than even bee that made the wrong choice were motivated to interact with the VR.

Consistent with their first choice, bumblebees spent more time fixating CS+ than CS-(Fig. 12C; V = 5781, p < 0.00001). In the corresponding cumulative heat map (Fig. 12E), we can see that bumblebees did interact with both sides and explored both cylinders.

other with water or quinine solution 120 mM, perform better if they experience quinine on the CS-targets rather than water [START_REF] Chittka | Bees trade off foraging speed for accuracy[END_REF]. It is also coherent with previous VR results

where honeybees were able to solve the discrimination when CS-was associated with quinine, but not when it was associated with NaCl (Buatois et al., 2017). In Experiment 3, bumblebees also show a significant decrease in both the distance walked and the latency to make a choice (i.e. approaching and centering a stimulus), confirming that they acquire the task over trials and learned to navigate the VR environment.

While learning performances were similar, bumblebees had nevertheless a higher motivation than honeybees, as 60% of honeybees did not make enough choices to be kept in the analysis against only 11% in bumblebees. Artificially forcing the insect to walk might conflict less with the natural ecology of Bombus terrestris who lives in underground nests than it does for honeybee foragers who spend most of their time in flight. This contrast would have been increased further in our study as the honeybees were foragers collected from a feeder in our apiary while the bumblebees were collected from their commercial nest box that they had never left in their life. Additionally, the analysis of motor parameters revealed that bumblebees walk longer distances than honeybees, which was likely due to the bigger size of bumblebees since walking distance and speed were positively correlated with size in bumblebees. Thus, it is possible that bumblebees had an easier time walking on the treadmill due to their bigger size and ecology (Dahmen et al., 2017).

We found no relation between size and learning performances in our experiment. Several studies have shown that bigger bumblebees have a better visual acuity than smaller bees (Macuda et al., 2001;Spaethe and Chittka, 2003;Wertlen et al., 2008;Taylor et al., 2019).Throughout the experiment the cylinders subtended a visual angle of at least 6° at the beginning of a trial and up to 53° when making a choice, they were thus always above the minimal 5° necessary for color vision in honeybee (Giurfa et al., 1996). It is thus unlikely that higher visual acuity could help in better solving the discrimination. Larger bumblebees have been shown to learn faster in free flight condition (Worden et al., 2005). Larger bumblebees also assumes the role of forager for the colony (Goulson, 2003), a role for which they are better suited than smaller workers (Heinrich and Heinrich, 1983;Goulson et al., 2002;Spaethe and Weidenmüller, 2002;Ings et al., 2005;Klein et al., 2017). So it is possible that the learning abilities of larger bumblebees come from their life experience as foragers (Giurfa et al., 2003;Cabirol et al., 2017), however in this study all bumblebees were naive as they never left their nest box prior to the experiment which would have prevented the bigger workers to improve their learning abilities through higher foraging experience. Finally, this is not the first study to find no link between size and learning success, in 2008 Raine and Chittka trained free flying bumblebees to collect a sucrose rewarded from colored feeders and found no correlation between insect size and learning speed (Raine and Chittka, 2008).

On the other hand, we found a negative effect of body weight on honeybee's learning performances, it is possible that bees with a higher fat storage had a lower motivation to solve the task as foraging is coupled with a reduced rate of fat in this species (Toth et al., 2005;Toth and Robinson, 2005). Consequently, fatter bees might have been less experienced young foragers, thus explaining relative lower performances.

Conclusion

Our results clearly demonstrate the ability of bumblebees to solve a color discrimination task under VR condition. While learning performances were similar to those of honeybees, bumblebees engaged more with the VR and thus less individuals were discarded because of a lack of choices in bumblebees, confirming the suitability of bumblebees as a model for the study of visual learning in VR. We found that bumblebees solved the discrimination better when the CS-was paired with quinine rather than NaCl. No correlation between bumblebee size and learning success was evidenced which might suggest that the better learning success observed in larger bumblebees in the literature were linked to a higher foraging experience as the bumblebees we tested were all naive. These results, associated with their high resilience, point at the bumblebees as a prime candidate to explore the underlying mechanisms of visual learning by coupling VR experiments with invasive electrophysiological or calcium imaging studies.

discussed by all authors. M. G. obtained the funding necessary for this work. All authors reviewed and approved the final version of the manuscript.

General Discussion

5521 5522 2018; Zwaka et al., 2019;Buatois et al., 2020) which would have had a weight of about 7.8g and thus require from the bee 31 time the force required to move itself on a flat surface compared to only 5 time in our current setup (Dahmen et al., 2017). We recently acquired specially made hollow balls that might allow to drive that number further down. Our VR software is based on a video game engine (Unity) which offers two main advantages. First, they have been the focus of intense optimization work in the past decades to reduce as much as possible the latency between player input and reaction from the software, thus in our current implementation we measured a latency of 18.00 ± 2.53 ms between a movement of the ball and a reaction in the VR. Keeping the latency as low as possible is very important to create a believable illusion as honeybee vision has a very high temporal resolution [START_REF] Srinivasan | Temporal acuity of honeybee vision: behavioural studies using moving stimuli[END_REF]. Second, video game engines offer a wide set of tools that make maintaining and expending the possibilities of our VR software easier. We have made our software open source (https://github.com/G-Lafon/BeeVR) which should provide a simple and adaptable solution to support a variety a conditioning protocol. Moreover, this gives us the opportunity to keep growing its possibilities outside of our own projects by allowing other teams to contribute the functionality they need in their work.

In chapter one, we focused on investigating the influence of motion cues, both frontal and ventral, on associative color learning in VR. By enriching the VR with a background producing optic flow we were expecting to improve bees' performances as optic flow has been shown to be crucial for flight navigation in bees (Baird et al., 2006;Frasnelli et al., 2018;Baird et al., 2021), as well as to improve visual learning in harnessed condition (Balamurali et al., 2015). It was thus surprising to find that motion cues from the background of the VR impaired color discrimination of objects located in the virtual foreground. Indeed, only when the background was empty were the bees able to properly learn the discrimination. However, in the condition where optic flow from the background was artificially suppressed, by fixing the grating to the bee's gaze, honeybees spend more time fixating the rewarded object during the non-rewarded test, despite not showing a clear preference in term of first choices, which suggest a learning effect albeit less strong than in the empty background condition. Suppressing motion cues allowed thus to rescue some learning abilities, suggesting that both the motion cues emanating from the background, and its illumination conditions, may have interfered with color learning in the VR arena. To explain this effect, we hypothesized that both the luminosity and motion cues from the background might have distracted the insects by decreasing their attention toward the objects of interest. By contrast ventral motion cues had no effect on color learning performances, in concordance with our previous conclusions as the ventral motion cues emerged from painted dots on the treadmill and provided thus no additional distracting luminosity while they were not competing spatially with the colored cuboids. However, ventral motion cues did affect the walking speed of bees. Honeybees were walking slower when the treadmill motion was generating more optic flow. We know that ventral optic flow is a measure of distance for flying bees, these results thus suggest that bees also use it to evaluate walked distances.

In the introduction we claimed that the existing VR setup were not at the level of efficiency of PER to trigger learning in bees as the associated learning rate is slower. Unfortunately, increasing the complexity of the VR by introducing a richer background did not prove to be a solution to improve learning performances in VR. So, if enriching the VR is not a solution, what options are left to improve learning rates in VR?

A few technical improvements are still possible: replacing the video projector with a panel of LEDs to increase the refresh rate beyond 200 Hz (limit of the bee's eye temporal resolution), and reducing the weight of the ball even further by using hollow Styrofoam ball. We could also introduce an automatized reward system, as was done in Zwaka et al. 2019, to increase consistency in reward delivery and the experimental productivity by running several replications in parallel due to a full automatization of the setup. However, concerning the virtual environment itself, it seems that we have reached the limit of what can be achieved as making the virtual world more complex reduced performances (chapter 1) while moving from 2D to 3D did not improve learning performances (chapter 3), where the same setup with 2D VR, lead to the same 58% of correct choices during the test. Thus, contrary to our expectations, increased complexity does not produce better results. In the context of visual learning, the future development of VR should hence focus on actually making the VR simpler. It might not improve learning performances but we now know that it will not impede them, and it will get us closer to an equivalent of PER for the exploration of visual learning with a simple and easily reproducible setup that can be used by many teams at little cost.

Now that we've established functioning protocols for differential learning in VR (chapter 1 and

2) (Buatois et al., 2017;Lafon et al., 2021) future developments can focus on adapting protocols for more complex learning task like reversal learning, or concept learning. This would open the higher-order cognitive abilities of the bees to deeper investigations.

Investigating the brain regions involved in differential visual learning in VR

In chapter 2, we used the 3D VR environment established in chapter 1 to study visual learning and determine if it leads to changes in immediate early gene (IEG) expression in specific areas of the bee brain. We focused on 3 IEGs related to bee foraging and orientation, kakusei, Hr38

and Erg1. This work represents the first controlled experiments on the variations of IEGs level in the honeybee brain as a result of visual learning, as previous study focused on foraging and orientation flights (Kiya et al., 2007;Lutz and Robinson, 2013;Fujita et al., 2013;Singh et al., 2018;Ugajin et al., 2018;Iino et al., 2020). Our experiment revealed an increase in Egr1 expression in the calyces of the mushroom bodies of learners compared to non-learners. Even though the implication of mushroom bodies in visual learning in the bee is expected given the crucial role of mushroom bodies for the acquisition, storage and retrieval of olfactory memories inputs presented to the insect during the conditioning. We can thus make sure that all bees had the same visual experience and that the difference observed between learners and non-learners is specific of the learning and not caused be a difference in sensory input.

The data from chapter 2 confirmed that the mushroom bodies are involved in visual learning and show that virtual reality can be successfully used to investigate the neural mechanism of visual learning. However, the analysis here was done ex-vivo after the conditioning. In order to go further it would be interesting to be able to perform live recordings from the bees as they learn. We know from previous works that it is possible to record electrophysiological activity from behaving bees in VR (Paulk et al., 2014;Zwaka et al., 2019;Rusch et al., 2021). Coupling our VR setup with electrophysiological recordings thus appear like a logical next step.

In the third chapter we reproduced the previous experiment but using a simpler, more restrictive 2D VR. Surprisingly we found a different pattern of activation by comparison with the 3D VR despite the fact that the conditioning protocol and the stimuli used were similar.

We showed that, in 2D conditions, associative color learning led to a downregulation of the three IEGs considered in different areas of the visual circuit in the learner group compared to the non-learners. While Egr1 was downregulated in the optic lobes, Hr38 and kakusei were coincidently downregulated in the MB calyces. This was doubly unexpected because results in 3D showed an upregulation instead, but also because increased neural activity resulting from experience-dependent phenomena is usually reflected by an upregulation of IEG expression (Bahrami and Drabløs, 2016). The downregulation suggests an inhibition of neural activity in key areas involved in visual processing-optic lobes and mushroom bodies -of the learner group.

Inhibition of the optic lobes, suggested by the downregulation of Egr1 in that region, is coherent with the multiple GABAergic fibers innervating the medulla and lobula (Schäfer and Bicker, 1986) and with the increase in Amgad expression, the gene coding for a key enzyme of the GABA synthesis, in the optic lobes in foragers (Kiya and Kubo, 2010). In the MB neural inhibition is provided by GABAergic feedback neurons (Av3 neurons) (Rybak and Menzel, 1993), which are responsible for the sparse coding responses exhibited by Kenyon cells. These neurons have been shown to be involved in negative patterning (Devaud et al., 2015). Inhibition at the level of the MBs may therefore be part of certain learning phenomena, which require enhanced neural sparseness to de-correlate stimulus representations and thus increase memory specificity. Both kakusei and Hr38 downregulation in the MBs in the 2D VR may be the consequence of plastic changes in GABAergic signaling in the calyces of the MBs.

Both VR experiments were done under similar handling conditions, used strictly the same behavioral criteria and presented the same colors as stimuli. Genetic analyses were also performed under the same conditions and using the same materials and methods. The main difference comes from the way the bees were able to inspect the stimuli: in 3D, the object were cuboids, could be approach from any side and would expand and retract as the bee got closer or further away from them; in 2D, the bee could only move the objects laterally and could not get closer or further away from them. Erg1 upregulation in the learner group in the 3D VR could thus results from an interaction between an exploratory drive of the environment and learning. In 2D VR, GABA-mediated inhibition may act as a gain control mechanism that enhances response efficiency and stimulus control. It has indeed been proposed to play a fundamental role in establishing selectivity for stimulus orientation and direction of motion in mammals (Rose and Blakemore, 1974;Sillito, 1979;Tsumoto et al., 1979). As the latter is particularly important in the 2D VR, enhanced GABA inhibition could be associated with learning to master the visual discrimination in this context. It is also possible that the two objects appear more similar in 2D as the insects have less opportunity to inspect them, it could thus require sparse coding that has been shown to be required to discriminate similar odors in flies (Lin et al., 2014). However, we measured no difference in learning success between 2D and 3D which goes against an increased difficulty of the task in 2D. Another possible explanation is a possible difference in the visual acquisition mechanisms recruited by these two scenarios. On Throughout chapter 2 and 3 we have relied heavily on two terms, learner and non-learner. We have defined the learners as the animals that made the correct choice during the non-rewarded test and the non-learners as the rest, meaning those that either chose wrong or did not chose.

During the first trial, naive bees have a 40% chance to choose the CS+ in 3D, as measured in the first trials of our 3D experiments (chapter 1) and about 50% in 2D (first trial, chapter 3), meaning that in both situations the probability to randomly be a learner is very high. Through the conditioning process the proportion of correct choices in the population increases significantly, meaning that the bees are changing their behavior and are not choosing randomly anymore, they are learning. That's why we still use that classification, because we think it is reasonable to expect that the bees that made a correct choice at the end are the ones that learned the association, since we clearly see that some bees are learning during the conditioning process.

But in truth we still have no certitude over who actually learned something and who still chose randomly. By using the first choice as a criterion we know that the learner group should contain more individual that learned something but we can't identify them and guarantee that they all actually learned something. First choice and other similar binomial variables are more informative about the population than the individuals [START_REF] Gallistel | The learning curve: implications of a quantitative analysis[END_REF][START_REF] Pamir | Average group behavior does not represent individual behavior in classical conditioning of the honeybee[END_REF][START_REF] Pamir | Rapid learning dynamics in individual honeybees during classical conditioning[END_REF]. However, in studies like the one in chapter 2 and 3, we need individual information to be able to confidently identify the animals that learned the task. Therefore, future studies aiming at measuring variations induced by learning in the brain of animal will require more quantitative variables like the latency to display the condition response, the stability of the conditioned response across trials [START_REF] Gallistel | The learning curve: implications of a quantitative analysis[END_REF][START_REF] Pamir | Rapid learning dynamics in individual honeybees during classical conditioning[END_REF] or the time spent choosing the rewarded stimulus. Using quantitative variables will still contain an element of arbitrariness in the establishment of thresholds but it will give us more information about each individual and get us closer to know which one actually learned something.

Bumblebees suitability for VR experiments

Finally, in the last part of this manuscript we explored the possibility of using bumblebees

Bombus terrestris as a model in VR by measuring their performances in a color discrimination task under VR condition and compared with honeybees. The idea for this experiment came after we observed that honeybees that were injected with PBS solution in the central complex, the calyces of the mushroom bodies or the antennal lobes were not able to solve the discrimination task in VR anymore (data not shown), we thus decided to test a more robust species that could potentially make this kind of experiment more likely to succeed.

First we measured the size and weight of every individual to control for a potential effect of weight on the ability to navigate the VR, since bigger animals should have an easier time in moving the ball (Dahmen et al., 2017).

While weight had no effect on distance or speed we found that larger bumblebees walked more than smaller ones. This is likely due a difference in the length of each step as larger bumblebees were also walking faster.

Size and weight did not seem to affect bumblebees' learning performance in our VR experiment. Several studies have shown that bigger bumblebees have a better visual acuity due to having bigger eyes (Macuda et al., 2001;Spaethe and Chittka, 2003;Taylor et al., 2019;Wertlen et al., 2008) and have better foraging success (Spaethe and Weidenmüller, 2002;Ings et al., 2005;Klein et al., 2017). But their increased foraging success can be explained by the fact that larger bees are able to forage in cooler conditions (Heinrich and Heinrich, 1983), may be able to forage over larger distances, and are perhaps also less vulnerable to predation (Goulson et al., 2002). Coherent with these observations, larger bumblebee often assume the role of forager for the colony (Goulson, 2003), it is thus possible that their increased learning performances observed in previous study (Worden et al., 2005) are actually a consequence of their life experience as foragers (Giurfa et al., 2003;Cabirol et al., 2017). While bumblebees used in Worden et al. study were allowed to forage freely at a feeder station, in our study we used commercially reared bumblebees that were completely naive and had no foraging experience prior to our experiment, which would then explain why the larger bumblebees did not perform better. When investigating correlations between learning speed and foraging success across colonies, Raine and Chittka did not evidence any link between individual forager size and learning speed when conditioning bumblebees to collect a sucrose reward from colored feeders in free flying conditions (Raine and Chittka, 2008). The bumblebees used in that study were also taken from commercial colonies and had no prior exposure to colored stimuli associated with food, this is coherent with our hypothesis that body size only affects learning speed indirectly through foraging experience.

Bumblebees were able to learn to discriminate cylinders differing in color and reinforcing outcome in a VR context. After 6 trials, about 60% of bumblebees chose the rewarded stimulus over the punished one. Those results are comparable with honeybees performances both in this experiments and in previous studies (Buatois et al., 2017;Lafon et al., 2021). We also found that using quinine solution as negative reinforcement gave better results as bumblebee conditioned with NaCl were only able to solve the discrimination when the rewarded cylinder was blue, a color they tend to learn faster (Gumbert, 2000). The weaker effect of salt observed here with bumblebees is coherent with previous results found in honeybees where bees were not able to solve the discrimination when the CS-was paired with NaCl solution (Buatois et al., 2017).

Throughout our work we have applied one consistent criterion to include or not insects in the study: if a bee doesn't make a choice for at least half the conditioning trials it is discarded. In this experiment this lead us to discard about 60% of honeybees. While for bumblebees, we only discarded 11% of individuals. The higher motivation of bumblebees could be explained by their ecology, since they tend to nest underground they might have less problem walking in the dark for prolonged amount of time compared to honeybees, it could also be explained by a size difference as larger bumblebees were making longer strides and thus had an easier time moving in the VR. In any case, this makes conditioning a large number of insects easier in bumblebees as the ratio of useable data over conditioned bees is much higher.

In the past decades bumblebees have proved to be a good model to complement honeybee research (Riveros and Gronenberg, 2009) thanks to their good cognitive abilities (Laverty, 1994;Laloi et al., 1999;Leadbeater and Chittka, 2005;Worden et al., 2005;[START_REF] Raine | Pollen foraging: learning a complex motor skill by bumblebees (Bombus terrestris)[END_REF]Leadbeater and Chittka, 2007) coupled with their robustness under restrained conditions and during electrophysiology recordings (Paulk et al., 2008(Paulk et al., , 2009;;Skorupski and Chittka, 2010;Vähäkainu et al., 2013;Rusanen et al., 2017). In this context our results strongly suggest that the bumblebee would be an excellent model for investigating further the neural correlate of visual learning in VR using invasive techniques, as they possess the ability to learn successfully in VR and the robustness to endure the required surgeries.

Conclusion

So, is VR the new revolution for the study of bee behavior?

Not quite. While it does allow to study visual learning in tethered animals, the throughput of tested animals is still lower than with olfactory PER conditioning and the technical entry cost is much higher. However, VR offers way more possibilities than PER as the animals are actually moving and thus are more prone to exhibit a richer behavior. VR is a versatile tool, which allows for manipulation of multiple variables and for sophisticated analyses of behavior.

Overall our work improved the field of visual learning by producing a robust 3D VR system that is inexpensive, open source and supports experiments on both bumblebees and honeybees.

We proved that it can reliably be used to condition bees in color discrimination through several different studies, that also allowed us to refine conditioning protocols in VR. We were able to use our setup to push our understanding of the neural mechanism of visual learning a little further by performing the first quantification of IEGs variations in a controlled visual learning experiment. In order to fully exploit the possibilities that this setup opens we now need to develop a way to couple it with live recording through either calcium imaging or electrophysiology. As a first step in that direction we also showed that bumblebees, known for their resilience, are also a good model for the study of visual learning in VR. Taken together our results open the way for a deeper exploration of visual learning through VR experimentation.

Figure 1 .

 1 Figure 1. Color vision of the honeybees. Experimental results from von Frisch experiment.

Figure 2 .

 2 Figure 2. Y-maze for honeybees. Prior to this conditioning honeybees learn to come to the entrance of the maze by flying from their hive on their own accord to collect a sucrose reward from each arm of the maze. During each conditioning trial only one arm is rewarded in association with a particular stimulus the bee can see from the decision chamber and has to associate with the reward. Adapted from Avarguès-Weber et al., 2011a.

Figure 3 .

 3 Figure 3. Rule learning in honeybees. (a) Honeybees trained in a delayed matching-to-sample task to collect sugar solution in a Y-maze where they first get a sample at the entrance and then a choice within the maze and need to choose the stimulus matching the initial sample. (b) Theyare trained on a series of patterns or colors to learn a rule of sameness. Bees trained on the patterns were tested on the colors and vice-versa. In both cases, bees chose the novel stimuli corresponding to the sample. Taken from Avarguès-Weber et al., 2011a.

Figure 3 .

 3 Figure 3. Automatic speed control and lateral positioning of a miniature hovercraft navigating in a 4 m long straight or tapered corridor (a). The hovercraft is equipped with a bee-inspired autopilot based on the dual optical flow regulators and is endowed with an insectinspired 4-pixel visual system. (b) Chronophotography (~1s time interval) of the hovercraft crossing the corridor. Adapted from Roubieu et al., 2014.

Figure 4 .

 4 Figure 4. Shuttle boxes for aversive visual learning in bees. In these setups, the bees are located in an elongated chamber where they shuttle back and forth. During a conditioning trial, each half of the chamber is identified by a visual cue (colored sheets of paper or color LEDs) and one of these cues is associated with electric shock delivered by a shock grid whenever the bees enters the compartment. The position of the bees is either assessed by a number of infrared barriers (a) or manually (b and c). (a) APIS setup taken from Kirkerud et al., 2017. (b) Taken from[START_REF] Agarwal | Dopamine and Octopamine Influence Avoidance Learning of Honey Bees in a Place Preference Assay[END_REF] Taken fromMarchal et al., 2019.

Figure 5 .

 5 Figure 5. Olfactory conditioning of the proboscis extension reflex. (A) Before conditioning, stimulation with the neutral odor does not elicit the PER. (B) After forward pairing of odor stimulation and sucrose solution the honeybee responds to the odor with PER. Taken from

Figure 6 .

 6 Figure 6. Visual conditioning of the sting extension reflex (SER). (a) Experimental setup, a honeybee was individually harnessed in a holder allowing the delivery of a mild electric shock.

Figure 7 .

 7 Figure 7. Flight simulator build by Götz. (Left) Electric diagram and schematics of the torque meter used to measure the optomor respone of the fly. (right) Textured wall used to induce optomor response in the fly. Taken fromGötz, 1964

Figure 8 .

 8 Figure 8. Illustration of the measurement of the streamlining response. The streamlining response is defined as the orientation of the abdomen relative to the thorax. The sketches illustrate the definition of the response, and the images give three examples of how this is used to measure the response. Taken from Luu et al., 2011. More recently another flying VR was established to study the relative importance of motion

Figure 9 .

 9 Figure 9. Bumblebee free flight VR arena. (a) Diagram of the flight arena. (b) Behaviour results from congruent (top) and incongruent (bottom) VR. Virtual platform is in purple and virtual floor in salmon. The green crosses represent the positions at which individuals initially descended through the plane of the virtual platform. Adapted from Frasnelli et al., 2018.

Figure 10 .

 10 Figure 10. VR setup using a locomotion compensator. A) Global view of the VR system. 1: Semicircular projection screen made of tracing paper. 2: Holding frame to place the tethered bee on the treadmill. 3: Treadmill made of a Styrofoam ball floating on an air cushion. 4: Infrared mouse optic sensors allowing to record the displacement of the ball. 5: Air input. B) The tethering system. 1: Plastic cylinder containing a glass cannula into which a steel needle is inserted. 2: Needle attached to the thorax of the bee. 3: Its curved end is fixed to the thorax by means of melted bee wax. C) Example of stimuli presented to the insect, her two colored cuboids.

Figure 11 .

 11 Figure 11. Virtual reality set-up and visual stimuli used for negative patterning. (a) Global view of the virtual reality system. (b) Conditioned stimuli: green grating (A), blue grating (B)

Figure 12 .

 12 Figure 12. Illustration of a naturalistic VR: The Antarium. (Top) Picture of the fully assembled Antarium. (Bottom) The landscape panorama projected by the Antarium LEDs seen at 1.5 resolution, about twice the average resolution of ants. Taken from Kócsi et al., 2020.

Figure 13 .

 13 Figure 13. The different visual neuronal populations and pathways of the honeybee brain. The black arrow indicates color stimulation. La = lamina, ꭓο = outer chiasm, me = medulla, ꭓI = inner chiasm, lo = lobula, le = lateral calyx of the mushroom bodies, me = median calyx, α = alpha-lobe, β = beta-lobe, al = antennal lobe, ot = anterior optic tuberculum. MB: mushroom bodies; CC: central complex. Courtesy of M. Giurfa.

Figure 14 .

 14 Figure 14. Three-dimensional structure and neural connectivity of the anterior optic tubercle (AOTu). Three-dimensional reconstruction showing the different AOTu compartments (left inbox): major unit dorsal lobe (MU-DL; green), major unit ventral lobe (MU-VL; yellow), ventrolateral unit (VLU; red), and lateral unit (LU; blue). The schematic diagram summarizes neural pathways connecting the AOTu with other brain neuropils. La = lamina, Me = medulla, Lo = lobula, AL = antennal lobe, MBvl = mushroom body vertical lobe, MBca = mushroom body calyx, CB = central body, LAL = lateral accessory lobe, AOT = anterior optic tract, vITT = ventral inter-tubercle tract, mITT = medial inter-tubercle tract, TALT = tubercle-accessory lobe tract. Taken from Avarguès-Weber et al., 2012b.

Figure 15 .

 15 Figure 15. Procaine injection sites. (A) Alexa dye injections are shown in magenta (false color) in the MBC (left), VL (middle) and the CX (right). A DAPI-counterstain and autofluorescence of the brain tissue (false colored in cyan) allowed us to identify brain neuropils. Orientation of all three scans was aligned with rostral (neuraxis) facing upwards. Injections of vehicle (B) and of procaine solution (C) into the MBC. Injections into the VL (D). Injections of vehicle (E) and of procaine solution (F) into the central body (red dots) and injections located at the border of the lower division of the central body with spread into the noduli (red dots with black border). MBC, mushroom body calyces; VL, ventral lobes; HL, horizontal lobes; CBU, upper division of the central body; CBL lower division of the central body; Scale bar = 30 μm.

Figure 1 .

 1 Figure 1. Experimental setup for 3D virtual-reality (VR) studies in honey bees. A) Global view of the VR system. 1: Semicircular projection screen made of tracing paper. 2: Holding frame to place the tethered bee on the treadmill. 3: The treadmill was a Styrofoam ball positioned within a cylindrical support (not visible) and floating on an air cushion. 4: Infrared mouse optic sensors allowing to record the displacement of the ball and to reconstruct the bee's trajectory. 5: Air arrival. B) The tethering system. 1: Plastic cylinder held by the holding frame; the cylinder contained a glass cannula into which a steel needle was inserted. 2: The needle was attached to the thorax of the bee. 3: Its curved end was fixed to the thorax by means of melted bee wax. C, D) Two types of Styrofoam balls used for assessing the importance of the ventral optic flow. C) No ventral optic flow provided. D) Ventral optic flow provided. E) Color discrimination learning in the VR setup. The bee had to learn to discriminate a rewarded from a non-rewarded color cuboid. Cuboids were green and blue. In this case color training and testing was set in the 'Transparent Condition', i.e. no background was provided and the VR display contained only the two colored cuboids on an empty dark background. F) Same as in

Fig

  Fig. S1A): Red 10 (RGB: 26, 0, 0; irradiance: 13 μW/cm²; N = 19), Red 50 (RGB: 128, 0, 0; irradiance: 140 μW/cm²; N = 19) or Red 100 (RGB: 255, 0, 0; irradiance: 1130 μW/cm²; N =

Figure 2 .

 2 Figure 2. Choice and discrimination learning tasks in the VR setup. A) Experiment 1: Quantification of the spontaneous phototactic responses of bees towards a red cuboid against the absence of an equivalent stimulus in the symmetric position (dashed cuboid). Choice of the red cuboid was recorded if the bee reached a virtual area of a radius of 3 cm centered on the cuboid and fixed it frontally. B) Experiments 2 and 3: Color discrimination learning with a green and a blue cuboid. One cuboid was rewarded with sucrose solution and the other punished with either quinine solution (Experiment 2) or saline solution (Experiment 3) delivered by the experimenter. A choice was recorded when the bee reached an area of a radius of 3 cm centered on the cuboid and fixed it frontally. The cuboid image was then frozen during 8 s and the corresponding reinforcement (US) was delivered. C) Experimental schedule of color learning experiments (Experiments 2 and 3). Bees were trained along 10 conditioning trials that lasted a maximum of 1 min and that were spaced by 1 min (intertrial interval). After the end of conditioning, and following an additional interval of 1 min, bees were tested in extinction conditions with the two colored cuboids during 1 min.

Figure 3 .

 3 Figure 3. Experiment 1 -Choosing the red intensity for the achromatic black and red background. Quantification of the spontaneous phototactic responses of bees towards a red cuboid (see Fig. 2A). Three different intensities were assayed, each with a different group of bees: Red 10 (RGB: 26, 0, 0; irradiance: 13 μW/cm²; N = 19), Red 50 (RGB: 128, 0, 0; irradiance: 140 μW/cm²; N = 19) and Red 100 (RGB: 255, 0, 0; irradiance: 1130 μW/cm²; N = 20). For each intensity, the figure represents the pooled performance of four consecutive extinction tests in which the spontaneous attraction towards the red cuboid ('Red') was quantified. 'NS' (no stimulus) represents the choice of an equivalent empty area in the VR arena

Figure

  Figure 4A-D shows the learning curves of the four groups of bees trained to discriminate the green from the blue cuboid under different background conditions and the cumulative heat maps displaying the locations of the bees in their trajectories during the ten acquisition trials.

Figure 4 .

 4 Figure 4. Acquisition performances in a color discrimination learning task under four different background conditions. Each panel shows on the left the acquisition curves in terms of the percentage of bees responding to the CS+ (red), to the CS-(black) or not making any choice (NC; gray) during the ten conditioning trials. The pink, light gray and gray areas around the curves represent the 95% confidence interval of CS+, CSchoices and NC, respectively. On the right of each panel, a heat map shows the cumulative coordinates occupied by the bees trained under each background condition during the ten training trials. Coordinates were binned into 1 cm². Warmer colors depict locations more frequently occupied (see color bar). The highest frequency is cut down to 10 % of the maximum on the color bar. While the side of the rewarded stimulus was randomized along conditioning trials, it was placed arbitrarily on the right in the heat maps. A) In the 'Transparent Condition' (N = 24), the blue and green cuboids were displayed against a dark background. B) In the 'Vertical Grating -No Optic Flow Condition' (N = 17), the cuboids were

Figure 5 .

 5 Figure 5. Motor and temporal components of bee trajectories during the acquisition trials. For each background condition, the evolution of A) the distance walked, B) the walking speed, C) the tortuosity and D) the choice latency during training trials is shown. The tortuosity was the ratio between the total distance walked and the straight line connecting the first and the last point of the trajectory during a training trial. Transparent Condition' (N = 24), 'Vertical Grating -Optic Flow Condition' (N = 17), 'Vertical Grating -No Optic Flow Condition' (N = 17), 'Rotating Vertical Grating Condition' (N = 17). The dashed lines above and below the curves represent the 95% confidence interval.

  : 'Optic Flow Condition'; CS+ vs CS-, z = 1.41, p = 0.16; Fig. 6C: 'No Optic Flow Condition'; CS+ vs CS-, z = 1.0, p = 0.30; Fig. 6D: 'Rotating Grating Condition'; CS+ vs CS-, z = 0.4, p = 0.7). The comparisons with NC bees in these three conditions were all non-significant except in the 'No Optic Flow Condition' where CS+ responding bees and NC bees differed significantly (Fig. 6B: 'Optic Flow Condition'; CS+ vs. NC: z = 1.05, p = 0.30; CS-vs. NC: z = 0.38, p = 0.70; Fig. 6C: 'No Optic Flow Condition'; CS+ vs. NC: z = 2.38, p = 0.02; CS-vs. NC: z = 1.55, p = 0.12; Fig. 6D: 'Rotating Grating Condition'; CS+ vs. NC: z = -1.09, p = 0.28; CS-vs. NC: z = -0.75, p = 0.45).

Figure 6 .

 6 Figure 6. Test performances (1 st choice and fixation time) in a color discrimination learning task under four different background conditions. Panels A-D refer to the 1 st choice and show the percentage of bees responding to the CS+ (red), to the CS-(black) or not making any choice (NC; gray) during a retention test performed in extinction conditions after a 10-trial training. Error bars indicate 95% confidence intervals. *: p < 0.05; **: p < 0.01; ns: nonsignificant. A) In the 'Transparent Condition' (N = 24), the blue and green cuboids were displayed against a dark background. B) In the 'Vertical Grating -Optic Flow Condition' (N =

  Panels E-H refer to the fixation time, i.e. the time spent fixating either the CS+ or the CSduring the test. Bars represent the mean fixation time. Error bars indicate the standard error f the mean. *: p < 0.05; ns: non-significant. E) 'Transparent Condition' (N = 24). F) 'Vertical Grating -Optic Flow Condition' (N = 17). G) 'Vertical Grating -No Optic Flow Condition' (N = 17). H) 'Rotating Vertical Grating Condition' (N = 17). The bottom row shows the heat map corresponding to each condition. Each heat map shows the cumulative coordinates occupied by the bees under each background condition during the test. Coordinates were binned into 1 cm². Warmer colors depict locations more frequently occupied (see color bar). The highest frequency is cut down to 10 % of the maximum on the color bar. The rewarded stimulus was placed arbitrarily on the right. Experiment 3: Influence of ventral optic cues on visual discrimination learning We assessed the importance of the ventral optic flow by training two different groups of bees to discriminate the green from the blue cuboid in the previous 'Transparent Condition' in which color learning was possible. The groups differed in the Styrofoam ball onto which the bees walked.

Figure 7 .

 7 Figure 7. Acquisition performances in a color discrimination learning task under two different ventral optic-flow conditions. A) Color discrimination learning with motion cues available ventrally on the treadmill (N= 38). Left: Acquisition curves in terms of the percentage of bees responding to the CS+ (red), to the CS-(black) or not making any choice (NC; gray) during the ten conditioning trials. The pink, light gray and gray areas around the curves represent the 95% confidence interval of CS+, CS-choices and NC, respectively. Right: Heat map showing the cumulative coordinates occupied by the bees trained under this condition during the ten training trials. Coordinates were binned into 1 cm². Warmer colors depict locations more frequently occupied (see color bar). The highest frequency is cut down to 10 % of the maximum on the color bar. While the side of the rewarded stimulus was randomized along conditioning trials, it was placed arbitrarily on the right in the heat maps. B) Color discrimination learning in the absence of ventral motion cues on the treadmill (N= 29). Left: Acquisition curves as in A). Right: Heat map as in A). C) Data pooled for the two opticflow conditions and segregated according to the situation in which the CS+ color was Blue while the CS-color was Green. Acquisition curves in terms of the percentage of bees responding to the CS+ (blue), to the CS-(green) or not making any choice (NC; gray) during the ten conditioning trials. The blue, green and gray areas around the curves represent the 95% confidence interval of blue+, green-choices and NC, respectively. D) Data pooled for the two optic-flow conditions and segregated according to the situation in which the CS+ color was green while the CS-color was blue. Acquisition curves in terms of the percentage of bees responding to the CS+ (green), to the CS-(blue) or not making any choice (NC; gray) during the ten conditioning trials. The green, blue and gray areas around the curves represent the 95% confidence interval of green+, blue-choices and NC, respectively.

Figure 8 .

 8 Figure 8. Motor and temporal components of bee trajectories during the acquisition trials. For each ventral optic-flow condition, the evolution of A) the distance walked, B) the walking speed, C) the tortuosity and D) the choice latency during training trials is shown. The tortuosity was the ratio between the total distance walked and the straight line connecting the first and the last point of the trajectory during a training trial. 'Ventral Optic Flow' (N = 38), 'No Ventral Optic Flow' (N = 29). The dashed lines above and below the curves represent the 95% confidence interval.

Fig. 9 .

 9 Fig. 9. Test performances (1 st choice and fixation time) in a color discrimination learning task under two different ventral optic-flow conditions (with ventral optic flow and without ventral optic flow). A) 1 st choice performed during the test. As there were neither significant differences between the two ventral optic-flow conditions nor between the color conditions (blue or green rewarded), results were pooled and presented as a single bar diagram (N = 67). The graph shows the percentage of bees responding to the CS+ (red), to the CS-(black) or not making any choice (NC; gray) during the retention test. Error bars indicate 95% confidence intervals. *: p < 0.05; **: p < 0.01; ***: p < 0.001. B) & C) Fixation time during the test in the 'No Ventral Optic Flow Condition' and in the 'Ventral Optic Flow Condition', respectively. In this case, fixation times were separated according to the experimental condition, as different response patterns were observed with and without ventral optic flow. The graphs show the mean time (± S.E.) spent fixating either the CS+ or the CS-during the retention test. B) In the 'No Ventral Optic Flow Condition', bees fixated significantly longer the CS+ than the CS-. **: p < 0.01. C) In the 'Ventral Optic Flow Condition', bees fixated equally the CS+ and the CS-. NS: not significant. D) & E) Heat maps showing the cumulative coordinates occupied by the bees during the test in the 'No Ventral Optic Flow Condition' and in the 'Ventral Optic Flow Condition', respectively. The CS+ is shown on the right by convention. Coordinates were binned into 1 cm². Warmer colors depict locations more frequently occupied (see color bar). The highest frequency is cut down to 10 % of the maximum on the color bar. D) In the 'No Ventral Optic Flow Condition', bees clearly aimed at the CS+ besides choosing it more frequently in their first choice. E) In the 'Ventral Optic Flow Condition', bees also aimed at the CS+ but in a less clear way.

Figure 1 .

 1 Figure 1. Experimental setup and 3D environment. A) Global view of the VR system. 1: Semicircular projection screen made of tracing paper. 2: Holding frame to place the tethered bee on the treadmill. 3: The treadmill was a Styrofoam ball positioned within a cylindrical support (not visible) floating on an air cushion. 4: Infrared mouse optic sensors allowing to record the displacement of the ball and to reconstruct the bee's trajectory. 5: Air arrival. The video projector displating images on the screen from behind can be seen on top of the image. B) The tethering system. 1: Plastic cylinder held by the holding frame; the cylinder contained a glass cannula into which a steel needle was inserted. 2: The needle was attached to the thorax of the bee. 3: Its curved end was fixed to the thorax by means of melted bee wax. C) Color discrimination learning in the VR setup. The bee had to learn to discriminate two vertical cuboids based on their different color and their association with reward and punishment.

Figure 3 .

 3 Figure 3. Discrimination learning in the VR setup. A) Acquisition performance of learners (i.e. percentage of bees that chose the CS+ in the non-reinforced test; n=17). The red, black and grey curves show the percentages of bees choosing the CS+, the CS-or not making a choice (NC), respectively. Bees learned the discrimination between CS+ and CS-. B) Acquisition performance of non-learners (i.e. percentage of bees that chose the CS-or did not make a choice in the non-reinforced test; n=18). These bees did not learn to discriminate the CS+ from the CS-. C) Test performances of learners. Percentage of bees choosing in their first choice the CS+ (FC CS+), the CS-(FC CS-) or not making a choice (NC). Per definition, learners chose the CS+ in this test. Different letters on top of bars indicate significant differences (GLMM; p<0.05). D). Test performances of non-learners. Percentage of bees choosing in their first choice the CS+ (FC CS+), the CS-(FC CS-) or not making a choice (NC). Per definition, non-learners did not choose the CS+. Different letters on top of bars indicate significant differences (GLMM; p<0.05). In all panels, error bars indicate the 95% confidence interval.

Figure 4 .

 4 Figure 4. Motor components of learners (n=17) and non-learners (n = 18) in the VR setup during conditioning. A) Distance travelled (cm) during each conditioning trial. B) Tortuosity of the trajectories (see text for explanation) during each conditioning trial. C) Walking speed (cm/s) during each conditioning trial. The dashed lines above and below the curves represent the 95% confidence interval. Comparisons between curves refer to the significance of the interaction between the factors Trial (1 to 10) and Condition (learners vs. non-learners). All comparisons referring to Condition alone were non-significant. LMM; *: p<0.05; NS: nonsignificant.

Figure 5 B

 5 Figure 5 B-D shows the relative normalized expression of kakusei for the three brain

Figure 5 .

 5 Figure 5. Egr1, but neither kakusei nor Hr38, shows significant variation of relative expression in the mushroom bodies following visual associative learning in a 3D VR environment. A) Honey bee brain with sections used for quantifying IEG expression.

  Real-time PCR were carried out in 384-Well PCR Plates (Bio-Rad) cover with Microseal 'B' PCR Plate Sealing Film (Bio-Rad). The PCR reactions were performed using the SsoAdvancedTM Universal SYBR® Green Supermix (Bio-Rad) in a final volume of 10 μl containing 5 μl of 2X SsoAdvancedTM Universal SYBR® Green Supermix, 2 μl of cDNA template (1:3 dilution from the reverse transcription reaction), 0.5 μl of 10 μmol of each primer and 2 μl of ultrapure water. The reaction conditions were as follows: 95 °C for 30 s followed by 40 cycles of 95 °C for 10 s, 55 °C for 30 s and a final step at 95 °C for 10 s followed by a melt curve from 55 °C to 95 °C with 0.5 °C per second. The reaction was performed in a CFX384 Touch Real-Time PCR Detection System (Bio-Rad) and analyzed with the software Bio-Rad CFX Manager.

Figure 1 . 5 :

 15 Figure 1. Experimental setup, choice criterion and conditioning procedure. A) Global view of the setup. 1: Semicircular projection screen made of tracing paper. 2: Holding frame to place the tethered bee on the treadmill. 3: The treadmill was a Styrofoam ball positioned within a cylindrical support (not visible) floating on an air cushion. 4: Infrared mouse optic sensors allowing to record the displacement of the ball and to reconstruct the bee's trajectory. 5: Air arrival. The video projector displaying images on the screen from behind can be seen on top of the image. B) The tethering system. 1: Plastic cylinder held by the holding frame; the cylinder contained a glass cannula into which a steel needle was inserted. 2: The needle was attached to the thorax of the bee. 3: Its curved end was fixed to the thorax by means of melted bee wax. C) Color discrimination learning in the VR setup. The bee had to learn to discriminate two vertical bars based on their different color and their association with reward and punishment. Bars were green and blue on a dark background. Color intensities were adjusted to avoid phototactic biases independent of learning. Displacement of the bars was

Figure 2 .

 2 Figure 2. Acquisition and test performances of learners and non-learners. A) Acquisition performance of learners (i.e. bees that chose the CS+ in the non-reinforced test; n=23). The red, black and grey curves show the percentages of bees choosing the CS+, the CS-or not making a choice (NC), respectively. Bees learned the discrimination between CS+ and CS-. B) Acquisition performance of non-learners (i.e. bees that chose the CS-or did not make a choice in the non-reinforced test; n=17). These bees did not learn to discriminate the CS+ from the CS-. In A) and B) shaded areas around curves indicate the 95% confidence interval. C) Test performance of learners (% of bees choosing either the CS+, the CS-or not making a choice). Per definition these bees only chose the CS+ first. D) Test performance of non-learners. (% of bees choosing either the CS+, the CS-or not making a choice). Per definition these bees chose either the CS-or did not make a choice (NC). In C) and D), error bars represent the 95% confidence interval. E) Time (s) spent by learners fixating the CS+ and the CS-during the test. Learners spent more time fixating the CS+ consistently with their stimulus choice. Bars represent the time spent keeping the object within -12.5°/+12.5° in front of the bee. Scatter plots represent individual fixation times. ****: p < 0.0001. F) Time (s) spent by non-learners fixating the CS+ and the CS-during the test. Non-learners did not differ in their fixation time of the CS+ and the CS-. Bars represent the time spent keeping the object within -12.5°/+12.5° in front of the bee. Scatter plots represent individual fixation times. NS: non-significant. In E) and F), error bars represent the 95% confidence interval.

Figure 3 .

 3 Figure 3. Differential IEG expression as a consequence of associative color learning in a 2D VR environment. (A) Honey bee brain with sections used for quantifying IEG expression. Yellow labels indicate the brain regions used for the analysis: MB: mushroom body; CB: central brain; OL: optic lobes. The dashed lines indicate the sections performed. Ca: calyx of the mushroom body; li: lip; co: collar; and : and lobes of the mushroom body; CC: central complex; AL: antennal lobe; SEZ: subesophagic zone; OL: optic lobe; Me: medulla; lo: lobula. Relative normalized expression of (B-D) kakusei, (E-G) Hr38 and (H-J) Egr1 in three main regions of the bee brain, the optic lobes (B, E, H), the calyces of the mushroom bodies (C, F, I) and the central brain (D, G, J). The expression of each IEG was normalized to the geometric mean of Actin and Ef1a (reference genes). IEG expression was

  Real-time PCR were carried out in 384-Well PCR Plates (Bio-Rad) cover with Microseal 'B' PCR Plate Sealing Film (Bio-Rad). The PCR reactions were performed using the SsoAdvancedTM Universal SYBR® Green Supermix (Bio-Rad) in a final volume of 10 μl containing 5 μl of 2X SsoAdvancedTM Universal SYBR® Green Supermix, 2 μl of cDNA template (1:3 dilution from the reverse transcription reaction), 0.5 μl of 10 μmol of each primer and 2 μl of ultrapure water. The reaction conditions were as follows: 95 °C for 30 s followed by 40 cycles of 95 °C for 10 s, 55 °C for 30 s and a final step at 95 °C for 10 s followed by a melt curve from 55 °C to 95 °C with 0.5 °C per second. The reaction was performed in a CFX384 Touch Real-Time PCR Detection System (Bio-Rad) and analyzed with the software Bio-Rad CFX Manager.Each sample was run in triplicates. If the triplicates showed too much variability (SD > 0.3), the furthest triplicate was discarded. If the two remaining triplicates still showed too much variability (SD > 0.3) the sample was discarded. The samples were subjected to a relative quantification and normalization. First for each sample and for each reference gene per brain region, the relative quantity (Qr) was computed using the difference between the mean Ct value of each sample and the highest mean Ct value (ΔCt), using the following formula: Qr= (1+E)ΔCt

Figure 1 .

 1 Figure 1. Experimental setup and 3D environment. A) Global view of the VR system. 1: Semicircular projection screen made of tracing paper. 2: Holding frame to place the tethered bee on the treadmill. 3: The treadmill was a Styrofoam ball positioned within a cylindrical support (not visible) floating on an air cushion. 4: Infrared mouse optic sensors allowing to record the displacement of the ball and to reconstruct the bee's trajectory. 5: Air arrival. The video projector displaying images on the screen from behind can be seen on top of the image. B) The tethering system. 1: Plastic cylinder held by the holding frame; the cylinder contained a glass cannula into which a steel needle was inserted. 2: The needle was attached to the thorax of the bee. 3: Its curved end was fixed to the thorax by means of melted bee wax. C) Color discrimination learning in the VR setup. The bee had to learn to discriminate two vertical cylinders based on their different color and their association with reward and punishment. Cylinders were green and blue on a dark background. Color intensities were adjusted to avoid phototactic biases independent of learning.

  3.4f1), open-source code available at https://github.com/G-Lafon/BeeVR. The software updated the position of the bee within the VR every 0.017 s.The VR apparatus consisted of a spherical Styrofoam ball (diameter: 50 mm, weight 1.07 g), which acted as a treadmill. The ball was positioned within a 3D-printed, hollow, cylindrical support (cylinder: 50 mm high, 59 mm diameter). The cylinder allowed distributing an upwards air flow of 33 L.min -1 produced by an AquaOxy 2000 aquarium pump, and released through a small hole at the base of the cylindrical support at a pressure of 1.221 bar. The movements of the ball were recorded by two infrared optic-mouse sensors (Logitech M500, 1000 dpi) placed at a distance of 7 mm from the sphere and forming an angle of 90° angle relative to each other (i.e. 45° from the bee body axis).

Figure 2 .

 2 Figure 2. Choice criterion and conditioning protocol for color discrimination learning. A) Choice criterion.Left: A bee facing the two virtual cylinders. Center: A bee approaching a target cylinder; the cylinder has not yet been centered by the bee (gray area). Right: A bee having centered the target cylinder (gray area). A choice was recorded when the bee reached an area of a radius of 3 cm centered on the cylinder and fixed it frontally. The cylinder image was then frozen during 8 s for honeybees, or 14 s for bumblebees, and the corresponding reinforcement (US) was delivered. B) Conditioning protocol. Bees were trained along 6 conditioning trials that lasted a maximum of 3 min and that were spaced by 1 min (inter-trial interval). After the end of conditioning, and following an additional interval of 1 min, bees were tested in extinction conditions with the two colored cylinders during 3 min.

Figure 3 .Figure 5 A

 35 Figure 3. Experiment 1 -Choosing a balanced pair of green and blue for bumblebee conditioning. Quantification of the spontaneous phototactic responses of bumblebees (N = 51) towards a blue or green cylinder (see Fig. 2A). Two pairs of color were assayed in a random order, one called Green (R: 0, G:100, B:1.0; irradiance 24 370 μW/cm²) versus Bright Blue (R: 0, G:80, B:254) and a second pair called Dark Green (R: 0, G:51, B:1.0) versus Blue (R: 0, G:0, B:255; irradiance 161 000 μW/cm²). For each pair, the figure represents percentage of choice of each color. For subsequent experiments, the Dark Green vs Blue pair was used to condition bumblebees.

Figure 5 .

 5 Figure 5. Acquisition performances in a color discrimination learning task between bumblebees and honeybees. (Top) Acquisition curves in terms of the percentage of bees responding to the CS+ (red), to the CS-(black) or not making any choice (NC; gray) during the six conditioning trials. The pink, light gray and gray areas around the curves represent the 95% confidence interval of CS+, CS-choices and NC, respectively. (Bottom) Heat maps of the cumulative coordinates occupied by the bees during the six training trials. Warmer colors depict locations more frequently occupied (see color bar). A) Learning curve of bumblebees (N = 123). B) Learning curve of honeybees (N = 31). C) Cumulative heat maps of bumblebees' trajectories (N = 123). D) Cumulative heat maps of honeybees' trajectories (N = 31).

Figure 6 .

 6 Figure 6. Motor and temporal components of bee trajectories during the acquisition trials. For both bumblebees (N = 123) and Honeybees (N = 31), the evolution of A) the distance walked, B) the walking speed, and C) the choice latency during training trials is shown. The dashed lines above and below the curves represent the 95% confidence interval.

Figure 7 .

 7 Figure 7. Test performances in a color discrimination learning task for bumblebees and honeybees. Percentages of bees choosing each stimulus (CS+: rewarded, CS-: punished, NC: no choice) during the non-reinforced test. Results are represented according to the color of the rewarded stimulus. Left Shows 1 st choices from bumblebees (N = 123) when A) blue was rewarded (N = 61) and when C) green was rewarded (N = 62). Right Shows 1 st choices from honeybees (N = 31) when B) blue was rewarded (N = 16) and when D) green was rewarded (N = 15).

Figure 8 .

 8 Figure 8. Fixation time during the non-reinforced test. Time spent fixating either the CS+ or the CSduring the test. Bars represent the mean fixation time. Error bar represent the 95% confidence interval. Results are represented according to the color of the rewarded stimulus. Left Shows time from bumblebees (N = 123) when A) blue was rewarded (N = 61) and when C) green was rewarded (N = 62). Right Shows times from honeybees (N = 31) when B) blue was rewarded (N = 16) and when D) green was rewarded (N = 15).

Figure 9 .

 9 Figure 9. Impact of insect's size on the motor and temporal performances of the bees during the test. After the last trial we measured the inter wing distance for each bee, reported here as Size. This figure show the various parameter measured during the test, distance walked, speed and latency to choose as a function of insect's size. Each point is a bee and the pink line shows the linear regression. A C E) Show bumblebees value (N = 123). B D F) Show honeybee values (N = 31). A B) Distance walked. C D) Walking speed. E F) Latency to make a choice.

Figure 10 .

 10 Figure 10. Impact of insect's weight on the motor and temporal performances of the bees during the test. After the last trial we measured the dry weight of each bee, reported here as Weight. This figure show the various parameter measured during the test, distance walked, speed and latency to choose as a function of insect's size. Each point is a bee and the pink line shows the linear regression. A C E) Show bumblebees value (N = 123). B D F) Show honeybee values (N = 31). A B) Distance walked. C D) Walking speed. E F) Latency to make a choice.

Figure 11 .

 11 Figure 11. Impact of insect's dimensions on the learning performances. Shows learning score as a function of the insect's size (A B) or weight (C D). For each bee we computed the learning score as the difference between the number CS+ and CS-choices from the second trial to the test. Each point represents a bee. The pink line shows the linear regression. A C) Bumblebees (N = 123). B D) Honeybees (N = 31).

Figure 12 .

 12 Figure 12. Test performances (1 st choice and fixation time) of bumblebees in a color discrimination learning task using quinine as negative reinforcement and sucrose as positive. As there were no significant differences the color conditions (blue or green rewarded), results were pooled (N = 235). The graph shows the percentage of bees responding to the CS+ (red), to the CS-(black) or not making any choice (NC; gray) during the training phase (A) and the retention test (B). C) Shows the time spent fixating each stimulus during the test. Error bars indicate 95% confidence intervals. *: p < 0.05; **: p < 0.01; ***: p < 0.001; ****: p < 0.0001. D) Cumulative heat maps of bumblebees' trajectories during the 6 learning trials (N = 235). E) Cumulative heat maps bumblebees' trajectories during the test (N = 235). Warmer color higher density of visits (see color scale).

Figure 13 .

 13 Figure 13. Motor and temporal components of bumblebee trajectories during the acquisition trials. For both bumblebees conditioned in Experiment 3 (N = 235), shows the evolution of A) the distance walked, B) the walking speed, and C) the choice latency during training trials is shown. The dashed lines above and below the curves represent the 95% confidence interval.

  one hand, the 3D experiment might include a navigation component where body movements translate into a displacement and a recognizable change in the visual scene, which can then be computed against the available internal information about the displacement. While, on the other hand, the 2D experiment is closer to a purely operant task where the animal needs to engage in two different, but stereotyped motor patterns, turn left or turn right, according to the position of CS+ on the screen. The observed differences in IEG expression between the two types of VR would then reflect the two different mechanisms used to reach the rewarded stimulus. One involving navigation while the other is purely operant.Taken together chapter 2 and 3 suggest the existence of a distributed memory trace along the visual system and highlight the importance of MBs for color learning in bees. They also point at the OLs, and the calyces of the MBs as region of interest for further investigation of the neural correlates of visual learning.

Figure 1 .

 1 Figure 1. The different visual neuronal populations and pathways of the honeybee brain. The black arrow indicates color stimulation. La = lamina, ꭓο = outer chiasm, me = medulla, ꭓI = inner chiasm, lo = lobula, le = lateral calyx of the mushroom bodies, me = median calyx, α = alpha-lobe, β = beta-lobe, al = antennal lobe, ot = anterior optic tuberculum. MB: mushroom bodies; CC: central complex. Courtesy of M. Giurfa.

Table 1 :

 1 Summary of the experimental conditions provided in Experiments 1 to 3. N: sample size of each condition.

Table 1 )

 1 . The Cq values of these reference genes for the different conditions of this experiment

				Amplicon	E	R 2
	Type of gene	Target Primer sequence 5' 3'	length	(%)
				(bp)
	Target genes	Kakusei CTACAACGTCCTCTTCGATT (forward)	149	96.4	0.991
			CCTACCTTGGTATTGCAGTT (reverse)
		Hr38 TGAGATCACCTGGTTGAAAG (forward)	118	106	0.995
			CGTAGCAGGATCAATTTCCA (reverse)
		Egr1	GAGAAACCGTTCTGCTGTGA (forward)	138	109	0.991
			GCTCTGAGGGTGATTTCTCG (reverse)
	Reference genes	Ef1	AAGAGCATCAAGAGCGGAGA (forward)
			CACTC TTAATGACGCCCACA (reverse)	148	106	0.993
		Actin	TGCCAACACTGTCCTTTCTG (forward)
			AGAATTGACCCACCAATCCA (reverse)	156	110	0.995

Table 1 .

 1 Primer sequences used to quantify RNA expression of genes of interest and reference genes by RT-qPCR. Amplicon length (bp), efficiency (E, %) and the coefficient of correlation obtained for the standard curve (R 2 ) are also shown. Hr38: Hormone receptor 38 gene; Egr1:

	Early growth response gene-1; Ef1α: Elongation factor 1 α gene.

Acknowledgements

This PhD thesis was made possible by the continuous support of many people. I would like first to thank Aurore Avarguès-Weber and Martin Giurfa for supervising my PhD and for their thorough revision of the various articles produced in this project.

I also thank sincerely Dr. Ludovic Dickel, Dr. Elisa Frasnelli and Dr. Stephane Viollet for accepting to be part of my PhD jury and evaluate my work.

Acknowledgements

We thank two anonymous reviewers for useful comments and suggestions on previous versions of this article. This work was supported by an ERC Advanced Grant ('Cognibrains') to M.G, who also thanks the Institut Universitaire de France (IUF), the CNRS and the University Paul Sabatier/Fondation Catalyses for support. S.R.H. was supported by a post-doctoral fellowship of the Fyssen Foundation.

ACKNOWLEDGMENTS

We thank three anonymous reviewers for constructive criticisms and Shiori Iino and Takeo Kubo for providing useful information on the timing of IEG expression. We also thank Benjamin H. Paffhausen, Marco Paoli and Dorian Champelovier for valuable discussions. This work was supported by an ERC Advanced Grant ('Cognibrains') to M.G, who also thanks the Institut Universitaire de France (IUF), the CNRS and the University Paul Sabatier for support.

Acknowledgements

We thank the valuable feedback of Profs. Takeo Kubo (University of Tokyo) and Hiroyuki Okuno (Kagoshima University, Japan) on our IEG analyses. We also thank Shiori Iino, Hiroki Kohno, Benjamin H. Paffhausen, Marco Paoli and Dorian Champelovier for valuable discussions and support, and two anonymous reviewers for useful comments on a previous version. This work was supported by an ERC Advanced Grant ('Cognibrains') to M.G, who also thanks the Institut Universitaire de France (IUF), the CNRS and the University Paul Sabatier for support.

Acknowledgements

We thank Mathieu Lihoreau for his help in acquiring and maintaining the bumblebee colonies.

We also thank Benjamin H. Paffhausen, and Marco Paoli for valuable discussions and support. This work was supported by an ERC Advanced Grant ('Cognibrains') to M.G, who also thanks the Institut Universitaire de France (IUF), the CNRS and the University Paul Sabatier for support.

DATA AND CODE AVAILABILITY

The datasets generated during this study are available at figshare.com with the following accession ID: https://figshare.com/s/1e868800d08a17dc300e

List of abbreviations:

Ethics Declarations

Competing interests

The authors declare no competing interests.

Chapter 2

Visual learning in a virtual reality environment upregulates immediate early gene expression in the mushroom bodies of honey bees Subject Areas: Behavior, Neuroscience

ABSTRACT

Free-flying bees learn efficiently to solve numerous visual tasks. Yet, the neural underpinnings of this capacity remain unexplored. We used a 3D virtual reality (VR) environment to study visual learning and determine if it leads to changes in immediate early gene (IEG) expression in specific areas of the bee brain. We focused on kakusei, Hr38 and Egr1, three IEGs that have been related to bee foraging and orientation, and compared their relative expression in the calyces of the mushroom bodies, the optic lobes and the rest of the brain after color discrimination learning. Bees learned to discriminate virtual stimuli displaying different colors and retained the information learned. Successful learners exhibited Egr1 upregulation only in the calyces of the mushroom bodies, thus uncovering a privileged involvement of these brain regions in associative color learning and the usefulness of Egr1 as a marker of neural activity induced by this phenomenon.

Keywords: Vision -Visual Learning -Virtual Reality -Honey Bee Brain -Immediate Early

Genes -Kakusei -Hr38 -Egr1 -Mushroom Bodies

Gene expression data

Statistical differences in gene expression were assessed for reference genes to check for stability and for target genes within a given brain region using One-Factor ANOVA for independent groups in the case of multiple comparisons or two-sample t test in the case of dual comparisons.

Pots hoc comparisons between groups were performed by means of a Tukey test following ANOVA. No cross-comparisons between brain regions or genes were performed due to withinarea normalization procedures. Statistical analyses were done either with R 3.5.1 software 92 or with Statistica 13 Software (TIBCO® Data Science).

REPORTING SUMMARY

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

CONTRIBUTIONS

ETHICS DECLARATIONS

Competing interests

The authors declare no competing interests.

Chapter 3

The neural signature of visual learning under restrictive virtual-reality conditions

The neural signature of visual learning under restrictive virtualreality conditions Gregory Lafon 1 *, Haiyang Geng* 1,2 , Aurore Avarguès-Weber 1 , Alexis

Abstract

Honey bees are reputed for their remarkable visual learning and navigation capabilities. These capacities can be studied in virtual reality (VR) environments, which allow studying performances of tethered animals in stationary flight or walk under full control of the sensory environment. Here we used a 2D VR setup in which a tethered bee walking stationary under restrictive closed-loop conditions learned to discriminate vertical rectangles differing in color and reinforcing outcome. Closed-loop conditions restricted stimulus control to lateral displacements. Consistently with prior VR analyses, bees learned to discriminate the trained stimuli. Ex vivo analyses on the brains of learners and non-learners showed that successful learning led to a downregulation of three immediate early genes in the main regions of the visual circuit, the optic lobes (OLs) and the calyces of the mushroom bodies (MBs). While Egr1 was downregulated in the OLs, Hr38 and kakusei were coincidently downregulated in the calyces of the MBs. Our work thus reveals that color discrimination learning induced a neural signature distributed along the sequential pathway of color processing that is consistent with an inhibitory trace. This trace may relate to the motor patterns required to solve the discrimination task, which are different from those underlying pathfinding in 3D VR scenarios allowing for navigation and exploratory learning and which lead to IEG upregulation.

Keywords

Vision -Visual Learning -Virtual Reality -Honey Bee Brain -Immediate Early Genes -Mushroom Bodies -Optic Lobes

Contributions

The project was conceived by AB, AAW, IM and MG. GL performed all the behavioral experiments. HG dissected and sectioned the brains of the bees trained in the VR setup and performed all the molecular analyses. Behavioral experiments were supervised by AAW and MG. Molecular experiments were supervised by IM and MG. Statistical analyses on behavioral data were performed by GL. Statistical analyses on gene-expression data were performed by HG and MG. The manuscript was written by MG and was corrected and discussed by all authors. MG obtained the funding necessary for this work. All authors reviewed and approved the final version of the manuscript.

Ethics declarations

Competing interests

The authors declare no competing interests.

Supplementary Materials

Supplementary Figure 1

Chapter 4 Comparison of associative visual learning in a 3D virtual reality between bumblebees and honeybees

Comparison of associative visual learning in a 3D virtual reality between bumblebees and honey bees.

Gregory Lafon 1 , Aurore Avarguès-Weber 1 , Martin Giurfa In chapter 2 and 3 we found that color discrimination learning induced a neural signature distributed along the sequential pathway of color processing by performing ex vivo analysis of the brains of learner and non-learner bees. To push our investigations further, we believe it would be interesting to use our VR setup for in vivo analysis. Preliminary experiments revealed that honey bees engage less with the VR after brain surgery. In order to overcome this limitation, we decided to investigate the possibility of using bumble bee in VR experiment.

Bumble bees are closely related to honey bees and known for being more resilient to surgery, thus providing a good alternative model for our experiments. By conditioning bumble bees in VR, we found that bumble bees are able to solve the color discrimination task with a success rate comparable to honey bee. We also found that they engage more and make more choices than honey bee, which leads to higher amount of useable data.

Tethering procedure

The tethering procedure was the same for both honeybees and bumblebees, referred to as bees in the following paragraphs. The procedure followed the same protocol as described in chapter 1 (Lafon et al., 2021).

Each bee was tethered by means of a 0.06 g steel needle, 0.5 mm in diameter and 40 mm in length, which was fixed to the thorax by melted beeswax. The needle was placed within a 3D printed resin tube (Black tough resin Prusa, Prusa Research a. s., Czech republic), 7 mm inner diameter, 1 cm outer diameter and 55 mm in length, which was fixed on a holding frame placed above the treadmill (Fig. 1 A-B). This system allowed the bee to adjust its position in the vertical axis once set on the ball, but did not allow rotational movements. The holding frame consisted of a vertical black, plastic half frame made of two vertical rectangular supports, 105 mm in length, connected to an upper, horizontal rectangular support, 120 mm in length. The latter held the black cylinder in the middle (Fig. 1B). After being attached to its tether, each bee was placed on a small (50 mm diameter) Styrofoam ball for familiarization to a provisory set-up and provided with 5 μl of 1.5 M sucrose solution. Each bee was held for 3 h in this provisory setup, which was kept in the dark and without visual stimulations.

Experiment 2: Comparing performances of bumblebees and honeybees in VR.

Having chosen a pair of colors that elicited the same amount of spontaneous attraction in bumblebees (see above), we trained bumblebees and honeybees, to discriminate between two vertical colored cylinders, one rewarded and the other not (see Training and testing procedure below). Both cylinders had the same dimensions of the cylinder employed in the previous experiment. For honeybees, one was blue (RGB: 0, 0, 255, with a dominant wavelength of 446 nm) and the other green (RGB: 0, 51, 0, with a dominant wavelength at 528 nm). Their intensity, measured at the level of the bee eye, was 161 000 μW/cm² (blue cylinder) and 24 370 μW/cm² (green cylinder). These values were shown to elicit the same level of spontaneous attraction (Buatois et al., 2017;Lafon et al., 2021). For bumblebees we used the Dark Green (R: 0, G:51, B:1.0) and Blue pair (RGB: 0, 0, 255, with a dominant wavelength of 446 nm) (see results from experiment 1).

The cylinders were positioned respectively at -50° and +50° from the bee's body axis at the beginning of each trial. As in the previous experiment, approaching a cylinder within an area of 3 cm surrounding its virtual surface followed by direct fixation of its center was recorded as a choice (Fig. 2A).

Experiment 3: Conditioning bumblebees with Quinine as punishment

In this experiment we repeated the protocol from the experiment 2 on bumblebees using 1.2g.L - 1 quinine solution (Dyer and Chittka, 2004) instead of 3 M NaCl as punishment. All other parameters were the same.

Training and testing procedure

Both honeybees and bumblebees, were trained during 6 trials using a differential conditioning procedure (Fig. 2B) in which one of the cylinders (i.e. one of the two colors, green or blue) was rewarded with 1.5 M sucrose solution (the appetitive conditioned stimulus or CS+) while the NaCl solution as a punishment associated with the CS-. Conditioning lasted for 6 trials followed by one non-reinforced test.

Comparing attendance between honeybees and bumblebees

During conditioning we discarded animals that failed to make a choice in 3 trials. Following that criterion, the proportion of discarded honeybee (59.74%) was significantly higher than the proportion of discarded bumblebees (10,87%) (Fig. 4) (z = 6.963, p < 0.0001). Overall, bumblebees made more choices and were discarded less than honeybees.

Motor and temporal components of bee trajectories during training

We analyzed if there was any difference in the displacement of bees during the training trials between bumblebees and honeybees (Fig. 6). To do so, we quantified the distance walked (Fig. 6A), and the walking speed (Fig. 6B) of the bees during each trial. We also measured the choice latency in each trial (Fig. 6C), i.e. the time between the beginning of a trial and the first choice of the animal.

We found no significant interaction between the trial number and the species of the bees (Trial*Species: χ² 0.7290, df:1, p = 0.40) suggesting no difference in the evolution of distance walked across trial between bumblebees and honeybees. The distance was, however, significantly affected by the species of the bee (Species: χ² = 10.91, df:1, p < 0.001) as bumblebees walked more than honeybees (Fig. 6A). But did not vary significantly over trials (Trial: χ² = 3.2609, df:1, p = 0.071).

The speed was significantly affected by the interaction between species and trials (Trial*Species: χ² 6.2991, df:1, p = 0.012) and by trials alone (Trial: χ² = 32.5642, df:1, p < 0.0001). Meaning that both honeybees and bumblebees increased their speed over trials but the increase was steeper for honeybees (Fig. 6B). However, there was no speed difference between species (Species: χ² = 2.9235, df:1, p = 0.087).

The latency to choose decreased significantly across trials (Fig. 6C) (Trial: χ² = 22.2383, df:1, p < 0.0001) at a similar rate in both species (Trial*Species: χ² 0.1819, df:1, p = 0.6697) and was not significantly different overall between bumblebees and honeybees (Species: χ² = 0.0009, df:1, p = 0.98).

When blue was the rewarded color bumblebees (Fig. 7A) chose the CS+ significantly more than both CS-(z = 4.714, p < 0.00001) and NC (z = 5.882, p < 0.00001). The proportion of CSchoices was also superior to the proportion of NC (z = -2.575, p = 0.01). Bumblebees were thus able to solve the discrimination task during the test when the rewarded stimulus was blue (Fig. 7A). However, they did not manage to choose correctly during the test when CS+ was green (Fig. 7C), as the proportion of CS+ was not significantly different from the proportion of CS-(z = 0.719, p = 0.472). The proportion of NC was significantly less than both CS+ (z = 4.643, p < 0.00001) and CS-(z = 4.185, p < 0.0001). Similarly, bumblebees did fixate the CS+ significantly longer than the CS-when blue was rewarded (Fig. 8A; V = 471, p = 0.001) but not when green was rewarded (Fig. 8C; V = 819, p = 0.271).

Honeybees test performances

Contrary to the training phase, and similarly to bumblebees, color of the rewarded stimulus had an effect on honeybees' choice during the test (Color: χ² = 15.143, df:2, p < 0.001). We thus represented and analyzed green rewarded and blue rewarded separately (Fig. 7B D).

When the rewarded cylinder was blue (Fig. 7B) bees chose CS+ significantly more than CS-(z = 3.238, p = 0.001). On the other hand, we found no significant difference neither between the proportion of NC and CS-(z = -0.007, p = 0.99) nor between NC and CS+ (z = 0.008, p = 0.99).

When the reward was green (Fig. 7D) we found no significant difference between any of the

Discussion

We have shown that bumblebees are able to solve a color discrimination task in a 3D VR setup.

After six conditioning trials about 60% of bumblebees were able to correctly chose the rewarded stimulus during the non-reinforced test, which is similar to the performances displayed by honeybees, both in this study and in previous ones using a comparable protocol (Buatois et al., 2017;Lafon et al., 2021). This success rate is however lower than what has been observed in visual conditioning of the PER [START_REF] Riveros | Decision-making and associative color learning in harnessed bumblebees (Bombus impatiens)[END_REF] where more than 70% of bumblebees responded to the rewarded stimulus after seven trials. Walking speed and latency to make a choice were also similar between bumblebees and honeybees, confirming the suitability of the setup for both species.

When we used NaCl solution as punishment, bees from both species were only able to learn the discrimination if the rewarded stimulus was blue. Previous experiment in VR with honeybees have shown that NaCl was not always sufficient to produce a conditioned response (Buatois et al., 2017). Bumblebees, and honeybees, have an innate preference for short wavelength colors like blue [START_REF] Giurfa | Colour preferences of flower-naive honeybees[END_REF][START_REF] Riveros | Decision-making and associative color learning in harnessed bumblebees (Bombus impatiens)[END_REF] but we did control in Experiment 1 that the blue and green used elicited the same level of spontaneous attraction in bumblebees (Fig. 3), and the color used for honeybees have been controlled in previous studies (Buatois et al., 2017). This result is confirmed by the fact that naïve bees in Experiment 2 did chose as much green as blue (Fig. 4A). However, bumblebees have also been shown to learn short wavelength colors faster (Gumbert, 2000). Thus, we can speculate that the strength of the NaCl reinforcement was not enough to overcome the difference in learning speed between green and blue. When conditioned with Quinine, however, bumblebees displayed no color bias and were able to solve the discrimination task whether the reward was associated with blue or green, in accordance with previous findings where free-flying bumblebees trained to discriminate between two perceptually similar colors, one associated with 1.75 M sucrose solution, and the

Contributions

The

Ethics declarations

Competing interests

The authors declare no competing interests.

General discussion

With this PhD work, we aimed at improving an existing VR set-up in order to overcome the impossibility of accessing the nervous system of flying bees solving visual problems. In the pursuit of this goal we established a new 3D VR and showed that increasing the complexity of the virtual background is detrimental to successful conditioning, as the presence of frontal background motion cues impaired the bees' performance in color discrimination learning. We then used that VR to condition honeybees in a controlled visual environment which was a prerequisite to quantify variation in IEGs expression specific to visual learning. This revealed an implication of the optic lobes and the calyces of the mushroom bodies in visual learning, and that solving a color discrimination task in 2D and 3D involved different neural mechanisms as both modes lead to different pattern of IEG expression. Finally, we showed that bumblebees would be an ideal alternative model for future VR experiments, since they are easier to condition in high numbers than honeybees, while being probably more robust to invasive brain recording techniques.

Virtual reality: progress, limitations and future developments

The first chapter of this thesis represents the first publication of a fully 3D virtual setup showing visual learning in harnessed honeybees. Zwaka et al did publish a setup where the colored stimuli, a blue and yellow colored stripped, could grow larger as the insect walked toward them in 2019. But the behavioral results don't show clearly whether or not the bees managed to learn the color discrimination. Their setup also included gray stripes of different heights in the background, simulating a far-distant skyline, which might have interfered with color learning as our work showed. Our setup has several advantages over previous attempts. First we paid more attention to the weight of the treadmill, using a smaller and lighter treadmill of 5cm for 1g which were thus easier to mobilize for the bees and resulted in a more natural behavior. This is an improvement on previous setups that used a 10 cm Styrofoam ball (Buatois et al., 2017, (Menzel, 1999, 2014;Devaud et al., 2015) and the parallels between visual and olfactory inputs at the level of the calyces, studies addressing the role of mushroom bodies in honey bee visual learning and memory remain rare. This finding is coherent with previous results that found an increase in the dopaminergic receptor gene Amdop1 in the calyces of the mushroom bodies as a result of aversive visual learning (Marchal et al., 2019). The fact that we found no visual learning induced variations of kakusei despite reports of enhanced expression in foragers or in orienting bees (Kiya et al., 2007;Kiya and Kubo, 2010;Ugajin et al., 2018) suggests that the expression is not necessarily related to learning occurring in these contexts.

We did not find any variation of expression in the central brain despite previous studies suggesting a role of the CX in visual learning (Plath et al., 2017). This could be due to the limited spatial resolution of our brain dissection. Indeed, our current technique did not allow us to isolate the CX from adjacent structures, mainly the subesophageal zone and the peduncle of the mushroom bodies. It is possible that variations in the CX got diluted by unspecific responses from the adjacent structures that got included in the dissection. Future experiments would benefit from more precise dissections using, for example, Laser-Capture Microdissection that would allow dissection at cellular resolution. Higher spatial resolution could also allow to investigate other sub regions of the brain, as previous studies suggested a role of the medial lobes of the mushroom bodies in visual learning (Plath et al., 2017) for example.

IEGs are transcribed transiently and rapidly in response to specific stimulations inducing neural activity without de novo protein synthesis (Bahrami and Drabløs, 2016), since their expression is part of the early stages of the cell response to an external stimulus, the quantification of IEGs expression level is a good proxy for cell activity. On the other hand, since they are at such an early stage of the cell response, they integrate signals from many different inputs. This means that in order to control the specificity of the measured response, the inputs must be tightly controlled. This is where VR was important, since it allows to completely control the visual Curriculum Vitae