domain: sdv shs

text:
Technological advances for gaining insight into the learning capacities of birds in

the vocal and auditory domains

Introduction

Birds have fascinated and inspired humans for centuries. Of particular interest have been oscine songbirds (belonging to the suborder Passeri of the perching birds, Passeriformes) that have historically been raised by humans, primarily for aesthetics reasons linked to their plumage and their natural song. But humans have also tried to educate them to produce human words and musical melodies. For example, wind instruments such as "flageolets" (woodwind instruments and members of the fipple flute family) and "serinettes" ('serin' is French for canary -a music box similar to a little barrel organ) were used to play tunes to birds to stimulate or train them to sing. These practices were very popular in Europe in the aristocratic milieu in the 18 th and the 19 th centuries. They took place in the context of conquering, taming and domesticating nature, and to some extent to "civilize" the animals but also to fuel the debate regarding the role of education on human nature (Clouzot & Kreutzer, 2019). Breeders of the past, especially from the 15th to the 18th centuries, discovered that innate predispositions to learn can be manipulated through an acute control of social factors during particular moments of the bird's life (Canary Serinus canaria: Hervieux de Chanteloup, 1709; Nightingale Luscinia megarhynchos: de Nobleville, 1751, see also the contribution of Baron Von Perneau: [START_REF] Stresemann | Baron von Pernau, pioneer student of bird behavior[END_REF]; see [START_REF] Birkhead | Bird-keeping and the development of ornithological science[END_REF] for a review). From these reports, we learned that: (1) birds need to be isolated from conspecifics during the vocal practice of these musical melodies but not necessarily during exposure and

(2) there is no need to overexpose the birds to the models to imitate: few lessons a day are good enough to trigger good imitations. These results were later confirmed using experiments in the controlled conditions of the laboratory (Thorpe, 1958;[START_REF] Tchernichovski | Vocal imitation in zebra finches is inversely related to model abundance[END_REF].

Particularly during the 18 th century, several species were captured in the wild and bred in captivity. Through several different and successive selection processes, wild caged birds of some species then became domesticated 'cage birds'. Breeders developed techniques to raise and to teach these birds some sounds far removed from their usual vocalizations, taking advantage of vocal production learning. Such learning is also an important component of language, particularly with respect to the ability to imitate environmental sounds especially conspecific ones, an ability exhibited by few taxonomic groups of birds (oscine songbirds, hummingbirds and parrots) and of mammals (whales, bats, seals, elephants and humans) but not in monkeys and apes (Petkov & Jarvis, 2012). Nevertheless, the traditional dichotomy between vocal learners and non-vocal learners seems obsolete since several cases of social influences on some aspects of vocal development have been reported in some species of birds or primates traditionally classified as non-vocal learners (see [START_REF] Watson | Vocal learning in the functionally referential food grunts of chimpanzees[END_REF][START_REF] Desmedt | Maternal presence influences vocal development in the Japanese quail (Coturnix c. japonica)[END_REF]Kroodsma et al., 2013 for example). A continuum with respect to graded vocal control seems a more appropriate description nowadays (Petkov & Jarvis, 2012;[START_REF] Tyack | A taxonomy for vocal learning[END_REF][START_REF] Cate | Re-evaluating vocal production learning in non-oscine birds[END_REF].

Since the invention of the sound spectrograph, birdsong became a useful model system to investigate how experience and genetically determined circuitry interact to shape a learned behavior [START_REF] Mets | Learning is enhanced by tailoring instruction to individual genetic differences[END_REF] and thousands of birds from tens of species have been raised in controlled conditions of the laboratory to determine how their social environment, and especially their auditory experience, shape their vocalizations. Birdsong acquisition exhibits many similarities with speech and language development [START_REF] Bolhuis | Birdsong, speech, and language: exploring the evolution of mind and brain[END_REF][START_REF] Marler | Birdsong and speech development: Could there be parallels? There may be basic rules governing vocal learning to which many species conform, including man[END_REF][START_REF] Doupe | Birdsong and human speech: common themes and mechanisms[END_REF][START_REF] Hyland Bruno | Birdsong learning and culture: analogies with human spoken language[END_REF][START_REF] Lipkind | The development of structured vocalizations in songbirds and humans: a comparative analysis[END_REF].

Birdsong learning is based on an auditory-vocal integration: song models heard during a sensory phase are compared to the bird's own vocalizations during the sensorimotor phase when intense practice occurs (Konishi, 1965). As in human infants, vocal developmental learning goes through different stages in juvenile oscine songbirds: following the first weeks of avian babbling, plastic versions of the future song notes and syllables (a terminology inherited from studies on language and music and attested in Medieval and Modern encyclopedias: Fritz, 2016) appear before a process of stabilization called "crystallization" (Thorpe, 1958). From these studies, it was discovered that some species can learn to imitate a unique song during a short sensitive period of early life ("age-limited learners") whereas others can learn to modify their vocal repertoire during their whole life ("open-ended learners"). It was also discovered that some species do not need to be exposed for weeks to models and can modify their vocalizations through imitation almost immediately [START_REF] Balsby | Vocal matching by orange-fronted conures (Aratinga canicularis)[END_REF]. For a long time, it was believed that oscine songbirds could not learn beyond the sensitive period. But successive studies have shown that song learning could still occur in young birds raised in social isolation or exposed to inappropriate models such as tape-recorded songs during the so-called sensitive period before being exposed to an appropriate model. For example, Baptista and Petrinovitch (1984) demonstrated that young white-crowned sparrows (Zonotrichia leucophrys) could learn songs after the end of the sensitive period if they were exposed to live tutors but not if they were exposed to taped songs. Most of these studies, done in the laboratory and/or with data collected in the field, provided insight into the processes of song learning, and more generally of vocal learning, in a more relevant context (Carouso-Peck et al., 2021).

Significant knowledge has been gained on the sensorimotor aspects of vocal production and learning these last 25 years with the development of computers and software. Here we present these different improvements for different purposes linked to learning in the vocal domain such as vocal recording and analysis, song tutoring, conditioning experiments and controlled vocal interactions. Even if most research deals with birdsong learning in oscine songbirds, our review is not restricted to vocal production learning capacities and could embrace other types of learning in the vocal domain such as contextual learning [START_REF] Janik | Vocal learning in mammals[END_REF] that could be applied to a wide range of bird species including species traditionally classified as non-vocal-learners. We will conclude by suggesting potential lines of research on the use of computers in the field of avian communication that seem feasible in the future.

Continuous vocal recording and analysis: from a single individual to group members

Before the age of computer technology, sound recordings were saved on tapes which drastically limited quantitative accumulation and analysis of data. It is also worth noting that the Kay Sona-Graph (which was also prohibitively expensive for many researchers) could not display a sonogram representing more than 1.5sec of sound, which for a long time restricted birdsong research to short vocalizations and thus often to songs of only a few species. A major step was reached when vocal recording and analysis were completely digitalized [START_REF] Tchernichovski | Studying the song development process: rationale and methods[END_REF][START_REF] Gardner | Freedom and rules: the acquisition and reprogramming of a bird's learned song[END_REF]. Such continuous recording and analysis permitted, for example, the discovery of the role of sleep on developmental song learning [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF][START_REF] Shank | Sleep and sensorimotor integration during early vocal learning in a songbird[END_REF], the role of melatonin on vocal production and learning in both vocal and non-vocal learners [START_REF] Derégnaucourt | Melatonin affects the temporal pattern of vocal signatures in birds[END_REF] and the ability to track cultural evolution of birdsong in the laboratory [START_REF] Fehér | De novo establishment of wild-type song culture in the zebra finch[END_REF]. Such methods also allowed the exploration of vocal ontogeny in both vocal and non-vocal learner species [START_REF] Gardner | Freedom and rules: the acquisition and reprogramming of a bird's learned song[END_REF][START_REF] Derégnaucourt | Dynamics of crowing development in the domestic Japanese quail (Coturnix coturnix japonica)[END_REF] and the discovery of some vocal changes that would have been unnoticed without continuous recording. Such systems were developed to record a bird kept singly in a sound-proof chamber, and to analyze the sound recorded online, which also permitted researchers to detect vocalizations and to discard cage noises. The invention of spectrographic analysis also allowed researchers to focus on the accuracy of imitation during vocal learning. Initially, imitation was determined by researchers eyeballing spectrograms with some issues on inter-observer reliabilities depending on their experience [START_REF] Jones | The interobserver reliability of scoring sonagrams by eye: a study on methods, illustrated on zebra finch songs[END_REF]. Some progress has been made by automatizing digital similarity measurements between different sounds, e.g., between a tutor's song and its imitation by a pupil (Tchernichovski et al., 2000;[START_REF] Lachlan | Are there speciesuniversal categories in bird song phonology and syntax? A comparative study of chaffinches (Fringilla coelebs), zebra finches (Taeniopygia guttata), and swamp sparrows (Melospiza georgiana)[END_REF]. Such an approach depends on specific song features selected by experimenters. In a more recent attempt, Mets and Brainard (2018) used a statistical model to broadly capture the structure of each song, and then estimated the divergence between the model and its imitation. The authors emphasized that this measure could be used to quantify changes in other behaviors such as human speech (Mets & Brainard, 2018). Continuous recordings at the group level is now permitted thanks to the development lightweight wireless microphone backpacks [START_REF] Anisimov | Reconstruction of vocal interactions in a group of small songbirds[END_REF][START_REF] Gill | A minimum-impact, flexible tool to study vocal communication of small animals with precise individual-level resolution[END_REF]. These methods allow birds to behave freely in a social group and enable studies of vocal interactions. For example, [START_REF] Gill | Patterns of call communication between group-housed zebra finches change during the breeding cycle[END_REF] observed that call occurrences and repertoire changed over the breeding cycle in a group of zebra finches (Taeniopygia guttata).

In another study, simultaneous recordings of individual vocalizations and multiunit vocal

premotor activity in breeding pairs of white-browed sparrow-weaver (P. mahali) ranging freely in their natural habitat permitted researchers to document how the precise interindividual coordination of motor activity during duet singing is neuronally controlled (Hoffman et al., 2019). The authors observed that rhythmic social interactions that require temporal coordination are not just associated with a coherent oscillation of activity in large areas of both interacting partners' brains but with an inter-individually synchronized activation of small groups of neurons within the same brain nucleus in both partners. Again, such study would have not been possible without continuous recording. In the field, microphone arrays are also increasingly used to remotely monitor acoustically active animals including birds [START_REF] Blumstein | Acoustic monitoring in terrestrial environments using microphone arrays: applications, technological considerations and prospectus[END_REF] but to our knowledge, they had never been used to study directly the learning capacities of birds in the vocal domain even if changes in the soundscape tracked by studies in the field of eco-acoustics [START_REF] Stowell | Ecoacoustics: acoustic sensing for biodiversity monitoring at scale[END_REF] might capture these aspects.

Controlled vocal interactions

As mentioned above, continuous recordings permitted researchers to document vocal interactions between two individuals and at the group level. This necessitates automatic detection of sounds and adapted response with the possibility to accurately control the delay between sound detection and response. Turn-taking is an important component of language and has been demonstrated in early infancy in humans and also in other species [START_REF] Levinson | Turn-taking in human communication-origins and implications for language processing[END_REF]. Such a process has also been the subject of much research in birds, especially in the context of territory defense in oscine songbirds and has been simulated using interactive playback (Slater et al., 2003). For example, song type matching is found in some species such as the song sparrow (Melospiza melodia; [START_REF] Akçay | Song type matching is an honest early threat signal in a hierarchical animal communication system[END_REF][START_REF] Beecher | Song-type matching between neighbouring song sparrows[END_REF]. It has been observed that birds approached a type-matching stimulus much more aggressively than a nonmatching stimulus [START_REF] Vehrencamp | Is song-type matching a conventional signal of aggressive intentions[END_REF]. Using a call detector implemented into an automated playback paradigm, [START_REF] Okanoya | A software bird call detector and its application to automated playback experiments[END_REF] observed that Bengalese finches (Lonchura striata) responded only to the Bengalese finch calls, but not the zebra finch calls, and female calls were more efficient in inducing responses than male calls. Using a dedicated software that permits continuous recording and real-time analysis, and a plastic model of a zebra finch equipped with a speaker, Benichov and colleagues (2016) investigated the neurobiological basis of turn-taking in zebra finches. Using an isochronous call pattern (e.g. one call per second), they observed that finches adjusted their call timing rapidly. Then, they changed the timing to enhance jamming, namely to increase the probability to overlap the birds' calls with calls sent by the system. All zebra finches adjusted their call pattern to avoid jamming, either by calling earlier, later or both. When the experimenters used a pattern of alternating single and paired calls, finches adjusted the timing of their calls differently for the single compared to the paired calls. Interestingly, females performed better than males at these tasks. The authors observed that fully functional forebrain regions, part of the so-called "song control system"a set of several brain nuclei involved in song production, perception and learningare necessary for this precise timing in both males and females [START_REF] Benichov | The forebrain song system mediates predictive call timing in female and male zebra finches[END_REF]. Such a close-loop computer system was also used successfully in domestic canaries to investigate the functions and context of production of two types of calls: "single calls" and "repeated calls" (Figure 1, [START_REF] Lerch | Closed-loop bird-computer interactions: a new method to study the role of bird calls[END_REF]. Modified from Lerch et al., Animal Cognition, 2011. The experimental procedures involved different stimuli (either natural or artificially synthesized) and different scenarios (e.g. positive interaction: the computer responded with a single call when the bird produced a single call and with a repeated call when the bird produced a repeated call, and negative interaction: the computer responded with a single call when the bird produced a repeated call, and with a repeated call when the bird produced a single call). Only females were tested: they produced significantly more single than repeated calls (either in response to natural and synthetized calls), and their rate of single calls was associated with the context (more often higher in the case of positive interaction), whereas repeated calls were context independent. This experiment confirmed a long-term assumption that "single calls" act as contact calls in canaries, whereas "repeated calls" are mostly used in a context of stress [START_REF] Mulligan | Communication in canary courtship calls[END_REF].

Enhancing song tutoring

In the study of song ontogeny, there have been methodological advances and improvements since the early days of simple playback of taped songs. This includes control of song playback by subject, playback augmented by visual stimuli, playback of vocal interactions and more recently the development of robots as song tutors.

Song playback

Different artificial methods have been developed in the laboratory to train young birds to learn songs. Since the pioneering works of Peter Marler using tape playback to train young sparrows to learn a song [START_REF] Marler | Birdsong and speech development: Could there be parallels? There may be basic rules governing vocal learning to which many species conform, including man[END_REF]Marler & Peters, 1977), tens of species have been exposed to their specific songs or other models during the sensitive period with mixed results: if some species do learn pretty well using such a passive exposure, some do not (e.g., Zebra Finch; [START_REF] Derégnaucourt | Comparisons of different methods to train a young zebra finch (Taeniopygia guttata) to learn a song[END_REF]. In this species, when song presentation was made contingent upon key pressing, young males copied to a greater extent than controls receiving the same stimulation in a passive way (Adret, 1993a) and the effectiveness of the contingency for initiating song imitation was greater for key pressing rather than hopping on a reinforcing perch (Adret, 1993b, but see [START_REF] James | Learning biases underlie "universals" in avian vocal sequencing[END_REF]. As previously mentioned, few daily exposure of song models are sufficient to trigger good learning [START_REF] Tchernichovski | Vocal imitation in zebra finches is inversely related to model abundance[END_REF][START_REF] Deshpande | Rapid encoding of an internal model for imitative learning[END_REF]. Using this paradigm and other experimental and computational approaches to manipulate and analyze the birds' vocal development, several studies have revealed different rules during birdsong developmental learning that exhibit analogies with language acquisition in humans [START_REF] Lipkind | Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants[END_REF][START_REF] Lipkind | Songbirds work around computational complexity by learning song vocabulary independently of sequence[END_REF][START_REF] Lipkind | The development of structured vocalizations in songbirds and humans: a comparative analysis[END_REF][START_REF] James | Learning biases underlie "universals" in avian vocal sequencing[END_REF]Mets & Brainard, 2018).

For example, zebra finches could be exposed to non-conspecific songs as models such as heterospecific sounds (Gehrold et al., 2013) and canaries to synthetic sounds [START_REF] Gardner | Freedom and rules: the acquisition and reprogramming of a bird's learned song[END_REF]. It was observed that this ability to learn non-conspecific songs is restricted to the production of avian babbling and does not persist into adulthood [START_REF] Gardner | Freedom and rules: the acquisition and reprogramming of a bird's learned song[END_REF]. In another experiment, young male zebra finches were exposed to synthesized songs consisting of five species-typical syllables arranged in every possible five-syllable sequence (n = 120 possible sequences) [START_REF] James | Learning biases underlie "universals" in avian vocal sequencing[END_REF]. The authors observed that all birds converged towards similar song sequences showing that there exist learning biases underlying song organization in this species. Dina Lipkind and colleagues have used extensively a paradigm that consists of switching song models during the sensitive period for song learning to study how young birds cope with vocal changes both in the temporal and the spectral domains during ontogeny [START_REF] Lipkind | Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants[END_REF][START_REF] Lipkind | Songbirds work around computational complexity by learning song vocabulary independently of sequence[END_REF][START_REF] Lipkind | The development of structured vocalizations in songbirds and humans: a comparative analysis[END_REF]. For example, young finches were exposed to a song model composed of three syllables (A-B-C) and when they started to produce the right sequence, they were exposed to another song model with the same syllables in a different order (A-C-B). Some birds succeeded to modified their song but after a huge practice involving their learning of new syllable transitions such as A to C and C to B [START_REF] Lipkind | Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants[END_REF]. Such vocal changes imposed experimentally in birds exhibit analogies with language acquisition in humans [START_REF] Lipkind | Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants[END_REF]. The study of interactions between genetic predispositions and experience is a huge topic in ethology and in birdsong research in particular. In a series of experiments with Bengalese finches, Mets and Brainard used both cross-fostering and computerized instruction with synthetic songs to demonstrate that matching the tutor song to individual predispositions can improve learning across genetic backgrounds (2019). All together, these examples show how being able to manipulate sound playback to birds during song ontogeny could shed lights on general mechanisms of learning.

In the field, experimental evidence of birdsong learning was demonstrated recently in wild savannah sparrows (Passerculus sandwichensis, Mennil et al., 2018). The authors used an array of loudspeakers to broadcast conspecific songs recorded in North America to birds living on an island. They also observed that these new songs spread in the subsequent generations.

Influence of visual stimuli

Several experiments have shown that birdsong acquisition is not an auditory phenomenon per se and that multimodal cues are also involved in the learning process, including visual stimuli.

Their influence on birdsong learning was demonstrated for example in nightingales: song acquisition was enhanced when playback was associated with strobe light produced synchronously with the temporal and amplitude patterns of the song models [START_REF] Hultsch | Auditory-visual stimulus pairing enhances perceptual learning in a songbird[END_REF]. Several experiments used visual stimuli combined with audio playback including taxidermic mounts or plastic dummies (see Derégnaucourt, 2010 for a review). [START_REF] Adret | Discrimination of video images by zebra finches (Taeniopygia guttata): direct evidence from song performance[END_REF] trained finches to peck a key to trigger a video movie and observed poor learning. This result could be explained by the flickering frequency of Cathode Ray Tube (CRT) used for this experiment, that exhibited a time resolution too low for birds. TFT (Thin Film Transistors) have been used successfully to induce behavioral reactions in birds (e.g. courtship songs in zebra finches and Bengalese finches: [START_REF] Ikebuchi | Male zebra finches and Bengalese finches emit directed songs to the video images of conspecific females projected onto a TFT display[END_REF] and for the purpose of song tutoring, even if, to our knowledge, there have been only a couple of studies dealing with these aspects that have been published and presented below. In an experiment in zebra finches, [START_REF] Deshpande | Rapid encoding of an internal model for imitative learning[END_REF] used different paradigms during a sole day of training (20 key pecks allowed): following a key-peck as described above, the audio-visual components of song models were played either simultaneously (Sim) or in a staggered fashion where birds received either audio before video (AV) or video before audio (VA). When trained at 45 dph (day post-hatch), AV and Sim groups showed significantly higher similarity to the song model than did an untrained group of birds kept in social isolation during the sensitive period.

However, there were no significant differences between the different groups of trained birds.

By contrast, when trained at 35 dph, differences in the efficacy of these audio-visual training stimuli became apparent. Surprisingly, a single training session at this age was effective only in the AV condition. This result is reminiscent of the Colavita visual dominance effect [START_REF] Colavita | Human sensory dominance[END_REF]: when one has to respond simultaneously to an auditory signal and to a visual signal, one tends to respond only to the visual signal which illustrates visual dominance in attentional processes. But as mentioned by the authors, further work is required to explore these aspects.

In a recent experiment, dyads consisting of a young male and an unrelated young female were exposed passively to audio-video recordings of a singing adult male. If birds approached the setup more often and spent more time close to it than juveniles exposed to audio playback only or audio playback combined with pixelated and time-reversed videos, they did not exhibit better imitation of the tutor song [START_REF] Varkevisser | Adding colour-realistic video images to audio playbacks increases stimulus engagement but does not enhance vocal learning in zebra finches[END_REF].

Playback of vocal interactions

Several studies suggest that listening to vocal interactions between adults or between an adult and a young could enhance vocal learning. Such paradigm, inspired by the seminal work of Dietmar Todt (Model/rival approach; [START_REF] Todt | Social learning of vocal patterns and modes of their application in grey parrots (Psittacus erithacus)[END_REF] has been used with success by Irene Pepperberg to train Alex and other grey parrots (Psittacus erithacus) to learn human words and to associate them with different types of objects (social modeling theory, [START_REF] Pepperberg | Social modeling theory: A possible framework for understanding avian vocal learning[END_REF]. In this context, parrots could observe interactions between two humans, one person being the 'trainer' and one being the 'model' as well as the 'rival.' [START_REF] Pepperberg | Social modeling theory: A possible framework for understanding avian vocal learning[END_REF]. Young European starlings (Sturnus vulgaris) raised in social isolation learned better when they could hear live interactions than while exposed to playback of taped songs [START_REF] Poirier | Direct social contacts override auditory information in the song-learning process in starlings (Sturnus vulgaris)[END_REF]. In another experiment with song sparrows (Melospiza melodia), hand-raised birds were exposed sequentially to two pairs of adult males in Summer [START_REF] Burt | Assessing the importance of social factors in bird song learning: a test using computer-simulated tutors[END_REF]. During the first three months of the next year, a computer program was used to simulate a live, interactive tutor.

Individuals were exposed to computer presentations of songs from two of the original live tutors: one of the two "virtual" tutors sang interactively with the young bird while the other sang asynchronously. The song classifier program used a spectrograph cross-correlation algorithm to compare the detected subject's song against examples from the tutor's repertoire.

The classifier returned a score indicating the similarity of the detected subject song to each of the tutor examples. If the score of the most similar tutor example type was above a fixed threshold, then the virtual tutor concluded that the subject had sung that type, otherwise it considered the subject had sung an unknown type. The virtual tutor then acted upon this information. The classifier parameters and similarity threshold were calibrated a priori using a test set of different song sparrow songs and the classifier output was periodically assessed during tutoring to verify that it was working reasonably [START_REF] Burt | Assessing the importance of social factors in bird song learning: a test using computer-simulated tutors[END_REF]. Subjects learned relatively more from their interactive tutor and his early partner than from the non-interactive one [START_REF] Burt | Assessing the importance of social factors in bird song learning: a test using computer-simulated tutors[END_REF]. More recently, it has been shown that wild savannah sparrows can learn novel songs broadcast by speakers but that songs broadcast in a simulated countersinging context were not better learned than songs broadcast by speakers simulating solo singing (Mennil et al., 2019). To perform this experiment, the authors created playback stimuli by assembling tutor songs into stereo playback tracks. Each track featured two simulated tutors engaged in simultaneous bouts of interactive singing, or in alternating bouts of non-interactive singing. 4.4.Towards an integrative approach to study the influence of multimodality: use of robots As mentioned above, birdsong learning is a multimodal task involving behavioral contingency and several studies have emphasized that a live tutor could be better than taped playback or a computer to train a bird to imitate songs (Baptista & Petrinovitch, 1986;[START_REF] Derégnaucourt | Comparisons of different methods to train a young zebra finch (Taeniopygia guttata) to learn a song[END_REF]. More recently, Mets and Brainard (2018b) demonstrated that increasing the richness of the learning experience from poor (tutoring by computer) to a richer level of interactions (tutoring by a live bird) reduces the genetic influence in favor of experiential influence in Bengalese finches. Experiments involving a human tutor are the most emblematic to illustrate this phenomenon, e.g. those of Jürgen Nicolai training bullfinches to imitate whistled melodies [START_REF] Nicolai | Human melody singing by bullfinches (Pyrrhula pyrrula) gives hints about a cognitive note sequence processing[END_REF]. To study the multimodal aspects of song tutoring, one should be able to handle each behaviour of the tutor independently, and assess the impact of each stimulus not only on the establishment and maintenance of the relationship between the young bird and the adult bird, but also on the success of song imitation. There is a growing interest for the use of robots in ethology (see [START_REF] Romano | A review on animal-robot interaction: from bio-hybrid organisms to mixed societies[END_REF] for a review) and ethoroboticseven if it is primarily focused on human-robot relationships -is a fast-growing discipline. For example, in birds, robots were used successfully to study courtship behaviour in bowerbirds [START_REF] Patricelli | Male displays adjusted to female's response[END_REF][START_REF] Patricelli | Male satin bowerbirds, Ptilonorhynchus violaceus, adjust their display intensity in response to female startling: an experiment with robotic females[END_REF][START_REF] Perry | Hidden Markov models reveal tactical adjustment of temporally clustered courtship displays in response to the behaviors of a robotic female[END_REF], filial imprinting in juveniles quails [START_REF] Jolly | Animal-to-robot social attachment: initial requisites in a gallinaceous bird[END_REF], collective and social behaviour in chickens [START_REF] Gribovskiy | Designing a socially integrated mobile robot for ethological research[END_REF] and public information in starlings [START_REF] Butler | European starlings recognize the location of robotic conspecific attention[END_REF]. We recently developed a robotic zebra finch that we used as a song tutor for young finches. The robot is that of a life-sized zebra finch and is painted to represent an adult male; it is fixed on an axis and can move different parts of its plastic body. It can be programmed to behave contingently: for example, when the young bird sits on a perch close to it, it moves towards him and sings. The program is also implemented with a vocal loop (see above): when the bird produces a contact call, the robot replied with a contact call and it is possible to manipulate the latency of the response.

Using such a contingent program, we observed that young finches exposed to a robot one hour a day for five days a week during five weeks provided a good imitation of the song model. We did not find a significant difference in song imitation between birds tutored with a robot and birds tutored by a live male [START_REF] Araguas | Design of a robotic zebra finch for experimental studies on developmental song learning[END_REF], this latter condition being known until now as the best way so far to get a close-to-perfect imitation of a song model in zebra finches [START_REF] Tchernichovski | Vocal imitation in zebra finches is inversely related to model abundance[END_REF][START_REF] Derégnaucourt | Comparisons of different methods to train a young zebra finch (Taeniopygia guttata) to learn a song[END_REF].

Conditioning experiments

The usual paradigm used in conditioning experiments consists in providing a reward for good responses (e.g. food) and a penalty for bad responses (lights off or electric shock). In go-no go procedures, animals are usually food-restricted to make sure that they would be motivated to peck on keys or press on levers to participate in experiments. Recently, new techniques have been developed by taking into account the socio-sexual tendencies of zebra finches. In these scenarios, finches do not work for food but to see and/or interact vocally with conspecifics.

For example, [START_REF] Tokarev | A novel paradigm for auditory discrimination training with social reinforcement in songbirds[END_REF] developed a new design to measure auditory discrimination (Figure 3). A male is allowed to interact with a female while perching next to a small window. Playbacks of one syllable were followed by an air puff (aversive syllable), causing the bird to escape to the safe perch, while playbacks of the other syllable allowed the interaction to continue. Using this paradigm, it was shown that learning auditory discriminations from observation is efficient but less robust than learning from experience [START_REF] Narula | Learning auditory discriminations from observation is efficient but less robust than learning from experience[END_REF]. The authors trained an "experimenter" zebra finch to discriminate between auditory stimuli in the presence of an "observer" finch. Experimenters were slow to successfully discriminate the stimuli, but immediately generalize their ability to a new set of similar stimuli. By contrast, observers subjected to the same task were able to discriminate the initial stimulus set, but require more time for successful generalization [START_REF] Narula | Learning auditory discriminations from observation is efficient but less robust than learning from experience[END_REF]. Taking again into account the natural tendency for isolated finches to seek socio-sexual contacts, Macedo-Lima and Remage-Healey (2020) replaced the food reward by a social reward for an experiment to measure auditory discrimination in zebra finches. A male and a female were put in adjacent cages separated by a sheet of opaque polarized glass. If a GO tone was played, the glass would turn transparent for 6 s, resulting in a HIT (i.e., period of visual engagement with the adjacent social partner through the transparent barrier). The authors did not document whether training was faster with a social reward rather than a food reward but it is likely that a dedicated experiment to test this prediction should be run in order to answer this question.

Future directions

Birdsong has been established as a biological model of choice to study vocal learning and, as a consequence, the goal of several experimental procedures is to enhance song imitation or even to obtain a perfect copy of the model [START_REF] Derégnaucourt | Comparisons of different methods to train a young zebra finch (Taeniopygia guttata) to learn a song[END_REF][START_REF] Mets | Learning is enhanced by tailoring instruction to individual genetic differences[END_REF]. The robotic zebra finch, which permits both vocal and behavioural contingencies with a young bird's behaviour, offers new opportunities to reach this goal and to explore meticulously the multimodal cues involved in birdsong learning [START_REF] Araguas | Design of a robotic zebra finch for experimental studies on developmental song learning[END_REF]. Such multimodal aspects of communication in birds could also be explored using other computer systems, such as the one recently developed to control vocal exchanges in a group [START_REF] Rychen | A system for controlling vocal communication networks[END_REF]; as mentioned by the authors, it would be easy to combine the visual communication channel using cameras and computer screens. Despite the importance of flight in the life of birds, birdsong has mainly been studied in perching birds, and vocal production learning in flight presents unique challenges and opportunities for birds and for researchers [START_REF] Berg | Phylogenetic and kinematic constraints on avian flight signals[END_REF][START_REF] Arnold | Vision and vocal communication guide three-dimensional spatial coordination of zebra finches during wind-tunnel flights[END_REF]. Affordable, high-speed, high-resolution cameras and mic-video array technology now available to tackle a number of questions in spatial and biomechanical issues related to highly mobile communication strategies.

Since the pioneering works of Adret (1993a, b) showing that young male zebra finches learn quickly to press a button to trigger song playback, several experiments have highlighted the active role of young songbirds on birdsong learning. Automatic learning devices also have potential to advance the study of social learning. For example, devices such as the one used with monkeys by [START_REF] Fagot | Automatic testing of cognitive performance in baboons maintained in social groups[END_REF] could allow experiments in which birds living in a social group choose when to participate.

Experiments involving young birds raised in social isolation during several weeks or months raise ethical issues. In ethology, this experimental procedure is still considered as the baseline reference conditions even if it exhibits several pitfalls already pinpointed a long time ago [START_REF] Lehrman | A critique of Konrad Lorenz's theory of instinctive behavior[END_REF]. More recently, it has been shown that acute social isolation has immediate consequences on brain gene activity in zebra finches [START_REF] George | Acute social isolation alters neurogenomic state in songbird forebrain[END_REF]. The use of biomimetic robots appear as a promising avenue to fulfill the 3Rs' objectives (Replace, Reduce, Refine: Replacement alternatives refer to methods which avoid or replace the use of animals, reduction alternatives refer to any strategy that will result in fewer animals being used to obtain sufficient data to answer the research question, refinement alternatives refer to the modification of husbandry or experimental procedures to minimize pain and distress and to improve welfare; [START_REF] Bierbach | Biomimetic robots promote the 3Rs Principle in animal testing[END_REF]; in birdsong research, robots could replace live companions and therefore potentially reduce the stress of social isolation without compromising highly standardized conditions. Interactions between wild birds and machines open also exciting opportunities of research such as the ability to track cultural evolution of birdsong following the introduction of new song models, but it also raises ethical questions [START_REF] Mennill | Eavesdropping on adult vocal interactions does not enhance juvenile song learning: an experiment with wild songbirds[END_REF]. The use of biomimetic robots could also be relevant in the field of biological conservation. For example, a recent study has shown that young regent honeyeaters (Anthochaera phrygia) are more likely to learn songs from heterospecific models due to the lack of sufficient conspecific adult tutors [START_REF] Crates | Loss of vocal culture and fitness costs in a critically endangered songbird[END_REF]. Robots mimicking conspecific adults could be used for song tutoring and could permit the maintenance of endangered dialects.

The use of Virtual Reality (VR) is also a promising avenue to study the learning capacities of birds in the vocal domain, especially in the field of birdsong learning [START_REF] Ljubičić | Social influences on song learning[END_REF].

For example, birds could be exposed to a Virtual Social Environment (VSE) that simulate the social context of song learning, through iterative interactions between the developing bird and a computerized visual and vocal interface comprising one or several birds [START_REF] Ljubičić | Social influences on song learning[END_REF]. Such a use is still at its infancy for behavioural studies in non-human animals. Studies on insects have pioneered the implementation of VR paradigms to study perceptual and cognitive capacities [START_REF] Lafon | Motion cues from the background influence associative color learning of honey bees in a virtual-reality scenario[END_REF].

In conclusion, our knowledge on learning capacities of birds in the vocal and auditory domains has been rapidly growing these last 20 years thanks to technological developments.

Many setups developed recently are based on inexpensive devices and free and open-source softwares. Cloud platforms permit the storage of huge amount of data that can be easily shared and accessed by researchers. Such tools used both in the laboratory and the field will permit researchers to get more insight into the evolutionary processes that led to species differences regarding learning abilities and to explore more deeply the umwelt of birds. These new techniques offer some opportunities to improve our knowledge of birds' behaviour in more natural settings, ultimately improving some ethical aspects of experimental research.

Figure 1 .

 1 Figure 1. Scheme of the closed-loop interaction between a canary and the computer system.

  More recently, Rychen and colleagues (2021) developed a system for real-time control of vocal interactions among separately housed animals. The system is implemented on a Field-Programmable Gate Array (FPGA) and it can control communication networks by restricting vocal exchanges among chosen individuals up to four animals (Figure2). For example, an individual could interact with other individuals that could not hear each other. Such system also gives the possibility to mimic eavesdropping situations when one individual can listen to vocal exchanges between other individuals without being noticed by the others.

Figure 2 .

 2 Figure 2. System controlling vocal communication networks. (a) The female T on top is

Figure 3 .

 3 Figure 3. Social reinforcement apparatus: playbacks of the 'social' song allow birds to

Acknowledgements

We would like to thank Irene Pepperberg for giving us to opportunity to write this review. We thank Kazuo Okanoya and two anonymous reviewers for their valuable comments and the Institut Universitaire de France for financial support.


Nour Hobloss
email: nour.hobloss@conti-engineering.com

Joshua Maraval
email: joshua.maraval@b-com.com

Jérôme Fournier
email: jerome.fournier@orange.com

Nicolas Ramin
email: nicolas.ramin@b-com.com

Lu Zhang

MUSE: A Multi-view Synthesis Enhancer

Keywords: View synthesis, subjective test, quality assessment

In this paper, we introduce the MUSE (MUltiview Synthesis Enhancer) method, which is an evolution of our previously proposed HDSB method and is based on a hybrid algorithmic-learning-based scheme. MUSE generates novel views of a scene using an autoencoder specifically optimized for refining pre-synthesized views that have been derived from actual observations. Since the subjective test is the ultimate test of the visual rendering quality, we evaluate our proposed method by two subjective tests. Experimental results show that the MUSE brings a global gain compared to two tested state-of-the-art methods.

I. INTRODUCTION

To enhance the user experience, advanced works in visual multimedia technologies and computer graphics have enabled the development of new immersive visual medias. In an immersive interactive experience, the "6 degrees of freedom (6DoF)" offers the user the possibility to be tracked not only by his head movement but also his body location as he physically moves left, right, forward, backward up, and down. The user can thus move around to freely change his location and point of view while watching a video.

The video transition is one possible intermediate step towards the 6DoF, where the user is not entirely free but has the choice to change his point of view using a transition that can seamlessly join two adjacent points of view of real cameras in the scene. A typical format for these applications is the Multi-view Video composed of a set of N video sequences representing the same scene, referred to as real views, acquired simultaneously by a system of N cameras positioned under different spatial configurations. An alternative representation is the Multi-View-Plus-Depth (MVD) format [START_REF] Merkle | Multi-view video plus depth representation and coding[END_REF], where the depth and texture information are used for each viewpoint.

The MPEG (Moving Picture Experts Group) proved a considerable interest in the MVD formats for their capacity to support 3D video applications. They proposed the View Synthesis Reference Software (VSRS), Reference View Synthesis (RVS) and Versatile View Synthesizer (VVS) [START_REF] Jung | VVS: Versatile View Synthesizer for 6-DoF Immersive Video[END_REF], consecutively. The VVS was released facing to the 6DoF challenges, which improves the precision of the backward texture warping and better preserves edges, compared to previous reference softwares.

Many other methods trying to improve the view synthesis quality have also been proposed in the litterature, such as rendering-algorithmic-based techniques [START_REF] Lin | Fast multi-view image rendering method based on reverse search for matching[END_REF], [START_REF] Prakash | Hybrid image-based rendering for free-view synthesis[END_REF], disocclusion inpainting methods [START_REF] Thatte | A statistical model for disocclusions in depthbased novel view synthesis[END_REF], [START_REF] Satapathy | Robust depth map inpainting using superpixels and non-local gauss-markov random field prior[END_REF], learning-based methods [START_REF] Choi | Extreme view synthesis[END_REF], [START_REF] Volker | Learning light field synthesis with multi-plane images: Scene encoding as a recurrent segmentation task[END_REF] or methods based on radiance field estimation [START_REF] Mildenhall | Nerf: Representing scenes as neural radiance fields for view synthesis[END_REF], [START_REF] Müller | Instant neural graphics primitives with a multiresolution hash encoding[END_REF]. Learning-based and radiance-field-based view synthesis methods achieving encouraging and outstanding results accelerated and condensed the work in this field. However, some of them do not attach importance to the number of input images and the computational/memory cost, and the improvement is mainly performed on small baselines from the target view-point and many input viewpoints.

The larger the distance between the cameras is, the wider the baseline is. Our works focus on developing solutions that deal with large baselines view synthesis while mixing the advantages in algorithmic-based warping method with the benefits of Convolutional Neural Networks (ConvNets) on improving the final rendered image quality. In our previous work [START_REF] Hobloss | Hybrid dual stream blender for wide baseline view synthesis[END_REF], we proposed a Hybrid Dual Stream Blender for wide baseline view synthesis (HDSB), where reference views were algorithmically warped to the target position and then blended via a ConvNet, followed by a residual encoderdecoder for image blending with a Siamese encoder to keep the parameters low. However, the HDSB did not guarantee a network generalization due to the lack of variety in the limited available training database.

In this paper we propose an improved version of the HDSB, called MUSE: A Multi-view Synthesis Enhancer. It is still based on our previous idea of a hybrid algorithmic-learning scheme where reference views are preliminarily warped to the target position using an inpainting method built around a mean value to handle occlusions. But the MUSE differs from the HDSB in the following aspects:

• The warped and inpainted left and right reference views are preliminary merged by a sum weighted by a factor α, to increase the contribution of the closest reference view, reducing the impact of the projection of incorrect geometry estimated from distant points of view, and control the view-dependent appearance of non-Lambertian surfaces. • The masks of the disocclusion areas are preliminary merged by the boolean binary operator "and", to generate one mask that contains areas invisible by either the two reference views. • A concatenation of the generated mask and the merged references form the only one input data considered by the network, and a single encoder can thus be used. • The network does not blend the views but corrects the image artifacts resulting from the pre-merged image, and improves the quality of the final image. • The learning process of the network is called "intracontent" and therefore, it is different from the intercontent learning process used for the HDSB. This ensures that the network learns the specific content features of each sequence by over-learning on the available views.

Only the triplet images (left warped image, right warped image and the reference view) are considered for a given content. Another contribution of the paper is that we conducted two subjective tests to evaluate the rendered quality of our method, different from many works where the methods are only evaluated by objective quality metrics which don't always correlate with human perception.

In the following, we describe the proposed method in section II, present the experiments for the performance evaluation in section III, then show and discuss the results in section IV, finally give the conclusions and perspectives in section V.

II. PROPOSED METHOD

We illustrate the MUSE architecture in Fig. 1, which includes two main stages: a pre-synthesis process and a ConvNet-based view enhancer.

Encoder

Bottleneck Decoder ′ ′ α.

(1 -α).

A. Pre-synthesis process

The pre-synthsis process, in the same way as in the HDSB, uses the two reference views to generate the view to be synthesized. The steps leading to the generation of the warped and inpainted left and right reference texture images T Lw f T Rw f , and the generation of the binary masks (M L , M R ) corresponding to the disocclusion area in (T Lw , T Rw ), respectively, are the same as those described for the HDSB [START_REF] Hobloss | Hybrid dual stream blender for wide baseline view synthesis[END_REF].

But here, the view T ′ is pre-synthesized by merging the warped ans inpainted texture images T Lw f T Rw f by a sum weighted by a factor α:

T ′ = α • (T Lw f ) + (1 -α) • T Rw f (1)
We use a factor α that corresponds to the normalized distance between the virtual view and the two reference views and allows to increase the contribution of the reference view that is closest to the virtual one. For example, if the virtual view is very close to the left view, the factor α will be close to 1 and the contribution of the left view will be dominant. The fusion of the binary masks M Lb , M Rb is performed by the boolean operator and, and the produced common disocclusion map contains then only the areas that are not observed by either the left or the right view. The tensor B is the result of the concatenation of T ′ which is a 3-channels color image (in RGB or YUV format) and the mask M ′ of the disoccluded areas. B is the input to the neural network whose role is to improve the quality of the pre-synthesized view T v .

B. ConvNet-based view enhancer

The architecture of the neural network enhancer is similar to the architecture used by the HDSB. However, there are some remarkable differences between HDSB architecture and the view-enhancing problem considered here. First, we only need to deal with a single input (the concatenation of the merged reference views with the mask B), rather than four inputs in HDSB, thus only one encoder is used here, instead of two. Second, the pre-synthesized image is characterized by particular types of artifacts due to algorithmic blending/inpainting imperfections (such as ghosting problems shown in Fig. 2), instead of disocclusion problems. Our network should learn how to enhance the quality of the input blended image by reducing artifacts and ghosting effects and refining the image details using an adapted architecture. Thus, we use an encoder-decoder architecture that uses a single blended image as input and generates one output image and consists of three parts: encoder, bottleneck and decoder. The three parts are the same as in the HDSB architecture as they achieve the best tradeoff between the semantic depth and the spatial resolution of the output feature maps. As well as minimizing training time, we utilize the Mean Squared Error (MSE) loss function.

III. EXPERIMENTS

For the performance evaluation, we did two subjective tests, one for the perceived quality evaluation of the view synthesis methods and the other for the quantification of the differences in the quality of the image synthesis methods. We compared our new method MUSE, the MPEG reference software VVS and our previously proposed HDSB with the subjective results.

A. Dataset

We used 8 multi-view video sequences here, of which

B. MUSE training settings

For each sequence, we extract three neighbor views from the first 750 frames: the left and right views are used as references (T L , T R ), and the central view is used as the ground-truth. We firstly warp, inpaint, and blend T L and T R to obtain T ′ . Our network is then trained on the triplet images (T ′ , the corresponding binary mask M ′ and the ground-truth image).

From each sequence, we 75k triplets of co-located patches of size 64 × 64, and get 52k training patches, 15k validation patches, and 7500 test patches. During the training, we used batches of 128 patches, a learning rate of 0.0001 (leading to the convergence of our learning algorithm after 100 epochs), and Adam optimization algorithm with weight decay = 0 and betas = (0.9, 0.999). All the experiments are performed on a server with an NVIDIA RTX2080GPU.

C. Subjective test-bed

A 24-inch LED PC monitor (ASUS VG248QE) is used to manage the stimulis presentation to be evaluated. The voting interface is displayed after.

In order to reproduce video sequences of different frame rates, a screen with V-SYNC/G-SYNC capabilities is used to guarantee a real-time display without visual artifacts. Therefore, the constituted test-bed optimizes the display of uncompressed video sequences (yuv/1080p/8bits format), recorded on SSD and played with "mpv."s player (an open-source media player software based on ffmpeg [START_REF] Mpv | a free, open source, and cross-platform media player[END_REF]).

The test protocol follows the ITU-R BT.500-14 recommendation [START_REF]500-14: Methodologies for the subjective assessment of the quality of television images[END_REF]. Twenty-seven non-expert observers between the age of 23 and 53 participated in the two tests.

D. Two Subjective Tests

Our subjective tests aim to evaluate the visual quality of our method in the case of an actual view synthesis application, which is a video transition. A video transition is a technique used in post-production video-editing in order to join two shots together. In our tests, we consider the transition from one point of view to another in the same scene. Therefore, we use our method to create a virtual transition between two cameras in the same scene to change the user's point of view. To achieve this, we create several intermediate views that are played one after another, such as a "path" between the starting and ending points. Tab. I shows the scene properties for each sequence to create the transitions, i.e., the number of real cameras in the scene, the baseline... It also shows the number of synthesized images in every scene, necessary to transition between two real cameras, with its duration.

1) Test 1: Evaluation of the perceived quality of the view synthesis methods: Five-scale Absolute Category Rating (ACR) method (cf. P.910 [START_REF] Rec | Subjective video quality assessment methods for multimedia applications[END_REF]) is adopted in this test, because of the absence of the natural contents.

With 8 different sequences and 3 synthesis methods (VVS, HDSB, and MUSE), we have 24 test transitions in total. For each observer, the order of presentation of the sequences is different, and each sequence is repeated once. Before starting the test, the protocol and the objectives of the test are presented to each participant in a test instruction sheet.

2) Test 2: Quantification of the differences in the quality of the image synthesis methods: This test is more accurate than Test 1 in estimating the differences in rendering between the synthesis methods from a subjective point of view. Therefore, for each scene, it is possible to determine if MUSE offers a better visual rendering than the two other methods and quantify this difference on an appropriate perceptual scale. However, this method does not determine the level of perceived visual quality for each measurement point (a view synthesis method associated with a scene), as was the first subjective test case. The two tests are thus complementary. I: Summary of the characteristics of each scene and their associated capturing system Test 2 is similar to the Double Stimulus Continuous Quality Scale (DSCQS) (cf. BT.500 [START_REF]500-14: Methodologies for the subjective assessment of the quality of television images[END_REF]). It requires prior scheduling of the sequences to be tested because the view synthesis methods will be compared in pairs (video A and video B). For a given scene, three pairs are compared: (MUSE vs VVS), (MUSE, HDSB), and (VVS, HDSB). After that each pair is displayed twice, participants are asked to report their 7-scale score on the perceived quality difference (whether the quality of video B is "Much worse," "Less good," "Slightly worse," "Equivalent," "Slightly better," "Better," or "Much better" than video A).

IV. RESULTS AND DISCUSSIONS A. From Subjective Test 1

Tab. II lists the Mean Opinion Score (MOS) obtained by computing the average of the inter-observer and inter-scene scores for each tested method. The ANOVA test is also done and the p-values are alway below the level of 5% of significance. The results demonstrate thus that the performance of MUSE is significantly better than those of VVS and HDSB. If we look further into each scene (cf. Fig. 4), we can find that MUSE is better than VVS and HDSB on all the tested scences, except that VVS performs favorably for the VikingVillage sequence. For this scene, we could observe that prominent, annoying, and time-varying artifacts are predominant in the areas where occlusion alternate with disocclusion zones. We can reasonably hypothesize that the temporal smoothing provided by VVS would bring again compared to the pure spatial processing performed by MUSE. Thus, the MUSE method brings a global gain compared to the VVS method. The taking into account of the temporal neighborhood represents an improvement perspective of the MUSE method.

B. From Subjective Test 2

Tab. III lists the inter-observer and inter-scene MOS for each pair of the compared methods, from which we observe that: the quality results of HDSB are relatively equivalent to those of VVS; the quality results of MUSE are better than those of HDSB; the quality results of MUSE are better than those of VVS. Fig. 5 illustrates then the inter-observer MOS for each scene and pair of compared methods. • HDSB produces similar or worse results compared to VVS, except for the PoznanFencing scene, for which the visual quality of the HDSB method is judged better. The analysis of the two tests' results also shows that the quality of the view synthesis is strongly dependent on the content. The visible artifacts are related to errors in the depth maps, which are more critical when the disparity amplitude is more significant. As well as the camera focal lengths, the spacing, and the object relative distances in the scene, affect the quality of the synthesized views. The synthetic contents for which ideal depth maps have been used escape this rule. A more detailed study of the link between the configuration of the rig and the quality of the synthesized views would allow us to confirm these hypotheses.

V. CONCLUSIONS AND PERSPECTIVES

In this paper, we propose a new view synthesis method MUSE. We qualitatively evaluate our method by conducting two subjective tests that compare MUSE's performances with two other state-of-the-art view synthesis methods (HDSB and VVS). These tests were performed on an actual view synthesis use-case which is a video transition between two points of view in the same scene. Fig. 6: From multi-view capture to 6DoF rendering Fig. 6 illustrates the final targeted user experience towards the 6DoF video navigation, that can benefit from our new method. The user (1) visualizes, with the help of a 6-DoF player installed on a device such as a PC, a tablet, or a virtual reality headset, the video sequence associated with a real camera [START_REF] Jung | VVS: Versatile View Synthesizer for 6-DoF Immersive Video[END_REF]. He/she can navigate from one real point of view to another real point of view. Between two real points of view, the images associated with a virtual camera (3) are generated by an off-line synthesizer such as MUSE and played back by the player [START_REF] Merkle | Multi-view video plus depth representation and coding[END_REF].

Our future research regarding the view synthesis technique aims to exploit the temporal information to improve the hole filling procedure and impose temporal consistency among neighboring frames as an additional constraint at training and inference stages. Since the subjective tests are costly and timeconsuming, we only compared the MUSE with two other methods this time. But we will compare our method with more state-of-the-art view synthesis methods in the future.

Fig. 1 :

 1 Fig. 1: MUSE architecture illustration, where α corresponds to the normalized distance between the virtual view and two left and right reference views that frame the target view and regulate the reference views' mixing within the presynthesized view.

Fig. 2 :

 2 Fig. 2: Illustrations of algorithmic blending/inpainting imperfections in a pre-synthesized image.

3

 3 are synthetic (Adventure, OrangeShaman, and Viking Village) and the other 5 are from real video capturing (PandemoniumRig1, PoznanCarpark, PoznanFencing, PoznanStreet, and Technicol-orPainter), as illustrated in Fig. 3. The scenes Adventure and VikingVillage are generated by IRT b<>com from free Unity assets; OrangeShaman, PoznanCarpark, PoznanStreet and TechnicolorPainter are excerpts from the original scenes proposed to the MPEG Immersive Video working group; PandemoniumRig1 was captured by IRT b<>com with the same camera rig described in [12]. Tab. I summarizes the characteristics of the different sequences.

Fig. 3 :

 3 Fig. 3: Illustration of the selected contents.

AFig. 4 :

 4 Fig.4: MOS and 95% confidence intervals obtained for each scene and each view synthesis method.

Fig. 5 :

 5 Fig. 5: MOS and 95% inter-user confidence intervals for each scene and pair of compared methods

TABLE

TABLE II :

 II MOS for each view synthesis method and the bold font indicates the best result.

TABLE III :

 III MOS for each pair of compared methodsTest 2 allows us to quantify the difference in visual quality between the different view synthesis algorithms (HDSB, VVS and MUSE). The analysis of the results corroborates the conclusions of the first test and shows that:

	• MUSE performs favorably compared to VVS for all the
	scenes except VikingVillage. The contributions are most
	significant for the real scenes PandemoniumRig1 and
	PoznanFencing. The results for the OrangeShaman and
	PoznanStreet sequences are relatively comparable

• MUSE is a real improvement over HDSB. The results for PoznanStreet and TechnicolorPainter are slightly better, while the results for the other contents are much better.