Martin Killian
email: killian.martin@ens-lyon.fr

Olivier Adam

Nicolas Obin

Valérie Dufour

Rookognise: Acoustic detection and identification of individual rooks in field recordings using multi-task neural networks

Keywords: deep learning, bird recognition, outdoor setting, vocalisation-independent, corvid, multi-channel audio

Individual-level monitoring is essential in many behavioural and bioacoustics studies.

Collecting and annotating those data is costly in terms of human effort, but necessary prior to conducting analysis. In particular, many studies on bird vocalisations also involve manipulating the animals or human presence during observations, which may bias vocal production. Autonomous recording units can be used to collect large amounts of data without human supervision, largely removing those sources of bias. Deep learning can further facilitate the annotation of large amounts of data, for instance to detect vocalisations, identify the species, or recognise the vocalisation types in recordings. Acoustic individual identification, however, has so far largely remained limited to a single vocalisation type for a given species. This has limited the use of those techniques for automated data collection on raw recordings, where many individuals can produce vocalisations of varying complexity, potentially overlapping one another, with the additional presence of unknown and varying background noise. This paper aims at bridging this gap by developing a system to identify individual animals in those difficult conditions. Our system leverages a combination of multi-scale information integration, multi-channel audio and multi-task learning. The multi-task learning paradigm is based the overall task into four sub-tasks, three of which are auxiliary tasks: the detection and segmentation of vocalisations against other noises, the classification of individuals vocalising at any point during a sample, and the sexing of detected vocalisations. The fourth task is the overall identification of individuals. To test our approach, we recorded a captive group of rooks, a Eurasian social corvid with a diverse vocal repertoire. We used a multi-microphone array and collected a large scale dataset of time-stamped and identified vocalisations recorded, and found the system to work reliably for the defined tasks. To our knowledge, the system is the first to acoustically identify individuals regardless of the vocalisation produced. Our system can readily assist data collection and individual monitoring of groups of animals in both outdoor and indoor settings, even across long periods of time, and regardless of a species' vocal complexity. All data and code used in this article is available online.

Introduction

Deep learning systems have become invaluable tools in the ecological sciences in recent years.

They allow the automation of many monitoring tasks, enabling much wider spatio-temporal coverage than previous methods [START_REF] Christin | Applications for deep learning in ecology[END_REF][START_REF] Weinstein | Scene-specific convolutional neural networks for video-based biodiversity detection[END_REF]. Deep learning system have been applied to tasks as diverse as detecting species in complex environments (e.g [START_REF] Conrady | Automated detection and classification of southern african roman seabream using mask r-cnn[END_REF][START_REF] Dufourq | Passive acoustic monitoring of animal populations with transfer learning[END_REF][START_REF] Fu | A case study of utilizing yolot based quantitative detection algorithm for marine benthos[END_REF][START_REF] She | A method for automatic real-time detection and counting of fruit fly pests in orchards by trap bottles via convolutional neural network with attention mechanism added[END_REF][START_REF] Van Klink | Emerging technologies revolutionise insect ecology and monitoring[END_REF], censusing populations [START_REF] Adi | Acoustic censusing using automatic vocalization classification and identity recognition[END_REF], surveying breeding success of potentially endangered species [START_REF] Teixeira | Fledge or fail: Nest monitoring of endangered black-cockatoos using bioacoustics and open-source call recognition[END_REF], or tracking invasive species both for plants and animals [START_REF] Campos | Acoustic region workflow for efficient comparison of soundscapes under different invasive mammals' management regimes[END_REF][START_REF] Li | Classification and detection of insects from field images using deep learning for smart pest management: A systematic review[END_REF][START_REF] Takimoto | Using a twostage convolutional neural network to rapidly identify tiny herbivorous beetles in the field[END_REF]. Beyond the species level, observing individual animals remains a challenge for researchers [START_REF] Ferreira | Deep learning-based methods for individual recognition in small birds[END_REF] despite its essential role in ecological and behavioural studies [START_REF] Clutton-Brock | Individuals and populations: The role of long-term, individual-based studies of animals in ecology and evolutionary biology[END_REF][START_REF] Terry | The role of vocal individuality in conservation[END_REF]. Extensive research has evidenced the possibility of identifying individual animals by their vocalisations (see e.g. [START_REF] Beecher | Signalling systems for individual recognition: An information theory approach[END_REF][START_REF] Bradbury | Principles of Animal Communication (2nd Edition)[END_REF][START_REF] Linhart | Measuring individual identity information in animal signals: Overview and performance of available identity metrics[END_REF], opening the possibility of applying passive acoustic monitoring techniques [START_REF] Schneider | Past, present and future approaches using computer vision for animal re-identification from camera trap data[END_REF] to individual monitoring. Bird vocalisations have been studied in the ecological and evolutionary sciences due to their pervasive importance to the life history of birds (see e.g. [START_REF] Catchpole | Bird song: Biological themes and variations[END_REF] and Marler and Slabbekoorn (2004) for reviews), but also because of their potential analogies with human language (e.g. [START_REF] Sainburg | Parallels in the sequential organization of birdsong and human speech[END_REF]. Current research questions investigate, for example, the interplay between vocal production and sociality in the form of conversations, dialects and vocal signatures in many bird species.

The vocal signature is the set of bioacoustic features that are individually distinctive within a given call type. Such signatures have been found throughout the animal kingdom [START_REF] Jansen | Segmental concatenation of individual signatures and context cues in banded mongoose (Mungos mungo) close calls[END_REF][START_REF] Kershenbaum | The Encoding of Individual Identity in Dolphin Signature Whistles: How Much Information Is Needed?[END_REF][START_REF] Linhart | Measuring individual identity information in animal signals: Overview and performance of available identity metrics[END_REF][START_REF] Mccordic | Calls of North Atlantic right whales Eubalaena glacialis contain information on individual identity and age class[END_REF]. However, most studies in vocal signature have been limited to a single vocalisation type (e.g. [START_REF] Benti | Individual signature in the most common and contextindependent call of the Rook (Corvus frugilegus)[END_REF][START_REF] Boeckle | Who wants food? Individual characteristics in raven yells[END_REF][START_REF] Boeckle | Raven food calls indicate sender's age and sex[END_REF][START_REF] Laiolo | A study of Choughs' vocal repertoire: Variability related to individuals, sexes and ages[END_REF][START_REF] Mates | Acoustic profiling in a complexly social species, the American crow: Caws encode information on caller sex, identity and behavioural context[END_REF][START_REF] Stowell | Individual identity in songbirds: Signal representations and metric learning for locating the information in complex corvid calls[END_REF][START_REF] Stowell | Automatic acoustic identification of individual animals: Improving generalisation across species and recording conditions[END_REF][START_REF] Yorzinski | The Inflected Alarm Caw of the American Crow: Differences in Acoustic Structure Among Individuals and Sexes[END_REF]. When encompassing multiple different vocalisation types, the concept of vocal signature becomes a lot more nebulous, as different vocalisation types can bear different signatures [START_REF] Elie | Zebra finches identify individuals using vocal signatures unique to each call type[END_REF][START_REF] Keenan | The reliability of individual vocal signature varies across the bonobo's graded repertoire[END_REF]. This plurality of signatures obviously complicates the identification of individ-uals in species with a rich repertoire, at least with classical methods [START_REF] Catchpole | Bird song: Biological themes and variations[END_REF][START_REF] Elie | Zebra finches identify individuals using vocal signatures unique to each call type[END_REF]. Despite this, there have been a few cases of successful individual identification with multiple vocalisation types [START_REF] Cheng | A comparative study in birds: Call-type-independent species and individual recognition using four machine-learning methods and two acoustic features[END_REF][START_REF] Fox | Call-independent individual identification in birds[END_REF][START_REF] Kirschel | Using songs to identify individual mexican antthrush Formicarius moniliger: Comparison of four classification methods[END_REF], most successfully with pre-deep learning era neural networks. Still, these approaches were based on supervised algorithms trained on vocalisations manually extracted from the recordings, limiting their applicability to field recordings [START_REF] Kershenbaum | Acoustic sequences in non-human animals: A tutorial review and prospectus[END_REF][START_REF] Thompson | A system for describing bird song units[END_REF] Neural networks appear as the most appropriate tool to acoustically classify birds down to the individual level. Recent breakthroughs in general machine learning and specifically deep learning [START_REF] Stowell | Computational bioacoustics with deep learning: a review and roadmap[END_REF] have brought state-of-the-art results just in bird vocalisation detection [START_REF] Fanioudakis | Deep networks tag the location of bird vocalisations on audio spectrograms[END_REF][START_REF] Grill | Two convolutional neural networks for bird detection in audio signals[END_REF][START_REF] Kong | Joint detection and classification convolutional neural network on weakly labelled bird audio detection[END_REF][START_REF] Liaqat | Domain Tuning Methods For Bird Audio Detection[END_REF][START_REF] Lostanlen | Robust sound event detection in bioacoustic sensor networks[END_REF][START_REF] Sevilla | Audio bird classification with inception-v4 extended with time and time-frequency attention mechanisms[END_REF][START_REF] Stowell | Automatic acoustic detection of birds through deep learning: The first Bird Audio Detection challenge[END_REF], species classification [START_REF] Kahl | Overview of BIRDCLEF 2019: Large-scale bird recognition in soundscapes[END_REF][START_REF] Kahl | BirdNET: A deep learning solution for avian diversity monitoring[END_REF], and segmentation of song with classification of vocalisation types [START_REF] Cohen | Automated annotation of birdsong with a neural network that segments spectrograms[END_REF]. But the deep learning community has so far only rarely investigated the problem of individual identification. One possible explanation is that in situ recording remains challenging, in particular recording individual birds [START_REF] Folliot | Using acoustics and artificial intelligence to monitor pollination by insects and tree use by woodpeckers[END_REF][START_REF] Stowell | Automatic acoustic identification of individual animals: Improving generalisation across species and recording conditions[END_REF], due to interferences from other sound sources, including conspecifics, and simple wariness preventing the observer from approaching enough to obtain many recordings of good quality. As such, many studies relying on individual recordings simply record the bird alone in a separate chamber (e.g. [START_REF] Sainburg | Parallels in the sequential organization of birdsong and human speech[END_REF]; however, this experimental setting deprives birds of their natural and social environment and can bias vocal production. In addition, acquiring manually individually annotated data is time consuming and generally costly [START_REF] Stowell | Automatic acoustic identification of individual animals: Improving generalisation across species and recording conditions[END_REF]. Autonomous recording units (ARUs) can be used to record audio in a more ecologically valid environment, i.e. outdoor, in a social group, and without the need for human presence. They have been increasingly used in ecological surveys, since they allow wide coverage with little human effort [START_REF] Blumstein | Acoustic monitoring in terrestrial environments using microphone arrays : applications , technological considerations and prospectus[END_REF][START_REF] Darras | Autonomous sound recording outperforms human observation for sampling birds: a systematic map and user guide[END_REF][START_REF] Fristrup | Biacoustical monitoring in terrestrial environments[END_REF][START_REF] Potamitis | Automatic bird sound detection in long real-field recordings: Applications and tools[END_REF][START_REF] Schneider | Past, present and future approaches using computer vision for animal re-identification from camera trap data[END_REF][START_REF] Shonfield | Autonomous recording units in avian ecological research: current use and future applications[END_REF].

Deep learning is adequately suited to process the vast amount of complex data that can be collected with ARUs thanks to the representational power offered by these networks. Furthermore, the network's architecture can be adapted to treat the various tasks at (in our case, detection of vocalisation and individualisation). For instance, multiple tasks can be tackled using a multi-task learning approach, i.e. by learning the tasks jointly within a single network.

This approach enables the network to learn a shared representation, which has been suggested to effectively bias the learning process towards representations that benefit all tasks [START_REF] Caruana | Multitask Learning[END_REF][START_REF] Liebel | Auxiliary Tasks in Multi-task Learning[END_REF][START_REF] Ruder | An Overview of Multi-Task Learning in Deep Neural Networks[END_REF]. This shared representation can not only enhance training, but also the generalisation ability of the network. This is usually implemented by sharing early parts of the network, before splitting into task-specific heads. For instance, one head may learn to detect objects over the background, and another may learn to classify the detected objects [START_REF] Morfi | Deep learning for audio event detection and tagging on lowresource datasets[END_REF][START_REF] Pankajakshan | Polyphonic sound event and sound activity detection: A multi-task approach[END_REF]. In our approach, we define four such sub-tasks: the detection sub-task whose objective is to detect for each audio frame whether a bird is vocalising or not (versus any other sound), the sexing sub-task whose objective is to further classify whether the individual producing the vocalisation is a male or a female, the presence sub-task whose objective is to identity the presence of a vocalisation of a particular individual within a large audio clip regardless to its precise time position, and the identification sub-task, which returns time-stamped vocalisations with the identity of the individual producing each vocalisation.

We then tested the performance of the system on rooks (Corvus frugilegus), held in a social group and in an outdoor setting. Rooks are a European social corvid with a rich vocal repertoire [START_REF] Røskaft | Vocal communication by the rook Corvus frugilegus during the breeding season[END_REF]. Corvids are known for producing a large number of different vocalisations with a chaotic structure and a wide frequency range [START_REF] Brown | The Role of Song and Vocal Imitation among Common Crows (Corvus brachyrhynchos)[END_REF][START_REF] Marzluff | Cultural Coevolution: How the Human Bond with Crows and Ravens Extends Theory and Raises New Questions[END_REF][START_REF] Røskaft | Vocal communication by the rook Corvus frugilegus during the breeding season[END_REF], particularly difficult to parametrise using classical acoustic measures [START_REF] Fagerlund | Parametrization of inharmonic bird sounds for automatic recognition[END_REF][START_REF] Fletcher | A class of chaotic bird calls?[END_REF]. Furthermore, individuals only share some of their repertoire with their conspecifics [START_REF] Boeckle | Who wants food? Individual characteristics in raven yells[END_REF][START_REF] Kondo | Perceptual mechanism for vocal individual recognition in jungle crows (Corvus macrorhynchos): Contact call signature and discrimination[END_REF][START_REF] Mates | Acoustic profiling in a complexly social species, the American crow: Caws encode information on caller sex, identity and behavioural context[END_REF], further limiting the use of classical methods for individual identification.

Rooks, in particular, may possess an even richer repertoire than previously thought, as they produce series of vocalisations similar to the song of many other oscines [START_REF] Coombs | Observations on the Rook Corvus frugilegus in Southwest Cornwall[END_REF], that have never been described in detail. While further description is beyond the scope of this paper, rooks appear to be of particular interest to develop a network capable of vocalisationindependent individual identification.

To summarise, in this paper we present the following contributions, to overcome the aforementioned limitations:

1. We present the first system able to acoustically identify individual birds whatever their vocalisation types, which detects vocalisations from raw recordings and attributes them to the individual emitter. Furthermore, the system was developed using data sampled in an outdoors aviary and was trained to learn its tasks in an environment with multiple sources of noise. The system is trained in a fully supervised manner, on a dataset of identified rook vocalisations. The code for this system is available online.

2. The architecture is based on a multi-task learning approach. This architecture is used in order to learn information useful for multiple related tasks, and to increase the generalisation ability of the system compared to multiple single-task networks. The multi-task approach exploits information from a shared representation learned by all tasks, followed by task-specific heads. This formulation also lets us fuse the predictions of several subtasks together as an attention-like mechanism (e.g. where the detection head returns a low score, there is no need for the identification head to learn to identify an individual).

Furthermore, a multi-scale module is added to allow the network to exploit information from very local cues in the spectrogram up to vocalisation-wide patterns to find individually-distinctive information.

3. The proposed multi-task network operates on multi-channel audio signals, as recorded from a multi-microphone array. This allows the system to exploit spatial information to enhance the detection of rook vocalisations as well as the subsequent tasks. The multichannel recording also adds redundancy in the case of degraded, noisy or weak signals (e.g. saturation due to a bird vocalising close to the microphone, or strong non-vocal noises). To our knowledge, this is the first implementation of a neural network that exploits spatial information in the domain acoustic individual identification.

4. We collected the first large scale annotated dataset of rook vocalisations. This dataset includes vocalisations from 15 individual rooks collected over the course of one year, with each vocalisation time-stamped and attributed to the individual producing it. The data collection involved a multi-microphone array spread throughout the aviary to record each bird from as close as possible. As such, the recordings are composed of multichannel audio, unlike most approaches in the literature. The entire annotated dataset is freely available online under a Creative Common (CC/BY 4.0) Licence2 .

2 Material and methods

Data collection

We recorded the vocalisations produced by a captive colony of 15 adult rooks (8 males, 7 females) housed in an outdoor aviary in Strasbourg, France. All birds were identified by coloured leg rings and had been housed together since they had been caught as fledglings, except one adult male added shortly before the start of the recordings (H). We recorded the group using a multi-microphone array constituted of up to three ARUs, each with two microphones on 3 m cables (Song Meter 4 and SMM-A2 microphones, Wildlife Acoustics, USA). The ARUs were programmed to record for several consecutive hours each morning from January 2020 onwards.

The microphones were spaced across the aviary so the distance between microphone and bird was at most approximately 10 m. The resulting multi-channel recordings were digitised at 48 kHz with a 16-bit resolution. A schematic layout of the aviary, with microphone placements and the usual spots the birds stayed at during observations, can be found in the supplementary material (Fig. S1).

Among the daily recordings, eleven long sessions and a number of shorter recordings were conducted in the presence of an expert observer (KM). During these sessions, a custom Python program allowed the observer to note the identity of the bird that emitted each vocalisation (thanks to their leg rings). The program also recorded the time of each annotation, which later allowed synchronisation with the audio recordings. These on-site annotations were completed with video recordings also synchronisable to both the annotation and audio files. These videos were used to help resolve potential ambiguities (e.g. to identify which one of two birds vocalised if both were close to each other). These synchronised files, obtained throughout 2020 and 2021 (see Fig. S2 for a timeline) were used to build an annotated dataset by the same expert observer with the AudioSculpt software [START_REF] Bogaards | Sound Analysis and Processing with AudioSculpt 2[END_REF]. Each vocalisation was annotated with the start time, end time, emitter identity, and whether it was part of a sequence.

Due to the multi-microphone array and sound propagation speed, the start and end times were defined as the time when a vocalisation was perceived on any audio channel for the first and last time, respectively. A vocalisation was defined as either an uninterrupted sound or a short succession of small, similar elements that were always produced together. Despite both notes and video assistance, ambiguities sometimes remained regarding the identity of emitters. When this was the case, the emitter was noted as either "Unknown" (single emitter could not be identified), or "Multiple" (multiple emitters vocalised simultaneously). The final annotated dataset included 17,662 vocalisations in 17.4h of audio recordings. The vocalisations were not equally distributed among individuals; one male in particular vocalised much more often than all the other birds (Fig. 1). Rooks, like many birds, vocalise following two general modes, calls and song-like sequences. Since our objective is to identify individuals regardless of what vocalisations they produce, we do not separate the two classes in this study.

All EU ethical guidelines (Directive 2010/63/EU) were followed for the care of the rooks throughout the study. No experimental procedure was necessary for the data collection.

Audio representation

From the multi-channel raw audio recordings, we extracted Mel-spectrograms corresponding to 10 s of audio, using all microphones available (up to 6). The length of the clips was based on [START_REF] Grill | Two convolutional neural networks for bird detection in audio signals[END_REF], itself based on the lengths of most clips in the Bird Audio Detection Challenge dataset [START_REF] Stowell | Automatic acoustic detection of birds through deep learning: The first Bird Audio Detection challenge[END_REF]. This length was the largest value that fit in computer memory with the batch size chosen for the networks. We extracted 10 s clips, applied a pre-emphasis filter with the following equation: we computed the Short Time Fourier Transform of each clip using a 50 ms Hamming window with 75% overlap, then extracted the squared magnitude of the output. The spectrograms were then downscaled to 80 Mel-scale frequency coefficients to emphasise lower frequencies. We chose these parameters empirically based on the values that produced spectrograms with good time and frequency resolution during the annotations. This resulted in Mel-spectrograms with dimensions FxT xC, where F is the number of Mel-coefficients, T is the number of spectrogram time frames (800 in our implementation), and C is the number of microphones (6 in our implementation). Smaller clips were zero-padded at the end of each dimension.

y[t] = x[t] -x[t -1],

Network architecture

Shared front-end encoder. The front-end (illustrated in Fig. 2B) encodes information from the input Mel-spectrograms through successive convolution and pooling operations. The first layer is a Per-Channel Energy Normalisation layer (PCEN, Wang et al., 2017) implemented so that all parameters of the PCEN operation are learned during the training process, with each parameter encoded as a vector of size 80, so that each frequency band has its own value for each parameter of the PCEN equation. The rest of its architecture is based on the sparrow network proposed by [START_REF] Grill | Two convolutional neural networks for bird detection in audio signals[END_REF], variants of which were used in [START_REF] Schlüter | Bird identification from timestamped, geotagged audio recordings[END_REF] and [START_REF] Kahl | BirdNET: A deep learning solution for avian diversity monitoring[END_REF] to classify bird species with excellent results. Following [START_REF] Grill | Two convolutional neural networks for bird detection in audio signals[END_REF], we standardise each frequency band and audio channel separately with a Batch Normalisation layer between the PCEN layer and the rest of the front-end. Formally, the front-end corresponds to the following equation:

• • Female Male Vocalisation Individual 1 ••• Individual N Individual 1 ••• Individual N Conv 64, k = 3x3
h E = E(X)
where E is the front-end neural encoder, X is the input Mel-spectrogram, of dimensions (80, 800, 6), and h E is the corresponding latent encoding of X by E, of dimensions (1, 800, 512).

E has an overall receptive field size of (80, 39, 6), corresponding to the entire frequency axis, slightly under 0.5 s of time, and all audio channels.

Multi-Scale Densely-connected (MSD) module. Salient information for individual identification may be present at multiple scales from local to vocalisation-wide. Several mechanisms exist to integrate multi-scale information, but we chose a module proposed by [START_REF] Pelt | A mixed-scale dense convolutional neural network for image analysis[END_REF] for its simple design, which the authors note make it applicable to a wide variety of tasks aside from their original task in medical imaging. This module is built by stacking M successive blocks, each with L parallel dilated convolutional layers with identical parameters except for the dilation rate d. We build the module with M = 2 blocks and L = 3 convolutions per block, using dilation rates d of 1, 2, and 4 (Fig. 2E), although we did not extensively experiment with these settings. All convolutions use the same parameters except for this dilation rate: 64 filters, 1x3 filter size. The module is placed directly after the front-end. Formally, this corresponds to the equation:

h M = MSD(h E )
where MSD is the module and h M is the latent encoding of h E by MSD, of dimensions (1, 800,

512).

Task-specific heads. We implemented the multi-task learning objective by defining heads with dedicated architecture for each classification task. All heads received the output of either the front-end or the MSD module. The detection, sexing and identification head shared the same architecture (Fig. 2C) including a pointwise convolution layer, a spatial dropout layer, a bidirectional LSTM layer, and a final pointwise convolution. All three heads thus preserved time resolution. The presence head summarised over the time axis using three convolutions with kernel size 1x9 and stride 1x5, followed by a convolution summarising along the remaining time axis information, and finally a pointwise convolution. All heads end with a sigmoid activation for the specific classification task. Formally, this corresponds to: Fusing decisions. We finally investigated whether explicitly combining the outputs of different heads could facilitate learning. For instance, combining detection and identification may allow the latter to only attend to frames containing vocalisations, as frames with low detection outputs are unlikely to contain vocalisations. This combination operation was always added after the sigmoid at the end of each head. We tested two operations: element-wise minimum and element-wise product. In our experiments, we only fused outputs with shapes naturally compatible with element-wise operations: 1) detection, presence and identification, and 2) detection and sexing (Fig. 2A).

ŷtask = H task (h X ), task ∈ {det,

Implementation details

In this section we detail the implementation of the neural network. We first describe nonstandard elements used in the architecture, followed by the regularisation strategies used to prevent overfitting, and finally the loss function and the optimizer algorithm used to actually train the network.

Non-standard architecture elements. The field of deep learning has been extremely active in recent years, with both new applications and new architecture elements regularly proposed to improve previous results. We selected some of these elements for their theoretical benefits on performance, briefly described below, and included them in the final system after experimentally validating these benefits. As these elements are not part of our contributions, we do not focus on them further; however, in Table S2 we show how using these non-standard elements improved the system compared to their standard counterparts. First, we used the Mish activation function instead of the ReLU family of activations, for its continuous derivability and self-regularising property, that prevent gradient vanishing and thereby improve training [START_REF] Misra | Mish: A Self Regularized Non-Monotonic Activation Function[END_REF]. Second, we used Batch Renormalisation [START_REF] Ioffe | Batch Renormalization: Towards reducing minibatch dependence in batchnormalized models[END_REF] instead of Batch Normalisation.

Batch Normalisation improves training efficiency and adds robustness to specific initialisation quirks [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF], but suffers when batches are not independent and identically distributed, which Batch Renormalisation aims to correct. Third, we used strided convolutional pooling [START_REF] Pankajakshan | All-conv net for bird activity detection: Significance of learned pooling[END_REF] instead of max pooling. Max pooling essentially discards the majority of the data when downsampling since only the highest value for each pooling operation is kept. We expected this may learning, as individual identity may be encoded by fine-grained cues. Therefore, we chose strided convolutional pooling to avoid discarding any information during training.

Regularisation. Regularisation prevents the network from overfitting the training data, which risks losing generalisation ability to unseen data. We used several of these techniques in our experiments. First, we used early stopping with a waiting period of 15 epochs, to directly stop training when validation data performance no longer improved. The waiting period was based on the learning rate cycle (see below), to allow at least two minima of the learning rate to occur before stopping. Second, we used spatial dropout with a rate of 20% before the final layer of each head. Spatial dropout prevents overfitting by randomly dropping entire feature maps at each iteration of training, which prevents feature co-adaptation that could lead to overfitting [START_REF] Srivastava | Dropout: A simple way to prevent neural networks from overfitting[END_REF]. Third, we applied L2-norm weight decay with a strength of 10 -4 .

This penalises large absolute values of the weights that can destabilise training [START_REF] Smith | A Disciplined Approach To Neural Network Hyper-Parameters: Part 1 -Learning Rate, Batch Size, Momentum, and Weight Decay[END_REF].

Fourth, we used label smoothing so that the actual training labels used to minimise the loss functions were not 0 and 1 but 0.05 and 0.95, respectively. Label smoothing similarly encourages lower weights for stable training, particularly in the last layer of the network [START_REF] Szegedy | Rethinking the Inception Architecture for Computer Vision[END_REF].

Objective function and optimizer. All networks were trained to minimise a focal binary cross-entropy loss (Lin et al., 2020, FL, ) with the formula:

FL(y, ŷ) = -y • (1 -ŷ) γ • log( ŷ) -(1 -y) • ŷγ • log(1 -ŷ)
where y is a ground truth label, ŷ is the corresponding network prediction, and γ is a focusing parameter, which reduces the contribution of correctly classified samples to the loss. When γ = 0, FL reduces to the standard binary cross-entropy. We chose γ = 2 in our experiments as it improved over the standard binary cross-entropy (see Table S2 for details). The mean of the FL over all samples was used for each head. In the multi-task configurations, we used the unweighted sum of the loss corresponding to all four tasks:

FL multi-task = ∑ task∈{det,sex,pres,id} FL task (y task , ŷtask )
We used the Ranger21 optimiser [START_REF] Wright | Ranger21: a synergistic deep learning optimizer[END_REF], with weight decay 10 -4 .

Ranger21 serves to solve some issues with more usual optimisers such as Adam, which can be unstable at the beginning of the training process and so derail the training procedure, along with integrating a number of heuristics which improve various aspects of training (see Table S2 for a comparison between Ranger21 and Adam). We chose a cyclical learning rate schedule [START_REF] Smith | A Disciplined Approach To Neural Network Hyper-Parameters: Part 1 -Learning Rate, Batch Size, Momentum, and Weight Decay[END_REF], linearly interpolating between 10 -6 and 10 -3 with cycles of 6 epochs, and halved the maximum learning rate after each cycle. Cyclical learning rates can greatly accelerate convergence by allowing the network to update much faster when the learning rate grows, and converge to more precise minima of the loss when the learning rate decreases. Finally, we used mixed precision training [START_REF] Narang | Mixed precision training[END_REF] to accelerate training and decrease memory footprint.

Training procedure

The networks were trained for a maximum of 50 epochs to minimise the FL on the Melspectrograms of the multichannel recordings. We semi-randomly distributed the annotated sessions at the start of the experiments. We adjusted the distribution in the training data so all individuals were represented as equally as possible, although this also means not all individuals could be represented in the validation and test data (Fig. 1). The annotation files were not split during either training or evaluation, to avoid data leakage due to the way vocalisations were sampled during training.

For the training data, clips were randomly sampled around each vocalisation so as to include this vocalisation in its entirety (although other vocalisations in the same clip could be cut off).

This meant the network was unlikely to ever receive the same spectrogram twice, encouraging it to learn the underlying structure of the data. Additional clips were randomly sampled from background-only parts of the recordings to increase the variety of background noises. For the validation and the test data, the recordings were simply split into adjacent, non-overlapping clips and sent in order to the network.

Vocalisations from the Unknown or Multiple classes were excluded from training by assigning them a weight of 0 during loss computation. The detection task was exempt from this, as identity is irrelevant to whether or not a vocalisation occurs.

Finally, we used the same random seed to initialise all runs for fair comparison. This meant all networks started from the same initial weights, and the training data was always fed to the network in the same order at a given epoch, so all differences between networks should be due to differences in architecture.

Experiments

We investigated and compared several configurations to build the final network, following the description in section 2.3. For clarity, all configurations were named using the nomenclature in Table 1. First, single-task networks were trained and evaluated to form a baseline. For each sub-task, one network was trained without and with the MSD module. Then, we trained the multi-task network, using the best configurations of each sub-task from the single-task exper- iment. Following this, we included either of the two fusion operations described above in the multi-task network. As a final test, we investigated the impact of using multi-channel audio compared to using only one channel. We hypothesised that multi-channel audio is more efficient due to including additional information that the network may access, such as delays between channels or simple differences in distance: while with all six microphones the maximum distance between bird and microphone was only about 10 m, with one channel distance to microphone it might increase up to about 26 m. The configuration corresponding to this case of using a single audio channel was named the Rookognise RC (Random Channel) configuration.

Evaluation

We ranked our models using the Area Under the Receiver Operating Curve (AUROC) and the Area Under the Precision-Recall Curve (AUPRC). AUROC and AUPRC both summarise the performance of a network over its entire output range. AUROC indicates the probability a classifier will score a random positive sample higher than a random negative sample. No matter the dataset, random guessing leads to 50% AUROC and perfect classification to 100% AUROC [START_REF] Fawcett | An introduction to ROC analysis[END_REF]. However, AUROC can over-emphasise the correct rejection of negative sam-ples, especially if negative samples outnumber positive samples, in which case AUPRC can be more representative [START_REF] Saito | The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets[END_REF]. AUPRC is based on precision (the ratio of true positives over predicted positives) and recall (the ratio of true positives over labelled positives), and evaluates the ability of the classifier to correctly classify positive samples without false positives or negatives, and ignores true negatives (i.e. correct rejections). Unlike AU-ROC, AUPRC is not independent of the proportion of positive samples in the dataset, and so cannot be compared between different datasets. During our experiments, we saved the models at the epoch with the highest AUPRC, as computed on the validation dataset. In the multi-task configurations, only the identification task validation AUPRC was used. The reported values of AUROC and AUPRC were the average of the AUROC and AUPRC associated with each class. The average was computed from the classes present in a given dataset (i.e., only 13 of the 15 birds were present in the test dataset, so the corresponding results were averaged over 13 classes instead of 15).

The ROC and PRC curves represent the performance of the network at various decision thresholds. The AUROC and AUPRC metrics integrate over all possible threshold values; as such, while they are convenient to evaluate network performance, they are not sufficient when we are interested in selecting a threshold when we apply the network to actual unseen data. To do so, we used the curves themselves. Since our dataset is heavily imbalanced, we used the AUPRC curve, and we chose to directly plot precision and recall as a function of the threshold value for clearer visualisation.

Results

We first built the Rookognise system by comparing single-task networks with and without the MSD module, before combining the best configurations into a single multi-task network. We then compared the multi-task learning-only approach to the approaches incorporating output fusions, to arrive at the final system. We briefly investigated the benefit of using the multi-channel audio dataset compared to a mono-channel version of the dataset. Finally, we illustrated the use of the Rookognise system on unseen data from the test dataset.

Integrating multi-scale information with the MSD module

In this section we investigated the benefits of multi-scale information integrated by the MSD module. We established the baseline performance of the system by training single-task networks dedicated to each of the four sub-tasks, with and without the MSD module (Table 2). In the following, we report results on the test data: positive numbers where the MSD module improved performance, and negative numbers otherwise. For the detection sub-task, ST Det outperformed ST Det MSD (-7.34% AUROC, -3.23% AUPRC). For the sexing sub-task, ST Sex outperformed ST Sex MSD (-0.07% AUROC, -0.01% AUPRC). For the presence sub-task, ST Pres MSD outperformed ST Pres (+2.88% AUROC, +1.28% AUPRC). Finally, for the identification sub-task, ST Id MSD outperformed ST Id (+0.94% AUROC, +3.22% AUPRC).

Overall, the MSD module improved performance for the tasks involving more complex classification (in this case, with higher number of possible outputs). Though the improvement was not systematic for all the sub-tasks, the integration of multi-scale information in the MSD module likely improved the ability of the network to disentangle important information at both local and global scales to identify specific individuals. Overall, the MSD module efficiently captured multi-scale time-frequency information in audio signals for complex tasks. On the other hand, the front-end encoder alone was sufficient to detect rook vocalisations against all other sounds, and to classify rook vocalisations by sex, likely simpler tasks to learn.

Multi-task learning and fusing sub-task decisions

In this section, we investigated the impact of multi-task learning. We compare the best singletask networks determined in section 3.1 with the multi-task configurations MT None, MT Product, and MT Minimum (Table 2, middle part). In this case, we report positive numbers where the MT configurations outperformed the ST networks, and negative numbers otherwise. The MT None configuration already outperformed the single-task configurations on the presence (+1.47% AUROC, +8.34% AUPRC) and identification (+0.69% AUROC, +2.36% AUPRC) sub-tasks) sub-tasks, though not on the detection sub-task (-4.57% AUROC, -4.69% AURPC).

Adding either fusion operation maintained or even amplified the improved performance on presence and identification, and also improved detection performance. In particular, MT Product obtained greatly improved performance on all three sub-tasks compared to any other configuration. Interestingly, none of the MT configurations outperformed the single-task networks on the sexing sub-task, although MT Product was only slightly worse (-0.67% AUROC, -0.77% AUPRC). Overall, the multi-task approach greatly improved learning ability on the more complex tasks. Fusing the decisions of multiple sub-tasks further boosted performance, especially in the detection sub-task which did not benefit from the multi-task approach alone. In particular, using the element-wise product resulted in the strongest overall performance. Following these experiments, we selected the MT Product configuration as the final system, which hereafter corresponds to the "Rookognise" system.

The impact of multi-channel audio

Using multi-channel audio includes a lot more information than using only one audio channel.

As just one example, vocalisations clearly audible on one channel may not be audible on another (Fig. 3). In a final experiment, we investigated to what extent the Rookognise system may benefit from the multi-channel aspect of the training data. To do so, we simply re-trained the system in the Rookognise configuration, using a single-channel version of the dataset ob- 2, last line). This configuration obtained lower performance than the Rookognise system on all sub-tasks (detection: -4.59% AUROC, -7.53% AUPRC; sexing:

-2.94% AUROC, -7.30% AUPRC; presence: -3.89% AUROC, -4.94% AUPRC; identification:

-2.41% AUROC, -3.7% AUPRC). This shows the ability of the network to efficiently capture additional (e.g. spatial) information as provided by the multi-channel microphone array, in particular for the detection of rook vocalisations. Nevertheless, Rookognise RC still obtained reasonable performance despite using a single audio channel, making it usable if storage space for the audio recordings is a concern.

Detecting vocalisations with the Rookognise system

We examined in further detail the performance of the Rookognise system on unseen data, starting with the detection of rook vocalisations from raw recordings. We computed the precision (ratio of true positives to predicted positives) and recall (ratio of true positives to labelled positives) metrics, as well as the proportion of the recording data retained as a function of threshold value (between 0 and 1, corresponding to the output range of the system) for each sub-task (Fig. 4). For the detection sub-task (Fig. 4, upper left), a low threshold of 0.05 was already enough to discard approximately 75% of the data, i.e. most background noise could be easily eliminated. This low threshold missed very few vocalisations (95.0% recall) but also falsely classified some noise as rook vocalisations (35.9% precision). Conversely, a higher threshold of 0.49, computed to optimise the F1-score (the harmonic mean of precision and recall), had much better precision (90.1%) and retained only 8.5% of the data, at the cost of lower recall (79.2%). A good threshold value would of course depend on the user, but optimal thresholds may lie between those two values to balance human effort reduction (with a high threshold)

and false detections (with a low threshold).

Identifying emitters with the Rookognise system

Table 3: Confusion matrix computed from the test data predictions. The predictions were restricted to labelled vocalisations, with overlapping and non-attributed vocalisations removed. Birds are coloured by sex (males in blue, females in red). Columns correspond to labels, rows to predictions. For instance: for the cell in the first row (A), second column (B) the network predicted 2 frames as produced by A that were actually produced by B. Diagonal cells (in grey) correspond to correct identifications. For each bird, the second-to-last column indicates predicted frames, the last column indicates precision, the second-to-last row indicates labelled frames, and the last row indicates recall. The four bottom-right cells contain: total frames (top left), average recall (bottom left), average precision (top right) and overall accuracy (bottom right). Individuals that were absent from the test dataset (F and R) had their recall and precision left blank (note that recall is undefined and precision is 0 in the absence of positive samples) and did not contribute to the averages. We finally directly evaluated how well the network identified individual rooks. For the sake of this evaluation, we used only predictions corresponding to vocalisations from identified individuals in the test labels (i.e. we discarded all Unknown or Multiple class vocalisations, as well as overlapping vocalisations). Each vocalisation frame was then attributed to the individual with the highest predicted score. We computed the confusion matrix from the resulting predictions, as well as the corresponding precision and recall associated with each individual (Table 3). The Rookognise system had strong overall performance identifying individuals, attributing 70.6% of vocalisation frames to the correct individual, with an average precision of 65.2% and average recall of 64.7%. Seven individuals in particular were identified with both precision and recall over 75%. Several individuals (O, P, and E) were more poorly identified. While these individuals were among the less represented in the data (e.g. O has 428 vocalisations in the training dataset), other individuals (e.g. S with 239 vocalisations and J with 125 vocalisations) were much better identified (S: 89.5% precision and 78.5% recall; J: 84.2% precision and 76.8% recall). In general, we found no correlation between identification performance (whether measured as precision or recall) and number of vocalisations in either the training or test data (four Pearson correlation tests: either precision or recall, correlated with number of vocalisations in either training or test data, all p > 0.05). We further examined two noteworthy cases.

A B E F G H J K L M O P R S T
The first case was the misattribution of some of K's vocalisations to H (Table 3, 6th row, 8th column), which was almost exclusively due to the misidentification of one long song-like bout produced by K. These two males, especially H, were also the ones who produce the longest song-like bouts.

The second case was the misattribution of almost all of O's vocalisations to M (Table 3, 9th row, 10th column), resulting in O being by far the worst-identified individual in the group at only 3.9% precision and 6.6% recall. These two males shared several vocalisation types produced by not other individual, and to our ear, they sounded very similar in general. However, this was also the case for a pair of females, J and L, but these females were successfully identified by the network (J: 84.2% precision, 76.8% recall; J: 75.2% precision, 89.9% recall).

Post-processing decisions: an example on the test data

In this final section we combined the decisions determined in the previous sections, along with some simple post-processing steps which can improve the accuracy of the Rookognise predictions. We then separately compared their effects on the test data labels. First, we used the detection threshold of 0.05. Second, following [START_REF] Cohen | Automated annotation of birdsong with a neural network that segments spectrograms[END_REF], we discard short predictions that are likely false positives; we set the threshold at 5 successive frames, corresponding to 3. The black line represents the point where the ratio of highest prediction to second-highest prediction is 2. Identification predictions are only retained if the highest prediction sufficiently higher, meaning the network is confident in identifying one individual.

is the first approach that can acoustically identify individuals regardless of the vocalisation produced. We approach the task by combining three different mechanisms: the multi-scale densely-connected MSD module, the multi-task approach where we divide the complex overall task into a cascade of increasingly complex tasks, and the explicit combination of outputs during training. All three mechanisms had substantial benefits to generalisation performance on unseen data. Multi-task learning has been proposed as acting as a regularisation mechanism [START_REF] Liebel | Auxiliary Tasks in Multi-task Learning[END_REF], although naive multi-task approaches may impede learning due to conflicting objectives. This may have caused performance loss on the detection and sexing subtasks for the MT N one configuration. In addition to the regularising effect of multi-task learning, the auxiliary tasks allow users to separately determine task-specific thresholds for more direct control over the precision and recall trade-off. Furthermore, conditioning the outputs of more complex tasks on the outputs of simple tasks may help the learning process, possibly by acting as an attention or a hierarchical classification [START_REF] Nolasco | Rank-Based Loss for Learning Hierarchical Representations[END_REF] mechanism: in this case the detection and presence tasks allow early rejection of, respectively, non-vocalisations or individuals absent from the audio clip.

The final system had strong overall performance according to all metrics used. Unfortunately, there are as yet few comparable approaches to acoustically identify individuals irrespective of vocalisation types (although there have been approaches to identify individuals by image data in multiple species, including birds). [START_REF] Stowell | Automatic acoustic identification of individual animals: Improving generalisation across species and recording conditions[END_REF] reported highly successful individual identification in three bird species, the little owl (Athene noctua), the chiffchaff (Phylloscopus collybita), and the tree pitpit (Anthus trivialis). They used a Random Forest classifier attributing an entire audio file to one individual, with high AUROC (85-94%, slightly lower than Rookognise). However, their metric was computed as one decision per file and their dataset was by design restricted to one focal individual per file, so the comparison with our system is by necessity rather limited. Comparing different systems is generally an issue in deep learning, as many tasks lack a standardised benchmark with a well-defined dataset and associated metrics, to allow different approaches to be compared rigorously. For individual acoustic identification, this issue is even more prevalent due to the current absence of similar studies [START_REF] Stowell | Computational bioacoustics with deep learning: a review and roadmap[END_REF]. Nevertheless, Rookognise appears well suited to monitoring a social group in less controlled conditions, i.e. outdoors with multiple individuals in proximity.

The strong performance of the system was diminished in a few cases. From examining predictions on the test data, these errors consisted mostly of: 1) boundary errors, most often due to the network slightly overestimating the times spanned by vocalisations; 2) some false positives due to the acoustic properties of specific percussive noises, which share similar structure to some very short vocalisations produced by some of the females, and 3) identification errors, mostly confusions between specific individual pairs (e.g. K with H, O with M). In both cases, we could identify the source of the misattribution. In the K/H case, a long vocalisation bout produced by K was attributed to H. The system may have learned to preferentially attribute long song-like sequences to H, because he is the one with longest bouts. Other cases of identification errors may come from vocal similarity between the individuals in the study group. In the O/M case, for example, O and M shared their most common vocalisation, which accounted for a majority of O's vocalisation in the test data. Note that the system did not otherwise identify individuals proportionally to their dataset presence (i.e. individuals that vocalised less were not less correctly identified).

In many bird species, social proximity and vocal similarity are positively correlated [START_REF] Brown | The Role of Song and Vocal Imitation among Common Crows (Corvus brachyrhynchos)[END_REF][START_REF] Grießmann | Song Sharing in Neighboring and Non-Neighboring Thrush Nightingales (Luscinia luscinia) and its Implications for Communication[END_REF][START_REF] Hausberger | Song Sharing Reflects the Social Organization in a Captive Group of European Starlings (Sturnus vulgaris)[END_REF], but in our case the misidentified pairs did not correspond to a particularly close social bonds [START_REF] Boucherie | Bonding beyond the pair in a monogamous bird: Impact on social structure in adult rooks (Corvus frugilegus)[END_REF], while individuals involved in other bonds were not confused by the network (Table S.1). Individuals from similar genetic or cultural origins (same colonies) may also exhibit vocal similarities [START_REF] Lemasson | Social learning of vocal structure in a nonhuman primate ?[END_REF]. In our group, two individuals (E and J) were genetically related, but they were not substantially confused by the network (Table S1). Even if some birds may share vocal similarities due to these factors (social, genetic or cultural proximity), the system still correctly classified identities.

At present, the Rookognise system necessitates knowing the individuals present in the study group to build the network and actively identify each individual. Unknown individuals cannot be identified by the system, but the system is designed to filter these unknown individuals out if desired: vocalisations from unknown individual rooks can be detected by the system (i.e.

receive high scores from the detection sub-task) but without being attributed to an individual (i.e. receive low scores from the identification sub-task). As such, the Rookognise system can already be used as is to detect vocalisations from another group of rooks, which can greatly facilitate its deployment to other groups, i.e. only requiring to update the individual identification sub-task while other sub-tasks are already trained. More generally, we believe that the network can be deployed to monitor groups of any other species: birds, primates, cetaceans, provided that the number and identity of individuals in the group are known, and, of course, that the species produces vocalisations as part of its communication. More than a proof of concept, the system should be adaptable to any species no matter how harmonic their vocalisations are. Indeed, the system is already successful on rooks, birds that are characterised by their harsh, inharmonic calls like many corvids [START_REF] Fletcher | A class of chaotic bird calls?[END_REF]. Neural networks often generalise well, although the risk of overfitting to a particular problem is always present. For instance, [START_REF] Stowell | Automatic acoustic identification of individual animals: Improving generalisation across species and recording conditions[END_REF] pointed out that a classifier trained to recognise one particular individual associated with a particular background noise, may fail to recognise the same individual in a different background noise. Our system was designed to limit this by explicitly training on background noise samples. In addition, the dataset itself, due to the complexity of the background noises recorded in an outdoors setting, likely limits the potential for overfitting on this type of data. Finally, none of the decisions involved in the design of the system were specifically tailored to the species or the group. The main limit to deploying the system to the study of another species is that said species must vocalise frequently enough to collect a sufficient amount of vocalisations in a reasonable time.

Extending the system to conditions where the number of individuals is not known a priori requires reformulating the identification task as an open-set recognition problem, a much more technically difficult task. Previous approaches to this problem have largely been in the form of speaker verification [START_REF] Ntalampiras | Acoustic detection of unknown bird species and individuals[END_REF][START_REF] Ptacek | Automatic recognition of bird individuals on an open set using as-is recordings[END_REF][START_REF] Stowell | Automatic acoustic identification of individual animals: Improving generalisation across species and recording conditions[END_REF] or clustering approaches including deep learning-based ones [START_REF] Goffinet | Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires[END_REF][START_REF] Kirschel | Using songs to identify individual mexican antthrush Formicarius moniliger: Comparison of four classification methods[END_REF][START_REF] Pagliarini | What does the Canary Say? Low-Dimensional GAN Applied to Birdsong[END_REF] to determine whether vocalisations are produced by a previously-known individual or by a potentially new one. However, neither these approaches, nor our system, can discriminate 1) between two unknown individuals, or 2) between two different vocalisations produced by the same unknown individual.

Finally, the multi-task design makes the system easily extendable to other tasks. Any behavioural information that can be classified can be incorporated as an additional sub-task; for instance, the acoustic type of the vocalisation [START_REF] Bermant | Deep Machine Learning Techniques for the Detection and Classification of Sperm Whale Bioacoustics[END_REF][START_REF] Cohen | Automated annotation of birdsong with a neural network that segments spectrograms[END_REF][START_REF] Oikarinen | Deep convolutional network for animal sound classification and source attribution using dual audio recordings[END_REF], its behavioural context, or its emotional valence [START_REF] Briefer | Vocal expression of emotions in mammals: Mechanisms of production and evidence[END_REF][START_REF] Briefer | Classification of pig calls produced from birth to slaughter according to their emotional valence and context of production[END_REF]Marler and Evans, 1996. Briefer et al. (n.d.) recently proposed a system that could classify the emotional state of pigs by their vocalisations. The Rookognise system should be able to process these other informations as additional sub-tasks, which could have great implications, for instance for the assessment of welfare and management of captive animals [START_REF] Laurijs | Vocalisations in farm animals: A step towards positive welfare assessment[END_REF]. Finally, the system could be deployed in the wild to study the behaviours of any stable social group, so long as their identities are established.

Conclusions

In this paper, we proposed the Rookognise system as the first vocalisation-independent acoustic individual identification approach applicable to raw recordings, along with a large annotated dataset of rook vocalisation. The Rookognise system is designed to identify individual animals regardless of the vocal diversity exhibited at any level from the individual to the species. We leveraged several different mechanisms to help ensure good robustness for the problem at hand of automatic acoustic individual identification as well as great flexibility for the system to be deployed in other cases, including with additional or different tasks. The system leverages multi-task learning to improve generalisation performance on unseen data. This has two other consequences for users: first, the multiple outputs can allow separate interpretation and facilitate the extraction of particular vocalisations if desired. Second, the system is designed to be easily extendable to other tasks by simply adding or replacing heads corresponding to the task at hand. Our implementation also uses multi-channel audio to capture additional information to improve performance, although single-channel audio still provides acceptable performance Table S2: The non-standard architecture designs elements included in Rookognise increase its performance. We report the effects of incorporating one of each of the non-standard architecture design elements described in Section 2.4. Each element is tested by comparing the full Rookgonise system with all the non-standard elements included, a baseline with the same architecture but all the elements replaced by their standard counterparts, and a network with the same architecture, but with one non-standard element. The training and evaluation of all the networks reported here followed the same protocol as in the main text. The baseline elements were all based on the original sparrow network proposed by [START_REF] Grill | Two convolutional neural networks for bird detection in audio signals[END_REF]: LeakyReLU activation function with slope coefficient α = 0.01 (replaced by Mish), Batch Normalisation (replaced by Batch Renormalisation, Renorm), Max pooling (replaced by strided convolutional pooling, Conv), Binary cross-entropy loss (replaced by the Focal loss), and Adam optimiser (replaced by Ranger21). Each respective network is denoted by Baseline + X, where X is the non-standard element included in that network. Values in bold are above the baseline, while underlined values are above the full Rookognise system. Note that the Ranger21 optimiser and the Focal loss yielded the largest improvements of the single elements. Nonetheless, the Rookognise generally performed the best in general, demonstrating that the non standard elements had non-redundant effects on performance.

Figure 1 :

 1 Figure 1: Number of vocalisations per bird and dataset, in decreasing order based on the training dataset. The numbers next to the bars represent the exact number of vocalisations produced by each bird in this dataset. Bar colours represent the sex of the individual. Multiple and Unknown represent cases where vocalisations could not be attributed to an individual bird.

Figure 2 :

 2 Figure 2: The Rookognise system. A) Overview of the system. Elements in light blue (the MSD module and the two fusion operation, element-wise product and element-wise minimum) are optionally included at training time. On the right: example clip containing three vocalisations, two from a female and one from a male. Under the Mel-spectrogram are the corresponding ground truth labels (in the case of the presence output, stretched in time to match the shape of the other outputs). B) Front-end encoder detail. C) Head detail for the detection, identification and sexing tasks. D) Head detail for the presence task. E) MSD module detail. B-E: boxes represent one layer, with activation layers and Batch Renormalisation layers omitted for readability. Convolution layers (Conv) used the parameters in each box in the following manner, taking for instance the first grey Conv box in B: Conv 64, k=3x1, s=3x1 means a convolution with 64 filters of size 3x1 (3 along the frequency axis, 1 along the time axis) and stride 3x1 (likewise). Boxes coloured according to operation type (green: normalisation, white: convolution, grey: pooling, blue: LSTM, red: dropout, yellow: concatenation). N: number of classes for a particular task (detection: 1, sexing: 2, presence and identification: 15).

Figure 3 :

 3 Figure 3: Schematic illustration of the multi-channel setup. Here, a rook is vocalising closest to the rightmost microphone (left), corresponding to the fourth channel from the top in the audio recording (middle) and in the corresponding Mel-spectrograms (right). On the other hand, the leftmost microphone, furthest from the rook, barely records anything over the background noise

Figure 4 :

 4 Figure4: Test performance of the Rookognise system for each sub-task. Performance is measured using precision, recall and proportion of spectrogram frames kept (as a proxy of human effort reduction). For the sexing, presence, and identification sub-tasks the precision and recall curves are averaged over classes and the shaded area represents one standard deviation away from this average. In all plots, the black vertical lines corresponds to specific threshold values: dashed lines correspond to the conservative threshold at 0.05, dotted lines to the threshold optimising F1-score (the harmonic mean of recall and precision). Coloured horizontal lines are the values of all three metrics at the corresponding thresholds, with the associated values in the corresponding colours.

Figure 5 :

 5 Figure 5: Post-processing of the system predictions. A) Distribution of the Rookognise detection sub-task predictions (orange bars) and of the labelled vocalisations (blue bars) by length in frames. The black line represents the threshold, chosen at 5 consecutive frames. Given the distribution of lengths in the labels, Rookognise predictions shorter than 5 frames are likely false positives and can be safely removed. Only detections (for both labelled and predicted vocalisations) up to 20 frames long are included to avoid crowding the plot. B) Accuracy and proportion of frames retained as a function of the ratio between the highest and second-highest prediction for the identification task. Both measures are computed based on the frames corresponding to labelled vocalisations, as in Table3. The black line represents the point where the ratio of highest prediction to second-highest prediction is 2. Identification predictions are only retained if the highest prediction sufficiently higher, meaning the network is confident in identifying one individual.

  Figure S2: Timeline of the data collection and distribution of the annotated data by period and dataset. Each line corresponds to the training, validation, or test dataset. The density lines correspond to the distribution of vocalisations over the recording period, normalized to the same maximum height. Shaded areas are periods of the year where the vocal behaviour of the rooks changed: during the breeding season (red), the males produced overall fewer vocal sequences while virtually all individuals pairs were building nests; later in the season females stayed in the nest and produced loud begging calls. During fall (blue), the same behaviours occurred at a lower intensity and without nest building.

  where x[t] is the original audio at time t and y[t] is the corresponding filter output. From the output of this filter,

	Male
	Female

  sex, pres, id} such that ŷdet is the output of the detection head H det of dimensions (1, 800, 1), ŷsex is the output of the sexing head H sex of dimensions (1, 800, 2), ŷpres is the output of the presence head H

pres of dimensions (1, 1, 15) (15 corresponding to the number of individuals in the study group), and ŷid is the output of the identification head H id of dimensions

(1, 800, 15)

. h X can be the encoded output of either the front-end h E or the MSD module h M .

Table 1 :

 1 Description and nomenclature of the configurations in the experiments. From left to right: Configuration is the name used in the text for each network, the Task learning column denotes whether the network learned a single task or used the multi-task approach, the Task column corresponds to the task(s) learned in a given configuration, the MSD module column corresponds to whether the MSD module is included or not in a given configuration, and the Combination column refers to the element-wise operation used for combining decisions in the configuration, if any.

	Configuration Task learning	Task	MSD module Combination
	ST Det	Single	Detection	No
	ST Det MSD	Single	Detection	Yes
	ST Sex	Single	Sexing	No
	ST Sex MSD	Single	Sexing	Yes
	ST Pres	Single	Presence	No
	ST Pres MSD	Single	Presence	Yes
	ST Id	Single	Identification	No
	ST Id MSD	Single	Identification	Yes
	MT None	Multi	All	Yes
	MT Product	Multi	All	Yes	Product
	MT Min	Multi	All	Yes	Minimum

Table 2 :

 2 Test data performance of the networks, at the best epoch determined by validation data AUPRC. For each column, the best performance is emphasised in bold (separately for single-task and multi-task configurations). Underlined values represent where the multi-task configurations outperformed the corresponding best single-task network. The last line corresponds to the mono-channel configuration, obtained by training the network using only one of the microphones randomised for each audio clip.

		Detection	Sexing	Presence	Identification
	Configuration	AUROC AUPRC AUROC AUPRC AUROC AUPRC AUROC AUPRC
	ST Det	94.03	81.04
	ST Det MSD	86.69	77.81
	ST Sex			98.88	89.72
	ST Sex MSD			98.81	89.71
	ST Pres					83.62	53.11
	ST Pres MSD					86.50	54.39
	ST Id							92.87	55.28
	ST Id MSD							93.81	58.50
	MT None	89.46	76.35	97.83	87.26	87.97	62.73	94.50	60.86
	MT Min	94.69	81.72	97.64	87.47	89.94	64.65	93.47	61.30
	MT Product (Rookognise)	96.95	89.31	98.21	88.95	88.03	63.64	96.11	63.06
	Rookognise RC	92.35	81.78	95.04	81.65	84.15	58.70	93.70	59.29

https://gitlab.com/kimartin/rook-vocalisation-detection

https://doi.org/10.5281/zenodo.6091940

A B

Figure 6: Two example Mel-spectrograms from the test dataset (only one audio channel shown), with corresponding network outputs for detection, sexing, and identification, shown after the post-processing detailed in the text. A) Calls produced by one female (L) and one male (A). B) Song-like bout produced by one male (H). True negatives are in white, true positives in black, false negatives in blue, and false positives in red. 0.0625 s at our time resolution (Fig. 5A). The 5 frames threshold effectively discarded 37% of false positive detections, at the cost of removing 1.5% of the labelled vocalisations. This had relatively minor effect on the detection precision (37.8% instead of 35.9% without discarding short detections) and recall (94.7% instead of 94.9% without discarding short detections), since we remove relatively few frames in total. Third, we discarded predictions for the identification if the highest predicted score is not at least twice as high as the second highest score (Fig. 5B). This effectively discarded predictions where the network was not sufficiently confident in attributing the vocalisation to an individual, and increased accuracy to 84.7% (against 70.6% without this step) while retaining 72.0% of vocalisation frames. We show two examples of system outputs after applying these steps in Fig. 6. In Fig. 6A, four calls from the female L and one call from the male A were correctly detected, sexed, and attributed to the correct individual. In Fig. 6B, we illustrated a clip containing a long song-like bout produced by H. Again, the network correctly detected, sexed and attributed these vocalisations, including separating most vocalisations even if they occur in quick succession.

Discussion

We proposed Rookognise, a deep learning system for the acoustic detection and vocalisationindependent identification of rooks in an outdoor soundscape. To our knowledge, the system independent features or if different signatures exist for different vocalisation types, as has been suggested in zebra finches finches [START_REF] Elie | Zebra finches identify individuals using vocal signatures unique to each call type[END_REF].

Acknowledgements

This project was funded by a Michelin Foundation donation to V.D. (N°349-PE-3176). We are grateful to the Eurometropole of Strasbourg for granting us the use of the terrain for housing the animals. We also thank IRCAM for granting us access to the AudioSculpt software.

Conflict of interest

The authors declare no conflict of interest.