Théo Ardoin

Cédric Sueur

Automatic Identification of Stone-Handling behaviour in Japanese Macaques Using LabGym Artificial Intelligence

Keywords: Artificial Intelligence, Ethology, Primate Behaviour, Deep learning, Japanese Macaques

The latest advances in artificial intelligence technology have opened doors to the video analysis of complex behaviours. In light of this, ethologists are actively exploring the potential of these innovations to streamline the time-intensive behavioural analysis process using video data. Several tools have been developed for this purpose in primatology in the past decade. Nonetheless, each tool grapples with technical constraints. To address these limitations, we have established a comprehensive protocol designed to harness the capabilities of a cutting-edge tool, LabGym. The primary objective of this study was to evaluate LabGym's suitability for the analysis of primate behaviour, focusing on Japanese macaques as our model subjects. First, we developed a model that accurately detects Japanese macaques, allowing us to analyse their actions using LabGym. Our behavioural analysis model succeeded in recognising stone-handling-like behaviours on video. However, the absence of quantitative data within the specified timeframe limits our study's ability to draw definitive conclusions regarding the quality of the behavioural analysis. Nevertheless, as far as our knowledge extends, our model represents the pioneering endeavour in leveraging LabGym to analyse primate behaviours. It lays the groundwork for future research in this promising field to complexify our model using LabGym's latest version and associated tools such as multiclass detection and interactive behaviour analysis.

Introduction

The study of animal behaviour is essential for understanding the ecological niche of a species [START_REF] Barnard | Animal behaviour: ecology and evolution[END_REF]. Conducting an ethological study requires identifying the individuals under investigation and characterising their actions precisely. Traditionally, direct observation of wild animals in their natural habitat has been employed to collect behavioural data [START_REF] Altmann | Observational Study of Behaviour: Sampling Methods[END_REF][START_REF] Huntingford | The study of animal behaviour[END_REF]). However, this approach has limitations due to the real-time precision of observations and the disruptive effect of human presence on the behaviour of the studied species. Indeed, the degree of habituation to human presence varies considerably among species and individuals, thereby introducing numerous observation biases. Handheld cameras have revolutionised ethological field studies, which now involve recording videos for later analysis in the laboratory. This approach offers several advantages, including the possibility of delayed analysis, observation without human disturbance, and the creation of citizen science databases, which are valuable for research and conservation. Nevertheless, studying complex behaviours from video recordings remains challenging for untrained human observers. Moreover, the analysis of extensive video data is extremely time-consuming and demands substantial human resources. For instance, the detailed analysis of a video takes, on average, three times its duration [START_REF] Anderson | Toward a science of computational ethology[END_REF].

The evolution of computer technologies has ushered in new possibilities for image and video processing through artificial intelligence [START_REF] Kabra | JAABA: interactive machine learning for automatic annotation of animal behaviour[END_REF]Valletta et al. 2017;[START_REF] Chakravarty | Seek and learn: Automated identification of microevents in animal behaviour using envelopes of acceleration data and machine learning[END_REF][START_REF] Kleanthous | A survey of machine learning approaches in animal behaviour[END_REF]). The early 2010s marked a significant turning point in computing with the widespread availability of graphics processing units (GPUs), which are computer components capable of simultaneously handling multiple heavy calculations. This newfound capability enables our computers to support a sophisticated technology called Convolutional Neural Networks (CNNs) [START_REF] Gu | Recent advances in convolutional neural networks[END_REF][START_REF] De Cesarei | Do humans and deep convolutional neural networks use visual information similarly for the categorization of natural scenes?[END_REF][START_REF] Shi | MouseNet: A biologically constrained convolutional neural network model for the mouse visual cortex[END_REF]. CNNs are artificial neural networks whose architecture is inspired by the organisation of visual neurons in mammals. The depth of a CNN is determined by the number of artificial neuron layer that constitutes it, which is proportional to its capacity for complex analysis -referred to as 'network depth'. This brain-inspired structure allows CNNs to learn from training data [START_REF] Roffo | Ranking to learn and learning to rank: On the role of ranking in pattern recognition applications[END_REF]. This technological advancement led to the development of tracking softwares, such as Ethovision® [START_REF] Noldus | EthoVision: a versatile video tracking system for automation of behavioural experiments[END_REF], marking the beginning of computer-based animal behaviour analysis, also known as computational ethology. This software offers precise and objective means of studying animals by quantitatively analysing various spatiotemporal parameters that define their kinematics. As a result, it represents a substantial time-saving tool for researchers, enabling them to simultaneously collect and analyse behavioural data through computer input. Unfortunately, at that time, ordinary computers were not equipped to support CNNs with significant complexity.

Consequently, tracking software faced limitations regarding learning performance and detection quality, making them less versatile. Primarily, they were designed for studying laboratory animals against a neutral background, meeting the highest demand in controlled settings. However, due to limited learning capacity, recognising new animals or behaviours often requires manual source code updates by developers. Furthermore, analysing an animal's kinematics was not comprehensive enough to identify all behaviours. Imperfect animal detection often necessitated filming animals from above. While this angle simplified AI analysis, it inevitably resulted in a loss of information, such as the inability to quantify leg movements. These limitations made computational ethology [START_REF] Anderson | Toward a science of computational ethology[END_REF] impractical for studying wildlife.

In 2015, A technical revolution changed the landscape by introducing the first Residual Neural Network (ResNet) [START_REF] Liang | Image classification based on RESNET[END_REF]. This innovation addressed a common challenge faced by the scientific community working on deep CNNs -the vanishing gradient problem. This problem refers to the AI's difficulty in correcting its predictions, which increases exponentially as the number of layers in the artificial neural network grows. The ResNet architecture successfully overcame this issue, preventing a significant decrease in CNNs' learning capacity as their complexity increased. As a result, the first computational ethology algorithms inspired by ResNet emerged, providing versatile tools with continuously improving learning capabilities. The majority of AI developed during this period was designed for supervised training under the guidance of human experts. Consequently, from 2015 to 2020, numerous online annotation tools were developed to assist humans in image and video analysis. In the field of ethology, particularly in primatology, a notable example is the 'MacaquePose' project [START_REF] Labuguen | MacaquePose: a novel 'in the wild' macaque monkey pose dataset for markerless motion capture[END_REF]. In this project, a team of researchers used a database containing over 13,000 annotated images of macaques to train DeepLabCut AI for postural analysis [START_REF] Hardin | Using machine learning and DeepLabCut in animal behaviour[END_REF][START_REF] Lauer | Multi-animal pose estimation, identification and tracking with DeepLabCut[END_REF]. Achieving an accuracy of over 90%, this AI can determine the posture of macaques in each image extracted from a video. This high precision enables the crucial first step of behavioural analysis by monitoring macaques in their natural environment.

However, as noted by the article's authors, one significant challenge persists: the limitation of a 2dimensional view when analysing animal behaviour. The reduction to two dimensions may result in a potential loss of behavioural information. In order to address the issue, two solutions were considered. First, there is the option of using a 3-dimensional video capture approach employing multiple cameras. This method provides the most accurate recording of primate behaviour but demands substantial material resources, making it less applicable for studying primates in their natural habitat. The second approach involves enhancing two-dimensional image processing models, which is favoured for practical reasons and holds great promise for the scientific community. Many research teams are opting to integrate a new type of algorithm into their artificial intelligence architecture known as Mask R-CNN [START_REF] Bharati | Deep learning techniques - R-CNN to mask R-CNN: a survey[END_REF][START_REF] Xu | Automated cattle counting using Mask R-CNN in quadcopter vision system[END_REF][START_REF] Xu | Automatic sheep behaviour analysis using mask r-cnn[END_REF]. Currently, this represents one of the most advanced computer vision algorithms available. Notably, Mask R-CNN enables 'segmentation by instance' of the objects being analysed. This term refers to the AI's ability to precisely distinguish each instance of the same object in an image, a previously challenging task for classic CNNs. As a result, AI systems can now detect and differentiate several individuals as distinct entities, marking a significant advancement in computational ethology applied to wildlife research [START_REF] Schofield | Chimpanzee face recognition from videos in the wild using deep learning[END_REF][START_REF] Charpentier | Same father, same face: Deep learning reveals selection for signaling kinship in a wild primate[END_REF][START_REF] Rigoudy | The DeepFaune initiative: a collaborative effort toward the automatic identification of the French fauna in camera-trap images[END_REF][START_REF] Paulet | Deep Learning for Automatic Detection and Facial Recognition in Japanese Macaques: Illuminating Social Networks[END_REF]).

LabGym1.0, as introduced in reference [START_REF] Hu | LabGym: Quantification of user-defined animal behaviours using learning-based holistic assessment[END_REF], represents a significant milestone in computational ethology. This software empowers the general public to train artificial neural networks for animal identification and behaviour analysis thanks to laboratory or natural environment video footage. What sets LabGym1.0 apart from its competitors is its exceptional accessibility. First, LabGym1.0 distinguishes itself by not requiring a GPU for operation, making it cost-effective. This accessibility aligns with an inclusive approach to computational ethology, particularly relevant in fields like primatology, where many nature reserves are situated in economically developing countries. In such settings, data analysis often occurs on-site, where limited resources can be transported. By eliminating the GPU requirement, LabGym1.0 addresses this challenge. Furthermore, LabGym1.0 stands out as it is designed for users possessing limited computer skills. Its user-friendly interface eliminates the need for coding expertise, except during the initial installation process. This software choice was deliberate in our project, which aims to create a model for studying the behaviour of Japanese macaques [START_REF] Nakagawa | The Japanese macaques[END_REF]) using artificial intelligence. Our project serves a dual purpose. Firstly, it forms part of fundamental research, assessing the feasibility of developing such a model given the complexity of behavioural identification. Secondly, it aligns with a long-term objective of creating freely accessible AI tools capable of analysing individual and social behaviours of various primate species. This work can complement automatic identification models for primates, forming a comprehensive module for in-depth analysis of primate social and cultural behaviours. Indeed, studying and understanding social and cultural behaviours is essential for the conservation of primates [START_REF] Naud | Relations between Spatial Distribution, Social Affiliations and Dominance Hierarchy in a Semi-Free Mandrill Population[END_REF][START_REF] Brakes | Animal cultures matter for conservation[END_REF][START_REF] Romano | Stemming the Flow: Information, Infection, and Social Evolution[END_REF][START_REF] Sueur | Animal capital: a new way to define human-animal bond in view of global changes Valletta JJ[END_REF]. The creation of our model proceeded through three distinct stages. Initially, we trained an image analysis AI to detect Japanese macaques within the video frames. This step enabled us to acquire the precise position coordinates of each individual present in the video images. Utilising these position coordinates, we then proceeded to train an artificial neural network capable of recognising and classifying the behaviours exhibited by these detected primates. Finally, we assessed the AI's predictions to gauge its accuracy. Our project focused on analysing stone-handling behaviours, a valuable way to investigate culture within the Japanese macaque population. Our choice was motivated by the sheer diversity of stone-handling behaviours, encompassing 45 distinct behaviours. The extensive literature on these behaviours also provided a solid foundation for our research [START_REF] Leca | Japanese macaque cultures: inter-and intra-troop behavioural variability of stone handling patterns across 10 troops[END_REF][START_REF] Leca | The monkeys of stormy mountain: 60 years of primatological research on the Japanese macaques of Arashiyama[END_REF][START_REF] Huffman | Cultured Japanese macaques: a multidisciplinary approach to stone handling behaviour and its implications for the evolution of behavioural tradition in non-human primates[END_REF][START_REF] Nahallage | Stone handling, an object play behaviour in macaques: welfare and neurological health implications of a bio-culturally driven tradition[END_REF]. Our model serves as a foundational framework that can be expanded upon in future work, exploring other behaviours of interest. In the long term, we envision integrating our model with an automatic identification system for individual macaques within a group [START_REF] Paulet | Deep Learning for Automatic Detection and Facial Recognition in Japanese Macaques: Illuminating Social Networks[END_REF]. This holistic approach will enable comprehensive studies of individual and sociocultural behaviours among Japanese macaques and potentially other primate species, as well as non-primate species, broadening the scope of ethological research.

Material & Methods

Study subjects: Japanese macaques (Macaca fuscata)

The Japanese macaque holds the distinction of being the northernmost primate species [START_REF] Nakagawa | The Japanese macaques[END_REF]. Endemic to Japan, it thrives in diverse ecosystems and forms matrilineal societies. This species was among the earliest subjects of study in the field of primatology, dating back to 1950 [START_REF] Matsuzawa | Kinji Imanishi and 60 years of Japanese primatology[END_REF]. Since then, Japanese Macaques have enjoyed protected status in Japan, where they are the focus of research in reserves and in their natural habitats. The variety of environments they inhabit provides an opportunity to examine the impact of environmental factors on their behaviour. With 73 years of dedicated research, the scientific community has accumulated a wealth of data and knowledge about these primates [START_REF] Nakagawa | The Japanese macaques[END_REF]). Notably, it was among Japanese macaques that cultural transmission was first observed in a non-human animal. This term describes a scenario where young individuals learn behaviours by watching their elders, often their parents [START_REF] Whiten | The burgeoning reach of animal culture[END_REF]. This phenomenon has been the subject of longitudinal studies spanning more than three decades, leading to the identification of numerous cultural behaviours within the Japanese Macaque population. One of these behaviours, the stone-handling behaviour, has garnered increasing interest.

Study Behaviour: Stone-Handling or Stone Playing

Stone-Handling (SH) or Stone play is a solitary activity observed in various species of macaques, including the Japanese macaque [START_REF] Leca | Japanese macaque cultures: inter-and intra-troop behavioural variability of stone handling patterns across 10 troops[END_REF][START_REF] Leca | The monkeys of stormy mountain: 60 years of primatological research on the Japanese macaques of Arashiyama[END_REF][START_REF] Huffman | Cultured Japanese macaques: a multidisciplinary approach to stone handling behaviour and its implications for the evolution of behavioural tradition in non-human primates[END_REF][START_REF] Nahallage | Stone handling, an object play behaviour in macaques: welfare and neurological health implications of a bio-culturally driven tradition[END_REF]. This term encompasses a collection of 45 behaviours, each observed with varying frequencies within different groups. The frequencies of these Stone-Handling patterns within a group collectively define the 'Stone Handling culture' of that particular group. Notably, the culture of stone play is transmissible through observation. For instance, specialists in Japanese macaque behaviour have demonstrated that when a foreign male joins a group, it often brings Stone-Handling patterns from its original group into the new one, illustrating a cultural influence. Additionally, environmental factors also influence stone play in Japanese Macaques. A comprehensive comparative study examined various populations of Japanese Macaques across Japan, enabling researchers to assess the impact of environmental factors on the practice of stone play [START_REF] Leca | Japanese macaque cultures: inter-and intra-troop behavioural variability of stone handling patterns across 10 troops[END_REF][START_REF] Leca | Food provisioning and stone handling tradition in Japanese macaques: a comparative study of ten troops[END_REF][START_REF] Leca | Complexity in object manipulation by Japanese macaques (Macaca fuscata): A cross-sectional analysis of manual coordination in stone handling patterns[END_REF]. Among the climatic factors studied, researchers found a positive correlation between Stone-Handling factors like sunshine and high temperatures. Furthermore, stone play appears primarily in populations of macaques that humans feed. According to a commonly held hypothesis, when macaques are given food, they tend to spend less time foraging, which frees up their time for leisure activities. This surplus free time is often used for exploring their environment and engaging in self-entertaining activities such as Stone-Handling.

In some groups, Stone Handling has been observed for over three decades. Beyond its recreational aspect, stone handling may play a significant role in the development and maintenance of the sensorimotor cortex of macaques [START_REF] Nahallage | Stone handling, an object play behaviour in macaques: welfare and neurological health implications of a bio-culturally driven tradition[END_REF]). As a result, this behaviour may offer an evolutionary advantage. Stone Handling has become a part of the daily lives of some Japanese macaques; it aligns with a popular hypothesis in primatology, suggesting that the regular handling of stones may eventually lead to their use as tools. Therefore, Stone-Handling is both a cultural phenomenon and a potential evolutionary stepping stone toward developing new tools in the Japanese macaque inventory. Studying this behaviour sheds light on the adaptations of Japanese macaques to their environment, offering insights into their individual and species-level adjustments.

Analysis Software: Labgym LabGym1.0 is an open-source software designed for both qualitative and quantitative analysis of animal behaviours. With LabGym1.0, users can create artificial neural networks (ANNs) and customise their complexity [START_REF] Enquist | Neural networks and animal behaviour[END_REF]. The complexity of an Artificial Neural Network is primarily determined by its depth (the number of layers it comprises) and its width (the size of each layer). This feature enables users to personalise settings to meet their requirements. Shallow ANNs are ideal for tasks such as image and simple behaviour analysis, resulting in faster analysis. In contrast, deep ANNs can analyse more complex behaviours but take a longer computing time.

For an AI to accurately analyse an individual's movements, it must be able to detect the individual with a high degree of confidence. In the context of LabGym1.0, it is essential to isolate the animal from its background to focus the analysis accurately. To tackle this challenge, LabGym1.0 provides users with the ability to train two types of artificial intelligence. The first type is a 'Detector,' an AI whose architecture employs Detectron2 technology [START_REF] Jabir | Accuracy and efficiency comparison of object detection open-source models[END_REF], which is image recognition software developed by Meta, the parent company of Facebook. Once trained, this AI becomes proficient at identifying the set of pixels within an image that represents an animal. Consequently, the detector can determine the animal's location within the image and its posture.

It is important to note that the LabGym1.0 version, available online until the seventh of July 2023 (the release date of LabGym2.0), is limited to detecting a single class of objects simultaneously. For instance, it can simultaneously detect ten Japanese macaques in an image because they belong to the same class (Japanese macaque). However, it cannot detect concurrently a stone and a macaque during the same analysis, as they do not belong to the same object class. Therefore, this project must focus on the analysis of a single class of objects.

The position and posture of individuals in each image that makes up a video are then processed by another artificial intelligence called a 'categoriser'. This AI utilises the information provided by the detector and analyses it to identify specific actions or behaviours. The Categoriser uses a ResNET-type architecture (Liang 2020), capable of handling complex analyses necessary for extracting spatiotemporal phenomena from a series of images. Training the Categoriser, similar to the detector, requires robust and rigorous training.

Artificial Intelligence Training

The two artificial intelligences provided by LabGym1.0 require supervised training under the guidance of a human. The training process begins with the detector, followed by training the Categoriser. In supervised training, human-validated examples are provided to the corresponding Artificial Intelligence [START_REF] Gris | Supervised and unsupervised learning technology in the study of rodent behaviour[END_REF]Valletta et al. 2017;[START_REF] Bohnslav | DeepEthogram, a machine learning pipeline for supervised behaviour classification from raw pixels[END_REF].

Training the Detector: In this initial phase, the Detector AI is trained to recognise and locate the target objects or animals within images. Human supervisors provide annotated examples where they label the positions of the objects of interest in the pictures. The AI learns from these annotated examples and uses this knowledge to detect similar objects in new, unlabelled images. This process continues until the detector can accurately and reliably detect the specified objects (Japanese Macaques). The primary goal of our detector is to identify Japanese macaques within an image, which involves precisely selecting all the pixels representing an individual macaque. The process of labelling training examples involves outlining a perimeter on the image that includes all relevant pixels. The training of a versatile Japanese macaque detector required the labelling of a large number of macaques in diverse environmental settings and positions. Annotation is a standard method to train artificial intelligence; thus, several free online tools have been developed to simplify and streamline the annotation process. The Roboflow® platform [START_REF] Lin | RoboFlow: a data-centric workflow management system for developing AI-enhanced Robots[END_REF][START_REF] Paulet | Deep Learning for Automatic Detection and Facial Recognition in Japanese Macaques: Illuminating Social Networks[END_REF]) was chosen for the project due to its compatibility with datasets used by other research teams studying Japanese macaques. Images of Japanese macaques were annotated from videos obtained from various locations in Japan, provided by Cédric Sueur and his team. The images selected for training the detector were collected at different Japanese study sites without specific differentiation. These images have a resolution of 6720x4480 pixels, captured using a Canon EOS 5D Mark IV camera. This high resolution enables us to observe intricate details of the animals, including the precise contours of individuals standing side by side. This superior visual quality is crucial for overcoming the challenges associated with annotating Japanese macaques in groups, given their thick fur. The dataset used for training the detector consisted of 1490 images, with annotations for 4149 individuals. Following augmentation using Roboflow®, our final dataset comprises 4290 images, with annotations for 11,945 individuals (examples in Figure 1). In this context, 'augmentation' refers to the process of creating new annotated images by duplicating and then modifying the images from the initial dataset (see Table 1 and Figure 2). As the first research team to apply LabGym1.0 to the study of primates, simple and achievable objectives were opted for within the given timeframe. However, the availability of video data depicting stone play behaviours was limited, which reduced the ability to train the AI on less frequently observed forms of stone handling. Therefore, the analysis was focused on behaviours for which there is a substantial amount of video data.

To train the Categoriser effectively, we utilised the 'behaviour example generator' integrated into LabGym1.0. This generator is a necessary tool for preparing training data in an appropriate format. The process is as follows: -Selecting Videos: Initially, we selected a set of videos that depict the specific behaviours we are targeting for analysis.

-Video Selection: A set of videos depicting the specific behaviours targeted for analysis was initially chosen.

-Image Sequence Determination: The minimum number of successive images required to represent the desired behaviour adequately was then determined. This step assists LabGym1.0 in segmenting the selected videos into sequences of images (animations) without sound, effectively capturing the behaviour of interest.

-Movement Detection: The image sequences are subsequently subject to analysis by the previously mentioned Detector. This analysis produces a 'movement pattern' that highlights the movements of the macaque(s) in the animation. In this pattern, the detected animals at the start of the animation are depicted in blue, with their colour gradually transitioning to red as the animation progresses (refer to Figure 3). The movement pattern specifically represents the movements interpreted by the Detector as those of a Japanese macaque in the image throughout the animation.

-Categoriser Training: The Categoriser undergoes training using all pairs of animations and their corresponding movement patterns. This comprehensive training process equips the Categoriser for a more thorough and robust analysis of the targeted behaviours.

Following this procedure, it is ensured that the Categoriser is well prepared to analyse and classify the behaviours of interest, specifically 'Rub in hands' (RH) and 'Rub and roll on the ground' (RG), based on the movement patterns generated from the video data. To illustrate each behaviour of interest, animations of 45 images are made. This medium-length format is considered representative of the Stone-Handling behaviours. Short sequences (e.g. 10 frames) would not demonstrate accurate Stone-Handling. In contrast, longer sequences (e.g. 100 frames) are more likely to contain movement unrelated to SH. For instance, we avoided selecting sequences where a macaque engages in activities such as scratching itself immediately after playing with a stone. This precaution is taken to prevent any confusion in which the AI might associate unrelated gestures with the targeted behaviour.

To generate the training animations, we used videos featuring Japanese macaques engaged in SH, primarily obtained from the monkey parks of Arashiyama and Shodoshima, where the practice of stone play has been observed for over 30 years. These videos have a more modest resolution, specifically 1920x1080 pixels, and are captured at a frame rate of 24 frames per second. Our dataset comprises 85 videos, with a combined duration of 54 minutes and 35 seconds. Following the processing of these videos, we have generated a total of 644 animations, each containing 45 frames, to illustrate the behaviour of 'Rolling the stones on the ground.' Additionally, we have created 263 animations to illustrate the behaviour of 'Rubbing the stone between the hands.'

Estimation of the Detector Accuracy

LabGym1.0 lacks an integrated tool for assessing the accuracy of the Detector's Japanese macaque detection (which is now implemented in LabGym latest version). At the time, this evaluation could only be conducted by a human analyst who meticulously reviews a substantial number of images processed by the Detector. For our evaluation, we considered a detection as accurate if it included 95% or more of the surface area of the Japanese Macaque within the silhouette drawn by the detector. If the surface area of the macaque detected fell below 95%, we referred to it as an 'under-detection'. On the other hand, if more than 5% of the silhouette drawn by LabGym1.0 included elements external to the macaque's body, we classified it as an 'over-detection'. This distinction helped us assess the precision of the detection, with under-detection indicating that the Detector may have missed some parts of the macaque and over-detection suggesting that the Detector may have included non-macaque elements within the detection area (Figure 4, Supp mat 1).

Figure 4: Image illustrating a case of underdetection of the Japanese Macaque. The posterior part of the macaque is not detected here.

Based on the advice provided by Yujia HU, developer of LabGym1.0, we conducted a manual analysis of 1,100 images. To provide non-biased estimation, 100 images were randomly extracted from 11 videos processed by our latest Detector. This approach enabled us to observe variations in the precision of the Detector across different parameters, including factors such as the macaque's position and its environment within the image. By manually analysing these images, we assessed the accuracy and consistency of the Detector's performance across a diverse range of scenarios and contexts. This thorough evaluation helped us gain valuable insights into the Detector's capabilities and identify areas where improvements are needed.

Estimation of the Categoriser Accuracy

Unlike the Detector, the Categoriser offers quantitative data about the beginning and end of each SH episode detected within a video. In the given timeframe, only one video was analysed with our most complex and trained Categoriser. Consequently, the average precision could not be calculated due to the need for more data. A qualitative analysis was performed on a single video instead of conducting a quantitative assessment across multiple analyses. Although a quantitative approach could have provided more comprehensive insights, the qualitative analysis allowed for valuable observations and insights on how our model detects Stone-Handling-like behaviours of Japanese macaques.

Computing server

Our work was conducted on a computation server equipped with 60 processors and 8 gigabytes of RAM. The specific parameters were chosen after conducting tests for training the Detector on a dataset consisting of 15 annotated images (see Figure 5). This initial task enabled the establishment of a balance between processing time and the efficient utilisation of the processors by LabGym.

Results

Computational requirements

The processing time for the same task does not increase linearly with the number of processors utilised by the AI. Instead, a plateau is observed once the threshold of 60 virtual processors is surpassed. Beyond this point, the processing time remains constant (figure 5).

Figure 5: Processing time of a task depending on the number of virtual processors integrated into the calculation server.

Detector accuracy

Across 11 videos (Figure 6), each featuring a single macaque in the image, the Detector demonstrates an average accuracy of 77.3%, with a standard deviation of 18.88%. The detection accuracy varies according to the characteristics of the video (table 2). In these cases, where the animal's body is partially outside the frame of the video, and it is filmed from an overhead perspective, significant portions of the animal go undetected, resulting in a pronounced instance of under-detection.

55% 2,4,5,6,9,10,11 In situations where the animal is entirely within the frame, and the recording is from a frontal or profile angle, with a significant colour contrast between the macaque and its surroundings, there may be slight under-detection observed, primarily affecting the tips of the animal's fingers.

91.7%

3,7,8 When the animal is fully within the frame and the immediate surroundings, such as branches or stones, closely resemble the colour of the animal, there may be instances of over-detection where the environment in close proximity to the animal is also included in the macaque's detection.

33%

Table 2: Qualitative analysis of the content of the videos analysed by the detector.

Categoriser accuracy

We focused on a qualitative analysis of the results obtained for 'RG' (figure 7) and 'RH' (figure 8) behaviours within a 34-second video (Supp mat 2). This video was analysed using the most extensively trained Categoriser available, which yielded an estimated macaque detection accuracy of 83.7%.

Figure 7 compares the probability that the ongoing action is 'Rub and roll on the ground' (RG), depicted in blue, and the actual action performed by the macaque (ground truth), annotated by the experimenter and shown in red. Notably, the AI recognises the behaviour of interest with high confidence rates between 9.4 and 30.28 seconds, aligning with the anticipated results. However, it's important to highlight that the RG behaviour is not identified by the AI until 1.48 seconds into the sequence. The 'Rub in hands' (RH) behaviour occurs between 2.42 and 9.4 seconds in the video (figure 8), as annotated by the experimenter. However, the AI assigns a low probability to this behaviour, indicating poor detection by our Categoriser.

Discussion

As the first research team to employ LabGym1.0 to analyse primate behaviours, we encountered several challenges stemming from the scarcity of available data in the literature. Based on our results, we can identify factors that may contribute to the AI's lack of precision in tasks such as macaque detection and behaviour analysis.

Throughout this study, we observed certain limitations of LabGym1.0 regarding task processing time.

Notably, the AI's performance does not improve beyond the utilisation of 60 virtual processors since it was not originally developed to leverage such a powerful computing server. Additionally, we noticed an uneven distribution of computational operations within our server. During intensive computing tasks, which could extend over several days, only 30 of the 60 processors consistently operated at full capacity, while the remaining 30 seldom exceeded 10%. Addressing this inconsistency in workload distribution may require collaboration with LabGym1.0 developers. Regarding macaque detection, the outcomes primarily depend on two critical factors: the macaque's size within the image and the colour of its immediate surroundings. When the entire macaque's body is not visible in the image, the detector's precision notably decreases, sometimes failing to detect up to half of the individual's body. Moreover, the majority of cases categorised as 'under-detection' correspond to the macaque's fingers being under-detected, likely due to their darker colour compared to the fur, which appears to confuse the detector. Additionally, the AI might interpret the environment close to the macaque as part of the macaque itself when the macaque is near branches. These observations highlight the need for further refinement and optimisation in AI performance, particularly in addressing the challenges posed by image composition variations and environmental factors affecting detection accuracy. Reporting these issues to the LabGym1.0 developers could facilitate improvements in the system.

Nevertheless, the Detector exhibits an impressive average accuracy of 91.7% under optimal conditions. This allowed us to train the Categoriser with the highest quality detection. During the Categoriser's analysis, we observed that the behavioural analysis for the 'Rub and roll on the ground' behaviour generally yielded better results than the 'Rub in hand' behaviour. In contrast, the 'Rub in hand' behaviour was consistently misdetected by the AI, even after employing very large artificial neural networks capable of deep analysis and trained with many videos. This subpar performance can be attributed to the scarcity of training data illustrating the 'Rub in hand' behaviour, as we had 644 training animations for the 'Rub and roll on the ground' behaviour compared to only 263 for the 'Rub in hand' behaviour. To advance this project, it is imperative to substantially increase the quantity of videos used for training the Categoriser. However, this means a considerable time on the field to score and collect the behaviours on videos.

While we could not evaluate the average precision of our most advanced Categoriser due to data limitations, its performance in analysing the 'Rub and roll on the ground' behaviour is deemed satisfactory. Indeed, the initial objective of this study was to develop AI capable of detecting and analysing Japanese macaque behaviours using LabGym1.0. We reached our goal and met our expectations regarding Japanese macaque detection and promising preliminary results for AI behavioural analysis. The insights gained from this project can serve as a foundational framework for our team and other researchers interested in pursuing similar work in this field.

Throughout this study, we encountered technical challenges stemming from the constraints of LabGym1.0, which was limited to analysing a single class of objects per video. However, this constraint was lifted with the release of LabGym 2.0 on 07/12/2023 [START_REF] Hu | LabGym: Quantification of user-defined animal behaviours using learning-based holistic assessment[END_REF]. Regrettably, we were unable to transition to LabGym 2.0 at that time due to resource constraints. The new version retains the same overall functionality as its predecessor but enables the simultaneous detection of multiple classes of objects and the analysis of their interactions. This development holds great promise for improving Japanese macaque detection, particularly by identifying the macaque's hands as objects, which will facilitate the analysis of Stone-handling behaviours. In a broader context, LabGym2.0 opens up opportunities for studying all behaviours involving tools in primates.

This first study serves as a vital stepping stone in the application of AI to ethology and behavioural science. It not only sheds light on the challenges and opportunities of using AI tools but also emphasises the importance of data collection, collaboration with developers, and staying updated with technological advancements. Ultimately, it contributes to the broader goal of advancing our understanding of primate behaviours and animal cognition through innovative research methodologies.

Figure 1 :

 1 Figure 1: Picture featuring two Japanese macaques, with annotations highlighted in yellow using Roboflow®. Credit: Cédric SueurNature of image augmentation Chosen parameter 90° rotation Clockwise Counterclockwise Reversed 180° Cropping Maximum zoom of 50% Rotation -35° to +35° Shades of grey Applied to 10% of images Gaussian blur Maximum 6.5 pixels Background noise (random pixel replacement) Maximum 4% Table 1: Parameters used for increasing the dataset used for training the detector.

Figure 2 :

 2 Figure 2: Example of augmentation applied to an image using Roboflow ® Training the Categoriser: Once the Detector is proficient at identifying objects, the training of the Categoriser follows. This AI analyses the information provided by the Detector to categorise and understand the actions or behaviours of the identified objects. Again, human supervisors play a crucial role in the training process by providing annotated examples of various actions or behaviours associated with the detected objects. The Categoriser learns from these examples, allowing it to categorise and interpret actions in new images. We focused on analysing two specific Stone-Handlinglike behaviours exhibited by Japanese macaques: 'Rub in hands' (RH) and 'Rub and roll on the ground' (RG). This choice is motivated by several factors.As the first research team to apply LabGym1.0 to the study of primates, simple and achievable objectives were opted for within the given timeframe. However, the availability of video data depicting stone play behaviours was limited, which reduced the ability to train the AI on less frequently observed forms of stone handling. Therefore, the analysis was focused on behaviours for which there is a substantial amount of video data.

Figure 3 :

 3 Figure 3: Illustration of an animation (left) and the associated movement pattern (right).

Figure 6 :

 6 Figure 6: estimation of Japanese macaque detection accuracy by the detector. Each video analysed is indicated from 1 to 11.

Figure 7 :

 7 Figure 7: Categoriser results in detecting 'Rub and roll on the ground' behaviour

Acknowledgements

We thank Yujia HU, the developer of LabGym1.0, for his help in applying LabGym to our study. Cédric Sueur was funded by an invited researcher fellowship from Kyoto University.

Supplementary material

Supp mat 1: Video made with the first Detector (no behavioural analysis at this stage)


Conditions de vol

La réalisation de 2 vols de 10 à 12 minutes chacun a permis de couvrir une surface d'environ 3 ha à une hauteur constante de 50m par rapport au point le plus haut de la colline et à une vitesse moyenne de 4m/s. Les 2 vols ont couvert la même surface dans des directions perpendiculaires.

Traitement des nuages de points

Correction GPS et assemblage

Les données issues de ces vols ont été corrigées des imprécisions GPS selon la procédure intégrée aux données des bornes GPS de l'IGN (disponibles gratuitement sur le site www.rgp.ign.fr) et au logiciel PosPac 1 . (Cf les 2 rapports de correction différentielle extraits du logiciel PosPac). Après cette correction, la précision du positionnement atteint moins de 1 cm en X et Y et moins de 2 cm en Z, ce qui est tout à fait satisfaisant. Les points issus des deux vols ont été combinés dans un seul fichier comprenant 4 747 931 points, soit une moyenne de 148 pts/m² avant filtrage des points sols.

Extraction des points sols

Les points sols ont été extraits en utilisant le plugin CSF2 implémenté dans CloudCompare3 avec les paramètres suivants :

-Cloth resolution : 0.5 -Max iterations : 1000 -Classification threshold : 0.3 Les points sols obtenus sont au nombre de 2 632 038, soit environ 82 pts/m² en moyenne. On constate la présence de « trous » dans la répartition des points-sols, qui sont liés à la présence de tas de bois, aux aplombs rocheux ayant créé des masques dans la couverture malgré la réalisation de vols croisés, et à certaines essences persistantes (buis). Il faudra bien garder à l'esprit l'existence de ces « trous » au moment d'interpréter les résultats.

Génération des MNT ombrés

Toujours dans CloudCompare, avec l'outil Raster, un Modèle Numérique de Terrain (MNT) d'une résolution de 25 cm a été généré en ne prenant en compte que les points sols. Ce modèle a été optimisé en suivant la procédure décrite par Volker Wichmann 4 pour un lissage avec le logiciel SagaSig 5 , dont voici les caractéristiques :

Interprétation

Nous laissons le soin aux chercheurs investis scientifiquement sur ce site de proposer des interprétations fines des images produites.

Calibrated Installation Parameters

Reference-Primary GNSS Lever Arm

Figure 2 -

 2 Figure 2-Labarre, Foix (09)-Mission du 22/02/2019 -Nuage de points total

Figure 3 -

 3 Figure 3-Labarre, Foix (09)-Mission du 22/02/2019 -Nuage de points sol

Figure 5 -

 5 Figure 5 -Labarre, Foix (09)-Mission du 22/02/2019 -Multihilshade -16 directions -35 degrés

  QC Report -report_Mission 1.pdf -02/26/2019 15:58:07

Forward

  QC Report -report_Mission 1.pdf -02/26/2019 16:01:53

Figure 4 -Labarre, Foix (09)-Mission du 22/02/2019 -MNT ombré dans Saga GIS 4 Wichmann

  Modèle de relief local (SLRM) : suppression des grandes tendances de la topographie du terrain au profit de l'accentuation des micro-reliefs (locaux)

	Avec le logiciel RVT (Relief Visualization Toolbox 6 ) développé par l'Institute of
	Anthropological and Spatial Studies du ZRC-SAZU (Slovénie), il est possible d'appliquer plusieurs traitements au MNT ainsi produit :
	 Ombrage : simulation d'un éclairage et des ombrages correspondants selon une direction et un azimuth défini par l'opérateur
	1. Grid-Filter => DTM Filter (slope based) 1.2. Terrain Slope : 30 1.1. Search Radius : 10  Ombrages cumulés depuis des directions différentes  Analyse en Composantes Principales (ACP) de l'ombrage
	2. Grid -Spline Interpolation => Multilevel B-Spline Interpolation (from Grid) 3. Grid -Filter => Multi Direction Lee Filter
	4. Close gap 5. Terrain Analysis -Lighting, Visibility => Analytical Hillshading
	5.1. Azimuth : 315 5.2. Declination : 45
	5.3. Exaggeration : 4

, V.; Conrad, O.; Jochem, A.: LiDAR Point Cloud Processing with SAGA GIS, In: Hamburger Beiträge zur Physischen Geographie und Landschaftsökologie 20, S. 81-90, https://slidex.tips/download/lidar-pointcloud-processing-with 5 http://www.saga-gis.org/en/index.html  Gradient de pente   Sky-view factor : mesure du pourcentage de ciel visible depuis chaque pixel du modèle  Degrés d'Ouverture positive et négative : met en évidence les zones convexes et les zones concaves (creux et bosses) Ces différents traitements permettent de multiplier les visions du même espace et de mieux identifier les indicateurs archéologiques. Nous livrons ci-après les visualisations les plus efficaces.

Raw Data QC Raw IMU Import QC Summary IMU data input file

  Signalons d'emblée que l'analyse détaillée de ces images ne nous semble pas révéler de micro-reliefs susceptibles de témoigner de la présence d'éventuels vestiges enfouis, ni d'aménagements anthropiques autres que ceux déjà connus antérieurement. Nous nous contenterons ici de produire des coupes topographiques utiles à ces interprétations et à souligner quelques ruptures topographiques qui semblent avoir été accentuées par l'action humaine : plateforme sommitale (donjon ?), basse-cour et fossés barrant l'éperon au Nord-Ouest et Sud-Ouest.

	General Information Project File List Rover Data Summary
	Mission Information Rover Data Files First raw data file	YS-20190222-092637.T04
	Last raw data file	YS-20190222-092637.T04
	Project name File name Start GPS week	Labarre_vol1_MSGTstation File type 2041 imu_Mission 1.dat
	Processing date YS-20190222-092637.T04 Start time IMU data check log file	2019-02-26 14:55:06 T04 Rover Data 466021.000 (2/22/2019 9:27:01 AM) imudt_Mission 1.log
	Mission date Mission duration End time IMU Records Processed Input Files Start of fine alignment Termination Status	2019-02-22 09:27:01 00:12:00.985 466742.000 (2/22/2019 9:39:02 AM) 144225 466209.800 (2/22/2019 9:30:09 AM) Normal
	Processing mode GPS Station File Name Available subsystems IMU Anomalies msgt053z.19o IMU Installation Lever Arms & Mounting Angles Rover Hardware Information Ephm0530.19g Ephm0530.19n POS Event Input Correction data L1 Satellite Lock/Elevation	IN-Fusion Single Base Primary GNSS, IMU 0 File type MSGT GNSS SingleBase GLONASS Broadcast Ephemeris GPS Broadcast Ephemeris None None
	Product Serial number Reference to IMU lever arm [m] Output Files Reference to IMU mounting angles [deg]	APX 15 AVX 210 VER0 174 S/N001 -0.013 0.108 0.000 0.000	-0.007 0.000
	IMU type Filename Reference to Primary GNSS lever arm [m]	59 File type 0.006	0.218	-0.435
	Receiver type sbet_Mission 1.out Reference to Primary GNSS lever arm std dev [m]	APX-15v3 SBET Trajectory File 0.020
	Antenna type export_Mission 1.txt Aircraft to Reference mounting angles [deg]	AV14 ASCII Export Output 0.000 0.000	0.000

POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:

07 L2 Satellite Lock/Elevation GPS L1 SNR GPS L2 SNR GLONASS L1 SNR

POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07 GLONASS L2 SNR BEIDOU SNR POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07 Roll/

Pitch Heading North/East Velocity Down Velocity Total Speed Ground Speed

	Base Station Information GNSS QC Forward/Reverse Separation GNSS-Inertial Processor Configuration
	Station ID GNSS QC Statistics Processing mode	MSGT IN-Fusion Single Base
	Filename Stabilized mount	msgt053z.19o False
	Start date Statistics Base station	2/22/2019 7:00:00 AM Min Max MSGT		Mean
	End date Baseline length [km] Processing start time	2/22/2019 11:59:59 AM 12.95 13.17 466021.015 (2/22/2019 9:27:01 AM)
	Duration Number of GPS SV Processing end time	04:59:59.000 5 466742.000 (2/22/2019 9:39:02 AM) 6 6
	Data type Number of GLONASS SV Initial attitude source	GNSS 4 Primary GNSS Track 5		5
	Receiver manufacturer, model, serial no. Number of QZSS SV IMU Sensor Context	TPS 0 Processing with Onboard IMU NET-G5 0	1294-11729 0
	Antenna manufacturer, model Number of BEIDOU SV Reference to IMU lever arm [m]	Leica 0 -0.013	0 0.108	AR25 w/LEIT Dome 0 -0.007
	Antenna height [m] Total number of SV Reference to IMU mounting angles [deg]	0.000 9 0.000	11 0.000		11 0.000
	Antenna measurement method PDOP Reference to Primary GNSS lever arm [m]	Bottom of antenna mount 1.55 2.36 0.006 0.218	1.61 -0.435
	Offset from measured point to APC [m] QC Solution Gaps Reference to Primary GNSS lever arm std dev [m]	0.15508 0.00 0.100	0.00 0.100		0.100
	Latitude Solution Type Aircraft to Reference mounting angles [deg]	N42°52'46.58258" Fixed Float 0.000 0.000		No solution 0.000
	Longitude Epoch (s)	E1°37'45.94975" 681.00 0.00		0.00
	Ellipsoidal height [m] Percentage	548.63250 100.00	0.00		0.00
	Frame	ITRF00
	Epoch	1997
	Ellipsoid	WGS84
	Velocity North [mm/y]	0
	Velocity East [mm/y]	0
	Velocity Up [mm/y]	0
	PDOP

POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07 Body Acceleration Total Body Acceleration POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07

Smoothed Solution Status Processing Mode Number of Satellites POSPac

  MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07

Baseline Length SBET IAKAR Separation Project File List Rover Data Files

	Rover Data Summary Raw Data QC L2
	First raw data file Raw IMU Import QC Summary	YS-20190222-100740.T04
	Last raw data file	YS-20190222-100740.T04
	File name Start GPS week IMU data input file	File type 2041 imu_Mission 1.dat
	YS-20190222-100740.T04 Start time IMU data check log file	T04 Rover Data 468484.000 (2/22/2019 10:08:04 AM) imudt_Mission 1.log
	End time IMU Records Processed Input Files Start of fine alignment Termination Status	469177.000 (2/22/2019 10:19:37 AM) 138590 468550.600 (2/22/2019 10:09:10 AM) Normal
	File Name Available subsystems IMU Anomalies	File type Primary GNSS, IMU 0
	Ephm0530.19g POS Event Input	GLONASS Broadcast Ephemeris None
	Ephm0530.19n Correction data	GPS Broadcast Ephemeris None
	msgt053z.19o IMU Installation Lever Arms & Mounting Angles	GNSS SingleBase
	Reference to IMU lever arm [m] Output Files Reference to IMU mounting angles [deg]	-0.013 0.000	0.108 0.000	-0.007 0.000
	Filename Reference to Primary GNSS lever arm [m]	File type 0.006	0.218	-0.435
	sbet_Mission 1.out Reference to Primary GNSS lever arm std dev [m]	SBET Trajectory File 0.020
	export_Mission 1.txt Aircraft to Reference mounting angles [deg]	ASCII Export Output 0.000 0.000	0.000

Satellite Lock/Elevation GPS L1 SNR GPS L2 SNR GLONASS L1 SNR

POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 16:01:53 GLONASS L2 SNR BEIDOU SNR POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 16:01:

53 Roll/Pitch Heading North/East Velocity Down Velocity Total Speed Ground Speed Body Acceleration Total Body Acceleration

	POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 16:01:53
	GNSS QC
	GNSS QC Statistics
	Statistics	Min	Max	Mean
	Baseline length [km]	12.94	13.17
	Number of GPS SV	4	6	5
	Number of GLONASS SV	3	5	4
	Number of QZSS SV	0	0	0
	Number of BEIDOU SV	0	0	0
	Total number of SV	8	11	10
	PDOP	2.02	4.12	2.96
	QC Solution Gaps	0.00	0.00
	Solution Type	Fixed	Float	No solution
	Epoch (s)	679.00	2.00	0.00
	Percentage	99.71	0.29	0.00

Zhang W, Qi J, Wan P, Wang H, Xie D, Wang X, Yan G. An Easy-to-Use Airborne LiDAR Data Filtering Method Based on Cloth Simulation. Remote Sensing. 2016; 8(6):501.

https://www.danielgm.net/cc/

https://iaps.zrc-sazu.si/en/rvt#v

POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07

Y Accelerometer Scale Error (ppm) Z Accelerometer Scale Error (ppm)POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07

Y Gyro Scale Error (ppm) Z Gyro Scale Error (ppm)POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07

POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 16:01:53

Y Accelerometer Scale Error (ppm) Z Accelerometer Scale Error (ppm)POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 16:01:53

Y Scale Error (ppm) Z Gyro Scale Error (ppm)POSPac MMS 8.3 Diagnostic QC Report -report_Mission -02/26/2019 16:01:53

Estimated Position Accuracy

POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07 Accelerometer Scale Error (ppm) X Accelerometer Scale Error (ppm) POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 15:58:07 Estimated Position Accuracy POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 16:01:53 Accelerometer Scale Error (ppm) X Accelerometer Scale Error (ppm) POSPac MMS 8.3 Diagnostic QC Report -report_Mission 1.pdf -02/26/2019 16:01:53 Y Gyro Bias Z Gyro Bias (deg/h) POSPac MMS 8.3 Diagnostic QC Report -report_Mission -02/26/2019 16:01:53