Louis Lac

Jean-Pierre Da Costa

Marc Donias

Barna Keresztes

Alain Bardet

Crop Stem Detection and Tracking for Precision Hoeing Using Deep Learning

Keywords: precision agriculture, deep learning, neural network, object detection, tracking algorithm

Developing alternatives to the chemical weeding process usually carried out in vegetable crop farming is necessary in order to reach a more sustainable agriculture. However, a precise mechanical weeding requires specific sensors and advanced computer vision algorithms to process crop and weed discrimination in real-time.

In this paper we propose an algorithm able to detect, locate, and track the stem position of crops in images which is suitable for precision actions in vegetable fields such as mechanical hoeing within crop rows. The algorithm is two-fold: (i) a deep neural network for object detection is first used to detect crop stems in individual RGB images and then (ii) an aggregation algorithm further refines the detections taking advantage of the temporal redundancy in consecutive frames.

We evaluated the pipeline on images of maize and bean crops at an early stage of development, acquired in field conditions with a camera embedded in an experimental mechanical weeding system. We reported F1-scores of respectively 94.74 % and 93.82 % with a location accuracy around 0.7 cm when compared with human annotation. Moreover, this pipeline can operate in real-time on an embedded computer consuming as little power as 30 W.

Introduction

Vegetable market weighs around 1250 B$ worldwide and keeps increasing steadily at a rate of 2.4 % a year [START_REF] Indexbox | World -Vegetable -Market Analysis, Forecast, Size, Trends and Insights[END_REF]). In this context and to ensure sustainable yields, weeding of vegetable fields is required for almost all kinds of crops [START_REF] Van Heemst | The influence of weed competition on crop yield[END_REF]). Weeding is usually handled using herbicides, which are inexpensive and efficient but recent awareness and criticism about the negative impact of phytosanitary products on soils and wildlife [START_REF] Torretta | Critical Review of the Effects of Glyphosate Exposure to the Environment and Humans through the Food Supply Chain[END_REF]) has pushed organic farming practices in the spotlight [START_REF] Lamichhane | Chapter Two -Research and innovation priorities as defined by the Ecophyto plan to address current crop protection transformation challenges in France[END_REF]).

However, the solutions to weed eradication without chemical products are limited. Organic farms use a mix of manual and mechanical weeding (Sanbagavalli ( 2020)) which expensive and repetitive for workers. To address those issues new innovative solutions aim at automating the weeding process. Some reviews show that automatic inter-row weeding is both feasible and economically viable [START_REF] Pedersen | Agricultural robots-system analysis and economic feasibility[END_REF]) but intra-row weeding is still challenging as the space between crops is much lower and crop distribution in the row is not always predictable [START_REF] Griepentrog | Robotic Weeding[END_REF]).

Related Work

Several methods have been developed for crop and/or weed detection in vegetable farms which differ in the complexity and number of sensors embedded in the system as well as in the embedded algorithms.

Concerning the acquisition system, some methods employ simple RGB or RGB-NIR (Near Infra-Red) sensors [START_REF] Jeon | Robust Crop and Weed Segmentation under Uncontrolled Outdoor Illumination[END_REF]; [START_REF] Montalvo | Automatic detection of crop rows in maize fields with high weeds pressure[END_REF]; [START_REF] Lottes | UAV-based crop and weed classification for smart farming[END_REF]; [START_REF] Bah | Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images[END_REF]; [START_REF] Lottes | Fully Convolutional Networks With Sequential Information for Robust Crop and Weed Detection in Precision Farming[END_REF]) while others use more advanced sensors such as multi-spectral or hyper-spectral cameras [START_REF] Gerhards | Real-time weed detection, decision making and patch spraying in maize, sugarbeet, winter wheat and winter barley[END_REF]; [START_REF] Wendel | Self-supervised weed detection in vegetable crops using ground based hyperspectral imaging[END_REF]) or depth-camera [START_REF] Gai | Automated crop plant detection based on the fusion of color and depth images for robotic weed control[END_REF]). The first solution is often preferred as it is less expensive and usually more suitable for real-time applications [START_REF] Griepentrog | Robotic Weeding[END_REF]). Sensors are mostly embedded directly on the farming robot [START_REF] Gerhards | Real-time weed detection, decision making and patch spraying in maize, sugarbeet, winter wheat and winter barley[END_REF]; [START_REF] Jeon | Robust Crop and Weed Segmentation under Uncontrolled Outdoor Illumination[END_REF]; [START_REF] Montalvo | Automatic detection of crop rows in maize fields with high weeds pressure[END_REF]; [START_REF] Wendel | Self-supervised weed detection in vegetable crops using ground based hyperspectral imaging[END_REF]; [START_REF] Lottes | Fully Convolutional Networks With Sequential Information for Robust Crop and Weed Detection in Precision Farming[END_REF]) but can also be carried by Unmanned Aerial Vehicles (UAV) [START_REF] Lottes | UAV-based crop and weed classification for smart farming[END_REF]; [START_REF] Bah | Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images[END_REF]).

Regarding the crop and weed recognition task, semantic segmentation of images is almost always preferred, some only discriminate between crops and weeds while others also classify the species. Solutions that focus on inter-row hoeing of crops with high spacing often rely on standard computer vision methods to segment the image [START_REF] Gerhards | Real-time weed detection, decision making and patch spraying in maize, sugarbeet, winter wheat and winter barley[END_REF]; [START_REF] Montalvo | Automatic detection of crop rows in maize fields with high weeds pressure[END_REF]; [START_REF] Wendel | Self-supervised weed detection in vegetable crops using ground based hyperspectral imaging[END_REF]; [START_REF] Lottes | UAV-based crop and weed classification for smart farming[END_REF]; [START_REF] Gai | Automated crop plant detection based on the fusion of color and depth images for robotic weed control[END_REF]). Local features and descriptors are often extracted using either radiometric indices, e.g. Excess Green Index (EGI), NDVI (Normalized Difference Vegetation Index) or geometrical and textural features, e.g. Fourier descriptors or other hand-tuned features descriptors. Classification is performed using methods such as Markov Fields, Principal Component Analysis (PCA), Random Forest or other machine learning classifiers. Other work methods take advantage of new deep learning frameworks such as semantic segmentation networks to perform classification [START_REF] Jeon | Robust Crop and Weed Segmentation under Uncontrolled Outdoor Illumination[END_REF]; [START_REF] Bah | Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images[END_REF]; [START_REF] Lottes | Fully Convolutional Networks With Sequential Information for Robust Crop and Weed Detection in Precision Farming[END_REF]; [START_REF] Wu | Robotic weed control using automated weed and crop classification[END_REF]). [START_REF] Lottes | Fully Convolutional Networks With Sequential Information for Robust Crop and Weed Detection in Precision Farming[END_REF] also takes into account the temporal aspect of the data.

The BIPBIP Project

The BIPBIP (Bloc-outil et Imagerie de Précision pour le Binage Intra-rang Précoce) 1 project aims at developing a precision weeding module based on fine mechanical hoeing that is designed to weed crops in the intra-row (illustrated in Figure 1a) without use of phytosanitary products. It is designed to weed one row at a time, but it can be replicated in parallel to operate on multiple rows at the same time within the same lane. Moreover, it is independent of its carrier and can be easily transferred to another vehicle. The module is built around two components: (i) a vision system that detects and tracks crop stems, further 1 Tool-block and Precision Imaging for Early Intra-row Hoeing, http://challenge-rose. fr/en/projet/bipbip-project/ developed in section 2 and (ii) a mechanical weeding tool not addressed in this paper.

The module primarily targets market gardening (bean, onion, leek, etc.) but is also tested on field crops with large intra-row spacing (maize, sweet corn, rapeseed, etc.) as part of the ROSE Challenge organized by the French National Research Agency. Only stages of development between 2 and 5 weeks are considered as weed competition is at its highest during this growth period.

However, currently only maize and bean are supported in the configuration described in Figure 1b with d between from 75 cm to 80 cm and d within = 15 cm for maize and d between from 15 cm to 37.5 cm and d within from 3 cm to 8 cm for bean. In this configuration weed infestation may be high, occlusions and obfuscations can occur, so the detection module should handle those edge cases correctly.

The mechanical weeding tool is currently composed of a metal tip that scraps the soil to remove all weeds without distinction around each crop of interest.

A system not described in this paper can activate and move it along the row.

In addition, two mobile plowshares placed on both sides of the module assist weeding in the inter-row. This system imposes a speed constraint for the overall module of around 0.5 m/s. The advantage of such a system is that detecting weeds is not required for hoeing, only the crop stem positions need to be known as they are the only part of the crops to be avoided by the metal tip and the mobile plowshares during the weeding process.

Motivations and Contributions

This paper proposes a stem detection pipeline developed for the BIPBIP weeding module. This pipeline should operate in real-time and provide the stem position of crops with a great location accuracy which is required for the precision hoeing process. Our contribution is twofold:

• We propose a method to detect stem locations in images using an object detector, and we evaluate two alternatives: (i) approximating the stem location by the whole crop bounding box center, and (ii) detecting stems with a bounding box centered at its location.

• We propose a temporal aggregation algorithm which takes advantage of the temporal coherence of successive images to improve the detection performance of the network.

The article is organized as follows: the sensors and the database are presented in Section 2, the deep neural network and the aggregation algorithms are detailed in Section 3, results are described in Section 4 and conclusions are drawn in Section 5.

Materials

Vision System

The vision system illustrated in Figure 2 consists of a single 3 Mpixels industrial RGB camera (Basler acA2500-gc) equipped with a C125-0418-5M F1.8 f 4 mm Basler lens which can capture images at a rate of 15 frames per second (fps). It is pointing down with its principal optical axis perpendicular to the ground at a constant height of around 35 cm. This system is confined in a hull,

cm

Figure 2: Schematic representation of the sagittal cross-section of BIPBIP hoeing system with the mechanical weeding tool (left), the vision system with the embedded computer (orange), the two LED panels and the camera (black). Forward direction is to the right.

sealing it from natural lighting. Light conditions are artificially controlled by two 20 W LED panels in order to obtain a brightness as uniform as possible.

This setting avoids unpredictable conditions such as glare and overexposure and ensures a better robustness of the detection algorithm. Moreover, camera focus is set to match the camera height and the exposure and white balance are adjusted and fixed at the beginning of the weeding process.

The computation is processed in real time on an Nvidia Jetson Xavier which is an embedded computer optimized for deep learning and computer vision computations. Moreover, it can operate at the very low power consumption of 30 W max. The algorithms are developed in Python using Numpy and OpenCV, and the framework used for the neural network inference is written in C, C++ and CUDA.

Database

We acquired two databases2 both for training and validation of our method.

The acquisition is processed with the vision system described in Figure 2 embedded in a lightweight acquisition module that can be carried easily. Both The first one (that we call the image database) is an image collection used for training and validation of the deep learning algorithm presented in section 3.2. We used 80 % of the database for the training and the remaining 20 % for the evaluation. The images are either 3 Mpixels or 5 Mpixels and soil conditions diverge slightly: Nouvelle-Aquitaine soils are sandy while Montoldre ones are tougher and more dusty. This database currently supports three types of crops at an early stage of development (2 to 5 weeks): maize, bean and leek.

However, as the leek database is currently not large enough to be representative, leek results are not presented in this paper. It is also designed to cover as many situations as possible such as weed infestation levels, soil types, grown stages, crop overlap and obfuscation, but it is continuously extended with new images of previously unseen conditions. Some samples are presented in Figure 3.

The second database (which we call the video database) is composed of four 15 fps videos saved as consecutive frames, two for maize crops and two for bean crops. It is designed to mimic the real acquisition context of the weeding module and is dedicated to the evaluation of the aggregation algorithm presented in section 3.3. Soil conditions and growth stages are different for each video, the first two of each crop contain crops at more advanced growth stages and packed tightly while the two last contain crops at an earlier stage. Some samples can be seen in Figure 4.

Methods

The developed pipeline is two-fold: (i) an object detection based deep neural network first provides stem locations in individual RGB images and then (ii) an aggregation algorithm further filters the detections, leveraging the temporal aspect of the successive frames.

Moreover, we compare two approaches for the stem detection part. In the first one the neural network is trained to detect entire crops and the stem is approximated by the crop bounding box center. In the second one the network is trained to directly detect the stem as an object. We believe that the second approach should give better location accuracy, but the network may struggle to learn these uncommon objects. In the following we use labels such as Maize Crop to denote the first configuration and Maize Stem to denote the second configuration.

In the following, we first describe the database annotation process, then the two parts of the pipeline are detailed in Section 3.2 and Section 3.3. 155

Annotation

The databases presented in section 2.2 are annotated with bounding box ground-truths to provide labels for the neural network training and for the evaluation. The annotation work is performed with the labelImg3 software, and we annotated both stems and entire crops for maize and bean. The crop bounding box is a rectangular box around the whole crop (red and blue boxes in Figure 5) while the stem bounding box is a square box centered on the stem entry point in the ground and with a side length normalized to be equal to 7.5 % of the image's smallest length (orange and cyan boxes in Figure 5). For the image database introduced in section 2.2 we annotated all the images (statistics are shown in Table 1). As presented in section 4.1, it is further divided in two: one part for training (80 %) and the other one for validation (20 %).

The two maize videos are 1765 and 427 images long and 51 of them are annotated (144 crops). The two bean ones are 251 and 784 images long and 53 of them are annotated (263 crops). The annotation process is similar to the image database except that the annotation is not performed on every image to avoid tagging the same crop in multiple successive images, but every crop is annotated at least once.

Stem Detection with Neural Network

We propose to use an object detection neural network to regress stem locations. As our application requires real-time computation we chose a one-stage network over a two-stage because they achieve higher inference speed [START_REF] Huang | Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors[END_REF]; [START_REF] Jiao | A Survey of Deep Learning-Based Object Detection[END_REF]). In this family different designs are proposed, the most used being SSD [START_REF] Liu | SSD: Single Shot MultiBox Detector[END_REF]), RetinaNet Lin et al. (2017)) and YOLO [START_REF] Redmon | You Only Look Once: Unified, Real-Time Object Detection[END_REF]). In recent years SSD and RetinaNet were supplanted by more accurate and faster networks such as EfficientDet [START_REF] Tan | EfficientDet: Scalable and Efficient Object Detection[END_REF]) and ASFF [START_REF] Liu | Learning Spatial Fusion for Single-Shot Object Detection[END_REF]) while YOLO underwent several enhancement iterations (Redmon andFarhadi (2017, 2018)). Recently a team of researchers developed YOLOv4 [START_REF] Bochkovskiy | YOLOv4: Optimal Speed and Accuracy of Object Detection[END_REF]) with the aim to achieve a high quality object detector which is simple to train and ready for production by aggregating the newest deep learning features that can improve the speed-accuracy trade-off.

A simplified overview of the architecture is presented is Figure 6. We chose this framework as it is the most accurate and faster to our knowledge and is ready Input Backbone Neck Prediction Image for production. This framework still offers a wide variety of networks achieving different speed-accuracy trade-offs ranging from Tiny YOLOv4 -a smaller variant that can run faster-to YOLOv4 -a more accurate but slower variant.

During inference, we extract the bounding box centers, which represent the crop stem locations in our application. In the following, the stem detections for image I n are noted D n .

In section 4 we benchmark some variants to highlight the speed-accuracy trade-offs and to choose a variant suitable for our application.

Temporal Aggregation

Object detection operates image by image, but the weeding robot requires a unified detection in order to make a decision. For this purpose we can take advantage of the temporal redundancy of detections caused by the overlap of successive images to improve the accuracy and provide better confidence indicators. We propose a simple approach where stem detections from different images are first projected in a common referential via the computation of the optical flow, then they are aggregated to remove duplicates and recover missed detections.

We formulated several hypotheses to simplify the problem, (i) the ground and crops are assumed rigid bodies with no relative displacements between any part of them, (ii) the ground is assumed to be planar, (iii) the camera principal axis is perpendicular to the ground and (iv) the camera displacement is in the horizontal plane i.e. no changes of height. With these hypotheses in place the optical flow of the soil pixels between two consecutive images is assumed to be constant and equal at every pixel location, and it can be used to map ground points in an image to another image knowing that displacement. As we defined the stems as the entry point in the soil, stems from one image can be mapped to another one knowing the ground displacement between them.

We propose an iterative algorithm which operates at each new image. It

Soil Mask Extraction

The soil mask extraction algorithm (in blue in Figure 7) computes a binary

mask M n ∈ N W ×H [0,1]
where W and H are the image width and height, which means that in M n pixels of the soil class have a value of one. This mask is used in the displacement computation algorithm to compute the optical flow of the soil points only.

The first step is the Non-Vegetal Mask Extraction (NVME). The Excess Green Index (EGI) [START_REF] Woebbecke | Color Indices for Weed Identification Under Various Soil, Residue, and Lighting Conditions[END_REF]) of the image I n is first computed.

Δd n T n I n-1 I n T n-1 Δd n-1 Optical Flow NVME NVME Mask Blending Median Add Projection Association E n-1 E n M n F n δd n D n D′ n
Soil mask extraction Displacement computation Aggregation A threshold t e ∈ N is then applied to the EGI to obtain the mask of the soil:

E n =      1 if egi(I n ) < t e , 0 otherwise.
(1)

The same process is applied to I n-1 to obtain E n-1 .

The second step is the Mask Blending. This step combines the two successive masks E n and E n-1 to extract pixels of soil class from both images. As the dense optical flow computed in the next section can be less precise at object boundaries and at pixel locations that are obfuscated in one of the images, this operation helps to remove those pixels from the mask.

E n =      1 if E n = 1 and E n-1 = 1, 0 otherwise. (2)
A second operation applies a morphological closing (morphclose) with a disk kernel having a radius of ten pixels to remove small holes and a binary mask of the image safe area M s (area without visible tool or hull parts) is applied:

M n =      1 if morphclose(E n ) = 1 and M s = 1, 0 otherwise.
(3)

Illustrations of the intermediate soil masks and of the blended masks are shown in Figure 8a.

Displacement Computation

The displacement between the current image I n and the first image I 1 is computed by integration of the displacement between successive images from I 1 to I n . First, the dense Optical Flow

F n | Rn→Rn-1 ∈ R W ×H×2 from I 1 to I n-1
is computed using Farnebäck's polynomial expansion [START_REF] Farnebäck | Two-Frame Motion Estimation Based on Polynomial Expansion[END_REF]). The flow is then masked with M n to gather the flow of the soil pixels only and the median value is calculated:

δd n | Rn→Rn-1 = median F n (x, y)| Rn→Rn-1 \ M n (x, y) = 1 . (4)
This value represents the translation from R n to R n-1 as a real value 2-240 dimensional vector.

The total translation from R n to the first image reference frame R 1 is then computed by summation with the previous total translation:

∆d n | R1 = ∆d n-1 | R1 + δd n | Rn→Rn-1 . (5)
The relative translations and total translation are illustrated by the orange arrows in Figure 8c.

Aggregation

The aggregation process described in turquoise on Figure 7 is two-fold: (i)

245

the detections D n of image I n are projected in the first image referential R 1 , then (ii) those detections are associated with previous ones representing the same stem object, resulting in a set of aggregated detections T n that we called a "tracker". The stem detections D n are relative to the R n reference frame of image I n .

X 1 Y 1 T 3,1 T 3,3 T 3,2 T 3,4 X 3 Y 3 D 1,1 D 1,2 D 2,2 D 2,1 D 3,1 D 3,2 I 1 I 2 I 3 b c I 1 I 2 I 3 X 1 Y 1 X 2 Y 2 E 2 E 3 a δd 2 δd 3 Δd 3 E′ 3 ℛ 1 ℛ 2 ℛ 3 ℛ 1
The projected detections D n are first obtained by projection of D n in R 1 with an element-wise addition:

D n = D n + ∆d n | R1 . (6)
These projected detections are illustrated by pink crosses in Figure 8c.

In a second step, the projected detections of image I n are associated with detections from previous images. We call a tracker a set of stem detections from different images associated through this aggregation process and representing the same underlying stem object. We define the canonical position of a tracker as the average position of its stem detections (isobarycenter). Moreover, a tracker has a lifetime. It is considered active if a detection has been associated with it during the last max inactive ∈ N time steps, otherwise it is considered inactive.

Only active trackers are considered during the association process while the aggregate detections T n are composed of both active and inactive trackers.

The aggregation algorithm is based on the association process of the COCO AP evaluation metric [START_REF] Lin | Microsoft COCO: Common Objects in Context[END_REF]) and we use the Euclidean distance Algorithm 1 "Merge" algorithm for detection-tracker association. Trackers are illustrated in Figure 8c by turquoise bounding boxes.

Results

We evaluated the two components of our system -object detection and aggregation-independently. Firstly, in Section 4.1 we evaluate different YOLO architectures based on standard object detections metrics, and we select the one that best suits our needs for the following evaluation. Secondly, in Section 4.2 we evaluate the temporal aggregation algorithm with a custom metric designed to assess the stem detection ability. We compare the two approaches described in Section 3, i.e. the whole crop bounding box v.s. the stem bounding box, and we perform a grid-search on two main hyper-parameters of the temporal aggregation algorithm. Finally, we discuss the results in Section 4.3.

Stem Detection

We compare 3 networks of different depth and structure: (i) YOLOv4 is the deepest network thus potentially the most accurate but also the slower, (ii) Tiny YOLOv4 (YOLOv4 T) which is a shallow variant of YOLOv4 expected to be faster and (iii) Tiny YOLOv3 3L (YOLOv3 T3L) which is a former tiny YOLO variant trained for comparison purposes.

We evaluate the performance with the standard COCO object detection metrics [START_REF] Lin | Microsoft COCO: Common Objects in Context[END_REF]). More specifically, we use the AP 0.5:0.95 (AP), the AP 50 , AP 75 and the AR 100 . Moreover, we provide the mean Intersection over Union4 (mIoU), which is computed at a 50 % IoU threshold and a fixed confidence threshold of 80 % for YOLOv4 and 25 % for Tiny variants (found via a grid-search not presented in this paper). We also report the inference speed in frames per second (FPS).

We trained the networks to detect 6 different classes at once: maize, bean and leek crops and their stems on the image database presented in section 2.2.

In this paper we focus on maize and bean, so only those results are reported. We split the image database in a training set and a validation set with an 80 %-20 % ratio and trained for 10,000 iterations of batches of size 64 images. We also used transfer learning [START_REF] Athanasiadis | A Framework of Transfer Learning in Object Detection for Embedded Systems[END_REF]) from networks pre-trained on ImageNet [START_REF] Deng | ImageNet: A large-scale hierarchical image database[END_REF]). The database is also augmented to reduce overfitting with the following transformations: random image scaling (from x0.4 to x1.6 the original size), random color changes (hue ±10 %, saturation and exposure from 1 to 1.5) and image Mixup [START_REF] Zhang | Mixup: Beyond Empirical Risk Minimization[END_REF]). For practical reasons we chose an input size of 544 × 544. A higher input size would yield lower training and inference speeds and more working memory, and a lower size would degrade the accuracy too much. Preliminary experiments not listed in this paper showed that this input resolution achieves a suitable trade-off for our application.

The training is performed on a dedicated workstation running Ubuntu 18

LTS with an Intel Core i7-7700 4 Cores at 3.6 GHz CPU 32 GB and an Nvidia GeForce RTX 2080 SUPER 8 GB GPU. This setting is sufficient to handle training for our current database in approximately 2 hours for small networks (Tiny YOLOv4) to 7 hours for the largest one (YOLOv4).

Table 2 shows that there is a clear trade-off between object detection accuracy and the inference speed that can be obtained. On the Nvidia Jetson Xavier, Tiny YOLOv4 is more than 7 times faster than YOLOv4 at the cost of 6.59 % AP. YOLOv4 yields a 3.34 % higher AP 50 and an mIoU 2.86 % higher. YOLOv3 T3L is both slower (-5 fps i.e. -5 %), less accurate (-8.51 % AP) and less precise in bounding box regression (-2.66 % mIoU) compared to Tiny YOLOv4, which illustrates the performance gains introduced with newer YOLOv4 networks. The AR 100 illustrates the same trend. When comparing AP and AP 50 by crop type in Table 3 it appears that all networks have more difficulties in detecting the stems compared to the entire crop. For instance Tiny YOLOv4 loses 10.38 % AP 50 for maize stems and 20.31 % AP 50 for bean stems when compared to the whole crop (and respectively 29.53 % and 34.76 % for the AP). The drop in performance can be explained by two aspects: (i) the standard object detection metrics may not be suitable to evaluate the stems in the way we defined them as objects rather than keypoints, (ii) stems are more difficult to detect because of their small size, their less well-defined boundaries and the higher obfuscation they may suffer from.

Moreover, it can be observed that maize stems are better detected than bean stems. This difference can be explained by the crop layout for bean crops that is narrower than the maize one (cf Figure 1b), thus leading to more obfuscation and overlap.

These results made us choose Tiny YOLOv4 for our application. It is fast enough for real-time use while still leaving some GPU time for other algorithms and its accuracy is sufficient, and could be improved in the future by increasing the input resolution for instance.

Stem Aggregation

We chose to evaluate the stem aggregation algorithm with a metric that better models our algorithms and their application. While the COCO AP is suitable for object detection evaluation, it is not for keypoint evaluation. Thus, we replaced the IoU similarity metric of the AP by the Euclidean distance be-tween reference stems and predictions and the mean IoU is replaced by Location Accuracy (LAcc). Moreover, contrary to the AP we chose to fix the confidence threshold of the detector to its optimal value (25 % for Tiny YOLOv4) and the distance threshold (which is analog to the IoU threshold) is also fixed to 2 cm, which is a value suitable for the precision agriculture task targeted. This allows the evaluation of the precision, the recall, F1-score [START_REF] Olson | Advanced Data Mining Techniques[END_REF])

and Location Accuracy. We believe that this metric is more concrete than the AP and measures more directly the hoeing performance (potential crop losses and false alarms). This evaluation is performed on the video database presented in section 2.2.

We chose a value of max inactive = 30 for the tracking algorithm as this parameter is constrained by the speed of the hoeing module and the camera frame-rate. We also chose a value of t e = 40 by a qualitative observation of the generated soil masks.

In the following, we present three experiments: (i) as the aggregation algorithm depends on two hyper-parameters -namely minDets and maxDistwe produced a grid-search using the F1-score as the comparison metric to find the local optimum of those values for the stem detection task, (ii) we evaluated the performance of the aggregation algorithm and (iii) we compared the two approaches described in section 3 which are either using the crop bounding box or using the stem bounding box to regress the stem position.

Grid Search

We performed a grid-search for the two crop stems -maize stem and bean stem-independently in the following configuration: (i) minDets varies from 1 to 20 by 1 increment, (ii) maxDist varies from 3 % to 18 % by 3 % increment (expressed as a percent of the image's smallest side length). We plotted the precision-recall curves with respect to the combinations of those two parameters.

On Figure 9 each curve varies along the minDets parameter and the maxDist parameter variation is presented by different curve colors. We chose the best combination of the two parameters based on the F1-score at that point in the precision-recall curve. This value is shown in Figure 9 as well as the F1-score without the aggregation algorithm (red cross) for comparison.

Globally for the two grid-search presented in Figure 9 it can be observed that there is a local optimum for the maxDist parameter, in ascending order the curves first get closer to the top-right corner (which represents the best possible F1-score) and then move away from it. The same behavior is observed with the minDets parameter, a low value gives an excellent recall but a poor precision (right end of the curves) and vice-versa (left side of the curves); and in-between an optimum value is attained. The optimum F1-score of 94.74 % is obtained with minDets = 10 detections and maxDist = 12 % (4.6 cm) for maize stems and the optimum F1-score of 93.82 % with minDets = 13 detections and maxDist = 6 % (2.3 cm) for bean stems. The lower maxDist value for bean can be explained by the wider crop spacing for maize that results in less confusion between adjacent crops, thus the constraint on this parameter can be relaxed. suitable for precision hoeing in both cases. The standard errors of the location accuracies do not show that the difference in performance is significant.

To give a better insight of the location uncertainty and detection error, dispersion ellipses [START_REF] Saporta | Régions de confiance en analyse factorielle[END_REF]) are computed and illustrated in Figure 10. The ellipses are larger for big crops with many leaves (e.g. maize crops on the left of the top row) and for crops obfuscated by weeds (e.g. bean crops on the left of the bottom row), which indicate that the overall performance is lower in these more difficult cases. This figure also highlights one common location mistake done by the neural network which tends to detect the top of the crop when the stem entry point in the ground is hard to detect (obfuscation by leaves or weeds for instance). Due to the strong parallax introduced by our acquisition system this creates line patterns in a tracker's detections (green maize stem tracker in the first maize image and bean stems in the two last bean images).

Concerning the inference speed of the temporal aggregation algorithm, the latency is dominated by the computation of the optical flow which runs at around 15 fps while the aggregation in itself runs at more than 100 fps.

Crop vs Stem Detection

Table 4 shows that using a bounding box centered on the crop stem (e.g. "Maize Stem" configuration) yield better performance than detecting the whole crop and using the bounding box center as an approximation of the stem location (e.g. "Maize Crop" configuration). The F1-score is dramatically improved for maize crops (+59.82 %) and slightly improved for bean crops (+0.20 %). Also, for both crops the Location Accuracy is better when using the stem bounding box rather than the crop bounding box: -0.79 cm for maize stems and -0.18 cm for bean stems. The difference in performance between the two kinds of crop can be explained by the difference in their size: bean crops are generally much smaller than maize crops, thus the bean crop bounding box center is a better approximation of the stem location than the maize one. It can be noted that the bean crop configuration yields a better recall than the bean stem configuration (+1.90 %), which illustrates the better detection of whole crops compared to stem bounding boxes pointed out in section 4.1.

Discussion

We have shown that the proposed detection pipeline yields performances suitable for the targeted precision hoeing application, both in terms of detection (average F1-score of 94.28 %), location accuracy (average LAcc of 6.1 mm) and inference speed. The use of an off-the-shelf object detector to detect stem bounding boxes seems relevant as it yields a great accuracy and the implementation is well optimized for real-time applications. We note, however, that work such as [START_REF] Verucchi | A Systematic Assessment of Embedded Neural Networks for Object Detection[END_REF]) allows even higher inference speed on specialized embedded systems such as the NVIDIA Jetson Xavier, permitting the use of deeper networks like YOLOv4 to improve the accuracy even further without slowing down other algorithms.

The developed temporal aggregation algorithm improves the accuracy of the detection, on average there are fewer missed stems and fewer false positives. However, it does not improve the location accuracy. Though less than 2 mm in average which seems not significant, the explanation for this decrease needs further exploration to find the potential causes. We believe that the many hypotheses we stated about the hoeing module posture and soil model can be the cause of biases and noises in the aggregation process. For instance, these modeling errors could cause a dispersion or a drift of the aggregated stem detections, leading to a degradation of the location accuracy.

Our results also highlights the difference in performance between crop types.

Maize stems are better detected than bean stems no matter the configuration of the neural network. They also benefit more from the temporal aggregation accuracy boost. One possible explanation is the tighter crop layout of bean crops compared to maize crops which may generate more overlap between adjacent crops and more uncertainty on the precise stem location.

Comparing our work to the available literature is challenging as there are few public databases available, and to our knowledge there is no comparable hoeing method or public dataset matching our application needs. Future work will focus on the publication of our dataset to remedy this situation. An area of improvement is the addition of more crop types, growth stages and soil variability to assess the robustness of our algorithms.

Conclusions

In this paper we propose a computer vision pipeline able to detect in realtime the precise location of crop stems which can be used in challenging precision agriculture tasks such as mechanical hoeing of the intra-row. The developed method is two-fold: (i) an object detector based neural network is first used to detect stems in RGB images and then (ii) an aggregation algorithm is used to further refine the detections by leveraging the temporal nature of the successive frames. We measured the efficiency of our algorithms on our database composed of maize and bean crops in two configurations: (i) stems as crop bounding box centers and (ii) stems as objects.

We evaluated the algorithms with the F1-score as well as a location accuracy metric and reported the best results using the small variant of Yolo4 named Yolo4 Tiny in the configuration of stems as objects. Currently, the system can detect maize and bean stems with an F1-score of respectively 94.74 % and 93.82 % and a location accuracy of 0.7 cm and 0.5 cm, which is suitable for precision hoeing. Future work will focus on key-points based neural networks [START_REF] Zhou | Objects as Points[END_REF]) that may be best suited as our goal is to detect key-points rather than bounding boxes. Those networks are less common and require more work in order to obtain a stable training and a suitable inference speed. Additionally, our current execution speed is sufficient but recent work [START_REF] Verucchi | A Systematic Assessment of Embedded Neural Networks for Object Detection[END_REF] showed impressive results in optimizing execution speed of neural networks on specialized hardware which we can take advantage of to further improve inference speed or the power consumption of our system. Lastly we are planning on extending the current database with more images covering more conditions and more crop types.

Acknowledgments

We acknowledge the French Research Agency (ANR) for funding (grant ANR-17-ROSE-0001 -BIPBIP) and thank the organizers of the ROSE Challenge and all the partners of the BIPBIP project.

Figure 1

 1 Figure 1: 1a: One BIPBIP weeding module embedded under an electric tractor operating on the left row of a maize bed in an experimental plot; 1b: Between-rows (d between ) and within-row spacing (d within ). Row direction is indicated by the arrow.

Figure 3 :

 3 Figure 3: Samples of the image database used for the training and the validation of the object detector.

Figure 4 :

 4 Figure 4: Some samples of the four videos of the video database.

Figure 5 :

 5 Figure 5: Samples of the image database with the annotations overlaid. Maize crops are annotated in blue and the stems in cyan, bean crops in red and the stems in orange.

Figure 6 :

 6 Figure 6: YOLOv4 object detector simplified architecture with CSPDarknet53 backbone (Wang et al. (2019)), PANet neck[START_REF] Liu | Path Aggregation Network for Instance Segmentation[END_REF]) and YOLOv3 anchor-based prediction head[START_REF] Redmon | YOLOv3: An Incremental Improvement[END_REF]).

  takes as input the new image I n and the past one I n-1 , as well as the stem detections D n of image I n . It updates two pieces of data: ∆d n | R1 ∈ R 2 which is the total translation of the frame of reference R n associated to the image I n to the first image one R 1 , and a set of aggregated stem detections T n where each detection is a location p ∈ R 2 expressed in the frame of reference R 1 (the initial conditions are respectively zero and the empty set). The referential R n with axes (X n , Y n ) associated to the image I n has an origin located at the top-left image corner as illustrated in Figure 8. The algorithm is composed of three parts illustrated in Figure 7: (i) the soil mask extraction (in blue) extracts the pixels belonging to the soil, (ii) the displacement computation (in orange) computes the translation from the first image to the current one and (iii) the aggregation (in turquoise) projects the stem detections in R 1 and aggregates them.

Figure 7 :

 7 Figure 7: Overall scheme of the iterative aggregation process. Soil mask extraction (blue), displacement computation (yellow) and aggregation (turquoise). Mathematical notations are introduced in section 3.3 and reference frames have been omitted for clarity. NMVE stands for Non-Vegetal Mask Extraction.

Figure 8 :

 8 Figure 8: Simplified illustration of the aggregation process on three consecutive images. a: non-vegetation masks and blended soil mask. b: individual stem detections (pink) in their original reference frame. c: relative image translations and total displacement (orange), projected detections in the first image reference frame (pink) and trackers with past detections (turquoise). The frame-rate is artificially low for clarity.

Inputs:

  Tn-1 and D n Parameters: maxinactive and max dist trackers ← filterInactive(Tn-1, maxinactive) detections ← sortByDecreasingConfidence(D n ) matches ← new list for detection ∈ detections do (bestDistance, bestT racker) ← (+∞, nil) for tracker ∈ trackers do distance ← barycenter(tracker) -detection 2 if distance < bestDistance then (bestDistance, bestT racker) ← (distance, tracker) end if end for if bestDistance < maxDist and bestT racker / ∈ matches then matches ← matches ∪ (detection, bestT racker) D n and tracker isobarycenters as the similarity metric for the association. The aggregation process is detailed in Algorithm 1. The detections D n are first sorted by decreasing confidence. Then, for each detection the Euclidean distance to every active tracker T n-1 is computed. If 265 the closest tracker is at a distance below a threshold maxDist ∈ R and if it has not been previously associated with another detection, the detection is added to that tracker. This process results in an updated set of trackers T n . At inference and evaluation time, a tracker is considered valid if the number of detections it represents is above a threshold minDets ∈ N. In section 4 we 270 evaluate the influence of the distance threshold maxDist and detection number threshold minDets.

Figure 9 :

 9 Figure 9: Precision-recall curves of the temporal aggregation performance on maize and bean stems with respect to the minDets parameter (curve line) and maxDist (curve color).

  minDets ranges from 1 detection to 20 detections. The F1-score for the best combination of these parameters are highlighted and the F1-score of the stem detection without the aggregation algorithm is shown as a red cross.

Figure 10 :

 10 Figure 10: Examples of dispersion ellipses containing 1 sigma (i.e. 68 %) of the samples for maize (top row) and bean (bottom row). The first two columns correspond to the first video and the last two to the second video.

Table 1 :

 1 Number of images and annotations for each type of crop.

	Label Images Crop annotations Stem annotations
	Maize	1 034	2 095	2 133
	Bean	748	2 820	2 824
	Total	1 782	4 915	4 957

Table 2 :

 2 Object detection performance (in percent) and inference speed (fps) on the NVIDIA Jetson Xavier including video acquisition and post-processing for the three evaluated networks.

	Network	AP	AP50 AP75 AR100 mIoU FPS
	YOLOv4	53.87 89.71 54.59	61.20	80.96	13
	YOLOv4 T	47.28 86.37 44.79	55.36	78.10	95
	YOLOv3 T3L 38.77 82.31 31.64	48.62	75.44	90

Table 3 :

 3 Object detection performance by object class (%). MC: whole maize crop, BC: whole bean crop, MS: maize stem, BS: bean stem.

			AP50			AP
	Network
		MC	BC	MS	BS	MC	BC	MS	BS
	YOLOv4	94.47 95.07 90.60 78.69 69.06 71.11 41.84 33.49
	YOLOv4 T	93.01 95.07 82.63 74.76 60.00 64.18 35.53 29.42
	YOLOv3 T3L 92.02 91.50 77.59 68.13 51.01 51.82 28.09 24.13

The data is available on request to the author.

https://github.com/tzutalin/labelImg

The IoU is also known are the Jaccard index which is a measure of similarity between sets.

[cs] arXiv:1904.07850.

Standard errors for LAcc are provided. For the aggregation algorithm the optimal values for minDets and maxDist parameters found with the grid-search are used.

Temporal Aggregation Performance

Table 4 shows that the temporal aggregation algorithm improves the performance of the detection (e.g. the "Maize Stem" configuration) compared to the non-aggregated case (e.g. the "Maize Stem Aggr" configuration). At the optimal parameter values fixed in the previous section, the F1-score for maize stems is improved by +7.78 % and by 1.25 % for bean stems. The improvement is much higher for maize stems and the aggregation algorithm improves both the recall (+10.40 %) and the precision (+4.83 %) while for bean stems the recall is better (5.71 %) but the precision is lower (-1.73 %). We believe that this contrast can be explained by two factors: (i) as presented in section 4.1 the AP for bean stems is lower than the one of maize stems, thus the aggregation algorithm proceeds on lower quality detections ; and (ii) the crop layout for bean crops is tighter than the maize crop one, making the tracking -which is based on a distance metric-of bean stems less robust to erroneous associations during the "Association" step presented in section 3.3.

While the aggregation algorithm improves the detection performance it also slightly decreases the location accuracy of the detections. The maize stems location accuracy loses 0.22 cm and the bean stems location accuracy is lower by 0.08 cm. However, the location errors are low (6.1 mm on average) and


Valérian Dormoy
email: valerian.dormoy@univ-reims.fr

Jeanne-Marie Perotin

Philippe Gosset

Uwe Maskos

Myriam Polette

Gaëtan Deslée

Nicotinic receptors as SARS-CoV-2 Spike co-receptors?

Keywords: Nicotinic receptors, COVID-19, SARS-CoV-2, hACE2, lung

Nicotinic acetylcholine receptors (nAChRs) play an important role in homeostasis and respiratory diseases. Controversies regarding the association between COVID-19 hospitalizations and smoking suggest that nAChRs may contribute to SARS-CoV-2 respiratory syndrome. We recently detailed the expression and localization of all nAChR subunits in the human lung. Since virus association with nAChRs has been shown in the past, we hypothesize that nAChR subunits act as SARS-CoV-2 Spike co-receptors. Based on sequence alignment analysis, we report domains of high molecular similarities in nAChRs with the binding domain of hACE2 for SARS-CoV-2 Spike protein. This hypothesis supported by in silico pilot data provides a rational for the modelling and the in vitro experimental validation of the interaction between SARS-CoV-2 and the nAChRs.

Background

Severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) is responsible for the global pandemic of coronavirus disease 2019 (COVID-19) [START_REF] Lu | Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding[END_REF][START_REF] Zhu | A Novel Coronavirus from Patients with Pneumonia in China, 2019[END_REF]. Understanding its structure and molecular interactors are crucial to explain the transmission, viral infection, and replication [START_REF] Xu | Pathological findings of COVID-19 associated with acute respiratory distress syndrome[END_REF]. Recent controversies regarding the impact of smoking on COVID-19 pointed to the potential role of nicotinic acetylcholine receptors (nAChRs) in human angiotensin converting enzyme-2 (hACE2) interaction with SARS-CoV-2 [START_REF] Rossato | Current smoking is not associated with COVID-19[END_REF][START_REF] Leung | Smoking, ACE-2 and COVID-19: ongoing controversies[END_REF]. A similar convergence was established regarding the association of nAChRs with rabies virus where the binding was ultimately experimentally demonstrated a few decades ago [START_REF] Gastka | Rabies virus binding to the nicotinic acetylcholine receptor alpha subunit demonstrated by virus overlay protein binding assay[END_REF][START_REF] Lentz | Rabies virus binding to an acetylcholine receptor alpha-subunit peptide[END_REF][START_REF] Donnelly-Roberts | Synthetic peptides of neurotoxins and rabies virus glycoprotein behave as antagonists in a functional assay for the acetylcholine receptor[END_REF][START_REF] Lentz | Synthetic peptides corresponding to sequences of snake venom neurotoxins and rabies virus glycoprotein bind to the nicotinic acetylcholine receptor[END_REF][START_REF] Lentz | Is the acetylcholine receptor a rabies virus receptor?[END_REF].

Since it has been proposed that nAChRs may play a key role in the SARS-CoV-2-mediated inflammatory syndrome [START_REF] Gonzalez-Rubio | Cytokine Release Syndrome (CRS) and Nicotine in COVID-19 Patients: Trying to Calm the Storm[END_REF][START_REF] Lutchman | Could the smoking gun in the fight against COVID-19 be the (rh)ACE-2?[END_REF], additional avenues are currently explored considering that:

(i) nAChRs are widely expressed in organs targeted by the virus including lung, nose, brain, gastrointestinal tract, liver, and smooth muscles [START_REF] Zoli | Neuronal and Extraneuronal Nicotinic Acetylcholine Receptors[END_REF][START_REF] Keiger | Nicotinic Cholinergic Receptor Expression in the Human Nasal Mucosa[END_REF]; (ii) nAChRs are also present on endothelial and platelet cells [START_REF] Schedel | Human Platelets Express Functional α7-Nicotinic Acetylcholine Receptors[END_REF] while altered endothelial function induces vascular thrombosis and microangiopathy in COVID-19 [START_REF] Sardu | Hypertension, Thrombosis, Kidney Failure, and Diabetes: Is COVID-19 an Endothelial Disease? A Comprehensive Evaluation of Clinical and Basic Evidence[END_REF][START_REF] Levi | Coagulation abnormalities and thrombosis in patients with COVID-19[END_REF]; (iii) nAChRs are involved in angiogenesis and arterial diseases [START_REF] Santanam | Nicotinic acetylcholine receptor signaling in atherogenesis[END_REF], while significant vascular angiogenesis was found in lungs from patients who died from COVID-19 [START_REF] Ackermann | Pulmonary Vascular Endothelialitis, Thrombosis, and Angiogenesis in Covid-19[END_REF].

The hypothesis

It was recently suggested that smoking may promote cellular uptake mechanisms of SARS-CoV-2 through CHRNA7 signalling [START_REF] Russo | COVID-19 and smoking: is nicotine the hidden link?[END_REF], and subsequently, CHRNA7 transcripts were correlated with hACE2 levels [START_REF] Leung | COVID-19 and nicotine as a mediator of ACE-2[END_REF], feeding the possible connections between hACE2 and nAChR localizations in airway epithelial cells. In addition, in silico analysis of nAChR ligands suggested an interaction between SARS-CoV-2 and nAChRs [START_REF] Alexandris | Nicotinic cholinergic system and COVID-19: In silico evaluation of nicotinic acetylcholine receptor agonists as potential therapeutic interventions[END_REF][START_REF] Farsalinos | Nicotinic Cholinergic System and COVID-19: In Silico Identification of an Interaction between SARS-CoV-2 and Nicotinic Receptors with Potential Therapeutic Targeting Implications[END_REF][START_REF] Stefano | A Novel Vaccine Employing Non-Replicating Rabies Virus Expressing Chimeric SARS-CoV-2 Spike Protein Domains: Functional Inhibition of Viral/Nicotinic Acetylcholine Receptor Complexes[END_REF], while nicotine exposure may facilitate SARS-CoV-2 infection in vitro [START_REF] Lupacchini | Nicotine Changes Airway Epithelial Phenotype and May Increase the SARS-COV-2 Infection Severity[END_REF]. Nicotinic receptors are ligand-gated ion channels consisting of 5 membrane-spanning subunits selected from 16 proteins (CHRNA1-A7, A9-A10, B1-B4, D, E, G) that all harbour at least one extracellular domain of approximately 200 residues. Going one step further, we suggest that nAChRs may bind to SARS-CoV-2-RBD (Receptor Binding Domain) similar to hACE2 and potentially act as coreceptors for SARS-CoV-2-RBD at least in the airways where we documented the complete atlas of pulmonary nAChR subunits in terms of transcript expression and protein localization [START_REF] Diabasana | Nicotinic Receptor Subunits Atlas in the Adult Human Lung[END_REF]. Transcripts of all 16 subunits were detected in airway epithelial cells or whole lung tissues and only subunits α1/α2/α4/β3/γ were not observed via immunofluorescent stainings performed on formalin-fixed paraffin-embedded lung tissues.

Empirical data

The N-terminal domain of hACE2 binding to the spike (S) glycoprotein SARS-CoV-2-RBD and SARS-CoV-1-RBD has been characterized recently [START_REF] Wang | Structural and Functional Basis of SARS-CoV-2 Entry by Using Human ACE2[END_REF][START_REF] Shang | Structural basis of receptor recognition by SARS-CoV-2[END_REF][START_REF] Wrapp | Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation[END_REF][START_REF] Turoňová | In situ structural analysis of SARS-CoV-2 spike reveals flexibility mediated by three hinges[END_REF]. Twenty-four hACE2 residues were shown to interact with 21 SARS-CoV-2-RBD residues (region K417 to Y505) and 18 SARS-CoV-1-RBD residues (region R426 to Y491) organized in 4 motifs on hACE2 (Figure 1A andB).

Postulating that nAChRs may contain extracellular hACE2-like regions mimicking their interactions with SARS-CoV-2, and since structural data of the 16 nAChR subunits are incomplete, we performed bioinformatics analysis using the Clustal Omega program (Uniprot) to align each of the 16 nAChR subunits' extracellular topological domain protein sequences with the 4 motifs containing the binding residues of hACE2 with SARS-CoV-1 and 2. We identified identical residues in the 4 regions of interests (ROI) as well as conservations between groups of similar properties with a particular focus on residues located within van der Waals (vdw) contact distance between the viral ligand and the receptor (24 red residues,

Figure 1 and Figure 2).

Key residues to form a solid network between hACE2 and SARS-CoV-2-RBD are localized in the motif-1 (S19-Q42) and the motif-4 (K355-R357). The last residue (R393) was not included in the analysis since it was not in the proximity of other key residues involved in complex formation (Figure 1). Interestingly, CHRNA2/A4/A6/A7/A10/B1 presented no overlaps for the 4 motifs (Figure 2 and Table 1).

We established a score based on the percentage of matching residues either in the entire proposed binding region or solely considering the residues involved in vdw contacts (Table 1). Each of the nAChR subunits presented on average 68.5% ([59.1-81.8]) of resemblance with hACE2 binding regions to SARS-CoV-2-RBD and 64.4% ([47.8-73.9]) solely taking into account vdw contacts. The best scores for the 2 parameters were obtained by CHRNA6 (61.4%/73.9%), and CHRNB1 (81.8%/69.5%) (Figure 2 and Table 1). Most of nAChRs possessed mutations in the possible binding domains where a loss or a strong inhibition of the interaction with SARS-CoV-1 and 2 was reported (Table 1, [START_REF] Li | Receptor and viral determinants of SARScoronavirus adaptation to human ACE2[END_REF]). Mutagenesis might favour a beneficial role of nAChRs if they were found to bind SARS-CoV-2 but their levels of expression and specific localizations in various tissues weigh up the pros and cons in a complex and not fully elucidated biological scale.

Consequences of the hypothesis and discussion

Although these findings do not demonstrate that nAChRs can function as co-receptors for the ligand SARS-CoV-2-RBD, here we report a significant molecular similarity between hACE2 binding sites and extracellular domains of nAChRs. Structural biology to match protein structures in 3D would ideally complement this bioinformatics analysis but crystallized structures of all human nAChR subunits are currently not fully resolved. Notably, the protein folding of nAChRs subunit should reveal the critical residues to allow the interaction with SARS-CoV-2-RBD. In addition, nAChRs function as pentamers, the complexity of the composition of the channel may also contribute to the potential interaction between SARS-CoV-2 and one of the subunits where binding would be possible. Interestingly, CHRNA6 and CHRNB1 transcripts were highly expressed in lung tissues and small airway epithelial cells, and the proteins were strongly detected in bronchi strengthening the possibility of the direct interaction between nAChR subunits and SARS-CoV-2 [START_REF] Diabasana | Nicotinic Receptor Subunits Atlas in the Adult Human Lung[END_REF].

Although we focus here on similarities of a potential binding domain with hACE2, we do not exclude the possibility that SARS-CoV-2 may interact with alternative domains on nAChRs.

The hypothesis of SARS-CoV-2 binding to nAChRs was originally based on the identification of similar known nAChR antagonist/agonist motif sequences on SARS-CoV-2 spike protein.

This possible interaction has been sparsely discussed in silico [START_REF] Alexandris | Nicotinic cholinergic system and COVID-19: In silico evaluation of nicotinic acetylcholine receptor agonists as potential therapeutic interventions[END_REF][START_REF] Farsalinos | Nicotinic Cholinergic System and COVID-19: In Silico Identification of an Interaction between SARS-CoV-2 and Nicotinic Receptors with Potential Therapeutic Targeting Implications[END_REF][START_REF] Oliveira | A potential interaction between the SARS-CoV-2 spike protein and nicotinic acetylcholine receptors[END_REF]. Finally, although several pentamers are functionally characterized in the nervous system or muscles, the association of subunits in the airways is only partially elucidated and requires further investigations in the context of COVID-19 [START_REF] Diabasana | Nicotinic Receptor Subunits Atlas in the Adult Human Lung[END_REF]. Conversely, nAChRs may play a role in COVID-19 progression beyond the respiratory system including systemic inflammation and the nervous system.

The modulation of nAChRs in the lung may increase pro-inflammatory cytokine production, a process potentially leading to the "cytokine storm" during infection [START_REF] Borovikova | Vagus nerve stimulation attenuates the systemic inflammatory response to endotoxin[END_REF][START_REF] Pavlov | Neural regulation of immunity: molecular mechanisms and clinical translation[END_REF]. Moreover, α7 nAChR has been involved in the control of inflammation associated with influenza virus infection [START_REF] Gao | Vagal-α7nAChR signaling is required for lung anti-inflammatory responses and arginase 1 expression during an influenza infection[END_REF]. nAChR can also control COVID-19 physiopathology by modulating the Renin-Angiotensin System (RAS) [START_REF] Alifano | Renin-angiotensin system at the heart of COVID-19 pandemic[END_REF]. This nicotine-induced imbalance of the two arms of the RAS is likely responsible for cardio-vascular dysfunction, and acute/chronic lung diseases associated with severe forms of COVID-19.

The potential neurotropism of SARS-CoV-2 has been discussed extensively. SARS-CoV-2 has been identified in the brain of patients [START_REF] Song | Neuroinvasion of SARS-CoV-2 in human and mouse brain[END_REF]. There is by now a substantial amount of literature, and cases, that link neurological and psychiatric sequelae of COVID-19 to an implication of the human brain. The current state of the discussion has been reviewed [START_REF] Iadecola | Effects of COVID-19 on the Nervous System[END_REF].

Of specific interest is a recent description of a case of Parkinson's Disease directly linked to the infection [START_REF] Cohen | A case of probable Parkinson's disease after SARS-CoV-2 infection[END_REF]. The working hypothesis is that SARS-CoV-2 infection can be a "trigger" precipitating the onset of neuropathology. Nicotinic acetylcholine receptors (nAChRs) are important transmembrane neurotransmitter receptors for acetylcholine in the mammalian brain [START_REF] Picciotto | Acetylcholine as a Neuromodulator: Cholinergic Signaling Shapes Nervous System Function and Behavior[END_REF]. They modulate key neuronal functions such as excitability, synaptic transmission, and plasticity. They can change neuronal network states and consequently whole-brain responses to internal and external inputs and have been linked to disease [START_REF] Higley | Neuromodulation by acetylcholine: examples from schizophrenia and depression[END_REF]. They are also exogenously activated by nicotine and are responsible for the events leading to nicotine addiction [START_REF] Changeux | Nicotine addiction and nicotinic receptors: lessons from genetically modified mice[END_REF]. The predominant receptors in the brain are α4β2* containing hetero-, and (α7)5 homo-pentamers [START_REF] Corringer | Nicotinic Receptors at the Amino Acid Level[END_REF]. Being particularly expressed in the cortex, hippocampus, and dopaminergic reward system [START_REF] Besson | Long-term effects of chronic nicotine exposure on brain nicotinic receptors[END_REF], these receptors are considered to be important drug targets [START_REF] Taly | Nicotinic receptors: allosteric transitions and therapeutic targets in the nervous system[END_REF]. The α6 and α5-containing receptors are more restricted, but still very present in sub-cortical "modulatory" areas. The presence of SARS-CoV-2 in the brain parenchyma can therefore potentially interfere with higher cognitive function. nAChRs are potentially implicated in the penetration of SARS-CoV-2 into the central nervous system, as they are expressed in many nerve endings [START_REF] Marchi | Nicotinic modulation of glutamate receptor function at nerve terminal level: a fine-tuning of synaptic signals[END_REF]. The olfactory route has been established in previous types of coronaviruses able to enter nerve endings and use retrograde axonal transport [START_REF] Dubé | Axonal Transport Enables Neuron-to-Neuron Propagation of Human Coronavirus OC43[END_REF].

A possibility is also the vagus nerve. It is well established that it contains most nAChR subunits [START_REF] Cooper | Nicotinic Acetylcholine Receptors on Vagal Afferent Neurons[END_REF].

According to each individual clinical context including but not limited to respiratory diseases and smoking history [START_REF] Cai | Tobacco Smoking Increases the Lung Gene Expression of ACE2, the Receptor of SARS-CoV-2[END_REF][START_REF] Zhang | Expression of the SARS-CoV-2 ACE2 Receptor in the Human Airway Epithelium[END_REF], nAChR binding to SARS-CoV-2-RBD may facilitate virus attachment to cell surface before virus entry, or serve as a decoy either harmful as it would prevent the binding of acetylcholine, or beneficial as it would dampen the virus load that may infect epithelial cells [START_REF] Farsalinos | Editorial: Nicotine and SARS-CoV-2: COVID-19 may be a disease of the nicotinic cholinergic system[END_REF]. Structural biology and functional analysis on SARS-CoV-2-RBD in complex with nAChRs subunits alone or as pentamers is required to resolve the possible interactions and unveil additional therapeutic strategies.

Figure 1 .

 1 Figure 1. hACE2 binding motifs to SARS-CoV-1 and 2 sequences. (A) Localizations of motifs 1 to 4 on hACE2 sequence (Q9BYF1) are highlighted in orange, blue, purple, and green respectively. Red residues are residues involved in van der Waals contact distance between hACE2 and SARS-CoV-1 and 2. Framed residues correspond to residues that are either interacting with SARS-CoV-2-RBD only (E35) or SARS-CoV-1-RBD only (Q325, E329). The residue highlighted in yellow corresponds to the 24th residue involved in van der Waals contact distance but not in the immediate proximity of other residues involved in the binding, therefore it was not included in the analysis. (B) The crystal structure (Protein Data Bank: 6M0J) shows the 4 motifs highlighted in (A) with similar colour codes. The crystal structure of SARS-CoV-2 spike receptor-binding domain bound with hACE2 (PDB ID: 6M0J [52]) was highlighted and exported from the viewer of the Research Collaboratory for Structural Bioinformatics (RCSB; www.rcsb.org).

Figure 2 .

 2 Figure 2. Clustal alignment of nAChRs with hACE2 binding motifs. Data from the UniProt Knowledgebase (UniProtKB, [53]) were used to perform a sequence alignment study of matching residues between hACE2/SARS-CoV-2 binding sites and nAChR protein sequences. The UniProtKB/Swiss-Prot accession numbers of the sequences used for analyzing the similarities are the following: ACE2 (Q9BYF1), CHRNA1 (P02708), CHRNA2 (Q15822), CHRNA3 (P32297), CHRNA4 (P43681), CHRNA5 (P30532), CHRNA6 (Q15825), CHRNA7 (P36544), CHRNA9 (Q9UGM1), CHRNA10 (Q9GZZ6), CHRNB1 (P11230), CHRNB2 (P17787), CHRNB3 (Q05901), CHRNB4 (P30926), CHRNG (P07510), CHRNE (Q04844), CHRND (Q07001). Motifs 1 to 4 were aligned with the 16 nAChR sequences to identify molecular similarities. The nAChR extracellular topological domains' subunit sequences are represented with the partial alignment obtained with Clustal for the 4 motifs. *, position with a single fully conserved residue; :, position with a residue showing conservation between groups of strongly similar properties (scoring >0.5 in the Gonnet PAM 250 matrix); ., position with a residue showing conservation between groups of weakly similar properties (scoring =<0.5 in the Gonnet PAM 250 matrix).

353

  The binding regions correspond to the 4 motifs containing a total of 44 residues (R). The van der Waals (vdw) 354 contacts include 23 residues. ETDL, ETD position on protein sequence; BRO, BR overlaps; Id, fully conserved 355 residues (Number, % of total); Sim, residues with similar properties (Number, % of total); Tot, sums of the fully

Table 1 . Table presenting the sequence similarities with SARS-CoV-hACE2 binding motifs 351 for all nAChRs 352

 1 grey present overlap in binding regions (BR) in Extracellular topological domains (ETD).

					BR in ETD (44 R)	vdw contacts in ETD (23 R)
	CHRN	ETDL	BRO	Id	Sim	Tot	Id	Sim	Tot	Mutagenesis
				(R, (%))	(R, (%))	(R, (%))	(R, (%))	(R, (%))	(R, (%))
	A1	21-255	Y	11 (25)	17 (38.6) 28 (63.6) 4 (17.4)	9 (39.1)	(56.5)	353 K>D 357 R>A
	A2	27-264	N	14 (31.8) 16 (36.4) 30 (68.2) 5 (21.7)	8 (34.8)	(56.5)	353 K>D 357 R>A
	A3	32-240	Y	12 (27.3) 20 (45.4) 32 (72.7) 6 (26.1) 10 (43.4)	(69.5)	31 K>D
	A4	29-242	N	14 (31.8) 17 (38.6) 31 (70.4) 5 (21.7)	8 (34.8)	(56.5)	353 K>D 357 R>A
	A5	23-254	Y	13 (29.6) 18 (40.9) 31 (70.5) 6 (26.1) 11 (47.8)	(73.9)	31 K>D 353 K>D
	A6	26-239	N	16 (36.4)	11 (25)	27 (61.4) 9 (39.1)	8 (34.8)	(73.9)
	A7	23-230	N	12 (27.3) 14 (31.8) 26 (59.1) 5 (21.7)	9 (39.1)	(60.8)
	A9	26-237	Y	10 (22.7)	22 (50)	32 (72.7) 5 (21.7) 12 (52.2)	(73.9)
	A10	25-237	N	12 (27.3) 17 (38.6) 29 (65.9) 6 (26.1)	7 (30.4)	(56.5)	353 K>D 355 D>A
	B1	24-244	N	16 (36.4) 20 (45.4) 36 (81.8) 6 (26.1) 10 (43.4)	(69.5)	353 K>D
	B2	26-233	Y	15 (34.1) 16 (36.4) 31 (70.5) 7 (30.4)	9 (39.1)	(69.5)
	B3	25-232	Y	13 (29.6) 20 (45.4)	33 (75)	4 (17.4) 13 (56.5)	(73.9)	353 K>D
	B4	22-236	Y	14 (31.8) 15 (34.1) 29 (65.9) 5 (21.7) 11 (47.8)	(69.5)	31 K>D
	G	23-240	Y	14 (31.8) 14 (31.8) 28 (63.6) 5 (21.7)	7 (30.4)	(52.1)	353 K>D
	E	21-239	Y	12 (27.3) 17 (38.6) 29 (65.9)	2 (8.7)	9 (39.1)	(47.8)
	D	22-245	Y	13 (29.6) 17 (38.6) 30 (68.2) 6 (26.1) 10 (43.4)	(69.5)	353 K>D
	Highlighted nAChRs in

conserved and similar residues (Number, % of total). Bold numbers indicate the best two hits. Mutagenesis

indicates the positions of the residues that were shown to be involved in a loss of SARS-CoV interaction and 358 that are also present on nAChR sequences.359

Acknowledgments:

We thank the members of the Inserm UMR-S 1250 unit and our collaborators for their helpful comments and insights.

Funding: This work was supported by funding from the University of Reims Champagne-Ardenne (URCA), the French National Institute of Health and Medical Research (Inserm), and the National Agency for Research (ANR, RA-COVID-19).


MD Claude Bachmeyer
email: claude.bachmeyer@tnn.aphp.fr

MD Nicolas Belaube

MD Valeria Loi

Dominique Wendum

MD PhD d Mathieu Gauthé

MD PhD Jean Philippe Haymann

Hypercalcemia and Acute Renal Failure Indicating Peritoneal Sarcoidosis

To the Editor:

CASE REPORT

An 82-year-old Chinese man with a history of diabetes mellitus and chronic renal failure of undetermined origin (steady-state serum creatinine level 140 µmol/l) was referred for fatigue and decreased renal function. The clinical examination was unremarkable. Blood cell count was normal, serum creatinine 253 µmol/L (normal <100) and urea 13.

DISCUSSION

We report here a case of acute kidney failure and hypercalcemia indicating peritoneal sarcoidosis. Sarcoidosis is a systemic disease of unknown origin characterized by noncaseating epithelioid granulomas, requiring ruling out other granulomatous conditions. [START_REF] Valeyre | Sarcoidosis[END_REF] Involvement of any organ is reported but mainly lymph nodes, lungs, skin, and eyes. Peritoneal sarcoidosis is rare, often associated with other organ involvement. [START_REF] Lee | Peritoneal sarcoidosis: a case report[END_REF] Differential diagnosis with peritoneal tuberculosis or carcinomatosis is difficult on clinical features and imaging. Moreover, the association of these conditions is possible. [START_REF] Roh | Abdominal sarcoidosis mimicking peritoneal carcinomatosis[END_REF] Therefore surgery with pathologic examination can be required to establish a definite diagnosis.

Renal involvement of sarcoidosis is usually asymptomatic and consists mainly of granulomatous interstitial nephritis. [START_REF] Baughman | Calcium and vitamin D metabolism in sarcoidosis[END_REF] In our case, interstitial nephritis and nephrocalcinosis were also possible as no renal biopsy was performed and serum creatinine resumed to previous values at six-month follow-up. Hypercalcemia is a frequent complication of sarcoidosis reported in 2-63% of patients, associated with hypercalciuria in most cases. [START_REF] Baughman | Calcium and vitamin D metabolism in sarcoidosis[END_REF] It results from uncontrolled synthesis of 1,25-OH D3 by macrophages responsible for an increased absorption of calcium in the intestine, an increased resorption of calcium in the bone leading to a low PTH value. In our case report, PTH level was indeed low with an unexpected high 1,25-(OH)D3 level in the setting of acute renal failure suggesting a granulomatous disease.

Treatment of hypercalcemia in patients with sarcoidosis depends on the hypercalcemia level. [START_REF] Baughman | Calcium and vitamin D metabolism in sarcoidosis[END_REF] Severe hypercalcemia usually requires intravenous rehydration therapy, corticosteroids

Legend of figures

  7 mmol/L (normal <6.4), Serum protein electrophoresis and dosage of light chains were normal. Serum ionized calcium was increased at 1.47 mmol/L (normal <1.31), phosphate 1.49 mmol/L (normal <1.31), 1,25-(OH)D3 70 pg/mL (normal <67), 25-(OH)D3 40 ng/mL (normal 30-100), parathyroid hormone (PTH) 6.7 ng/L (normal > 8), urine calcium 3.01 mmol/24 h (normal 2.5-7.5). Thoraco-abdominopelvic CT scan with no contrast product showed normal kidney size and no urinary tract obstruction, but hilar lymphadenopathy, micronodules in the upper lobe of right lung, peritoneal nodules and thickening (Figure, A). 2-[ 18 F]FDG PET/CT revealed intense uptake of pleural, pericardial and peritoneal thickening, and of thoracic lymph nodes (Figure, B). Direct search for mycobacteria in sputum and urine, and QuantiFERON Tb Gold Test were negative. Pending explorative laparoscopy for a high suspicion of peritoneal tuberculosis delayed because of Covid19 pandemic conditions, rapid control of serum calcium was obtained under cinacalcet, but fatigue persisted and creatinine level plateaued at 280 µmol/l. The explorative laparoscopy revealed diffuse soft and white nodules, 2 -5 mm in diameter, in the peritoneal cavity (Figure, C), histologic examination displaying non-caseating granuloma with negative bacterial, mycobacterial, fungal cultures. Diagnosis of sarcoidosis was established with no other organ involvement, but the patient declined renal biopsy. Treatment with prednisone was given (1 mg/kg/d) tapered over 6 months, associated with isoniazid and rifampicin during 3 months (until the infectious investigations were definitely negative). This allowed cinacalcet withdrawal within two weeks and return of serum creatinine level to steady level and complete metabolic response on 2-[ 18 F]FDG PET/CT at six-month follow-up (Figure, D).

Figure

  Figure (A) Abdominal CT scan without contrast showing peritoneal thickening (star) and small peritoneal nodules (arrow). (B) 2-[ 18 F]FDG PET/CT showing increased uptake of the

(acting by inhibition 1 alpha-hydroxylase activity of macrophages) and bisphosphonates can be associated. Corticosteroids are also indicated in slightly increased hypercalcemia, when systemic symptoms are present and when severe organ involvement is associated. Our report suggests that calcimimetic agents may also be effective in the control of non-PTH dependent hypercalcemia pending specific treatment.