Ciguatera Food Poisening

Considered as the most frequent food poisoning of marine origine worldwide (~200 000 cases /year).

Symptoms : digestive, neurological (hypotension, bradycardia, paresthesia, cold allodynia, myalgia, arthralgia) and cardiac, potentially lethal.

Long term prevalence and body sensitization effect = increase of symptoms severity upon next exposure, cross effects (ex : alcohol).

September 28, 2020 4

Ciguatera Food Poisoning

SYMPTOMES

History :

First mention : Chen Tsang Chi, chinese doctor of the T'ang dynasty (~650).

From the XVth century, Colomb, Magellan, Cortes, Vasco de Gama describe the disease in the Caribbeans ; Fernandes de Quiros, Cook, Bligh,… in Pacific.

First precise description of a ciguatoxin intoxication : James Cook, Sept. 7th 1774 Nouvelles-Hébride.

The term ciguatera : attributed to cuban Felipe Poey, 1866 for a neurodigestive intoxication due to a marine gasteropode (Livona pica) whose cuban name is « cigua ».

-Among the most powerful marine biotoxines

(

CTXs Targets : Voltage Gated Sodium Channels

-VGSC play key roles in electric signal propagation in excitable tissues (nerves, heart, muscles).

-Mechanism : local activation of VGSCs provokes a transiant depolarization, responsible of the ascendant phase of the action potential.

-CTXs stay fixed and prevent repolarization of VGSCs.

CTXs Detection : problematic

-Current assays : heavy, difficult, long, expensive, and/or unethical (Mouse bioassay, RBA, CBA, HPLC…)

URGENT NEED FOR A SIMPLE DETECTION TOOL !!!

-Can we design a biosensor for Ciguatoxins ? -Yeast based bio sensors : interesting for several reasons

Biosensor strains design

September 28, 2020 14

Ciguatoxins

Cna

Cmd

Expression of reporter protein

Crz

Signal (fluorescence or colorimetry)

Cellular signalling pathway Model Yeast Saccharomyces cerevisiae

Cch1 Ca 2+

Response to Algal extracts

Increasing amounts of algal extracts

THE

  1 microgramme can kill a man) -Produced by the dinoflagellate micro-algae Gambierdiscus toxicus, upon corral reefs damage -Polycyclic polyethers, liposoluble, MW~1100 Da -3 famillies : Pacific = P-CTXs, Caribbean = C-CTXs, Indian Ocean = I-CTXs. No simple detection test available !!! Lewis, Marin Drugs, 2006). CFP Ciguatera Fish Poisoning CSP  Ciguatera Shell Fish Poisoning CTXs Targets : Voltage Gated Sodium Channels (Nicholson and Lewis, Marin Drugs, 2006).

  Approach, very powerfull and precise.Informations on nature and intensity of molecular interactions between key receptors of our neuronal and cardiac cells and ligands of pharma. and env. interest.Ex vivo testing of the inhibition of toxin fixation by potential therapeutic moleculesApply to other toxines (brevetoxines, ciguatoxines, rosmarinic acid, tetrodotoxin…)NeuroSens :Institut Louis Malardé, Papeete (French Polynesia) :Dr Mireille Chinain, Dr Hélène Taiana Darius.IRD Polynésie française : Dr Dominique Laurent.LAAS CNRS Toulouse : Dr Etienne Dague, Dr Emmanuelle Trévisiol, Dr Christophe Furger, Camille Gironde.Collaborations :Thanks for your attention ! Interested in Microbial Adhesion ? Come to BioPhysAdh meeting, 10-11 septembre, Toulouse. website : https://biophysadh.sciencesconf.org V rôles dépolarisation de la membrane du neurone et propagation du PA

September 28, 2020

(Thewes, 2014)

Scientific Questions :

Cell wall permeability/reactivity towards toxins?

Investigation tools : cell wall mutants, tagged toxins.

Membrane target (receptor)? Binding?

Investigation tools : Cloning, SMFS with Atomic Force Microscopy.

Cellular signalisation pathways?

Investigation tools : reporters, Fluorescent proteins, western blotting, RT-PCR.

September 28, 2020 13

Developping yeast-based Biosensors

Perspectives :

Understanding yeast cellular response to CTXs -Cell wall transfert -Competition CTXs -Rosmarinic acid and tetrodotoxin.

-Inhibitory Effect of HIT0 extracts ???? -Activators / repressors / blokers of VGSC and Cch1 -Hyper-active point mutants of Cch1 and Crz1, Chimera Cch1-VGSC

Response Optimisation

Developp a generic approach, apply to other toxins and drugs binding VGSCs.

Future applications : build a bio-sensor : huge interest for Pharmacology and Environmental domains


Ignacio Rocco

Relja Arandjelović

Josef Sivic

Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions

Keywords: Image matching, neighbourhood consensus, sparse CNN

In this work we target the problem of estimating accurately localised correspondences between a pair of images. We adopt the recent Neighbourhood Consensus Networks that have demonstrated promising performance for difficult correspondence problems and propose modifications to overcome their main limitations: large memory consumption, large inference time and poorly localised correspondences. Our proposed modifications can reduce the memory footprint and execution time more than 10×, with equivalent results. This is achieved by sparsifying the correlation tensor containing tentative matches, and its subsequent processing with a 4D CNN using submanifold sparse convolutions. Localisation accuracy is significantly improved by processing the input images in higher resolution, which is possible due to the reduced memory footprint, and by a novel two-stage correspondence relocalisation module. The proposed Sparse-NCNet method obtains state-of-the-art results on the HPatches Sequences and InLoc visual localisation benchmarks, and competitive results in the Aachen Day-Night benchmark.

Introduction

Finding correspondences between images depicting the same 3D scene is one of the fundamental tasks in computer vision [START_REF] Julesz | Towards the automation of binocular depth perception[END_REF][START_REF] Marr | Cooperative computation of stereo disparity[END_REF][START_REF] Mori | An iterative prediction and correction method for automatic stereocomparison[END_REF] with applications in 3D reconstruction [START_REF] Schönberger | Structure-from-motion revisited[END_REF][START_REF] Schönberger | Pixelwise view selection for unstructured multi-view stereo[END_REF][START_REF] Widya | Structure from motion using dense cnn features with keypoint relocalization[END_REF], visual localisation [START_REF] Germain | Sparse-to-Dense Hypercolumn Matching for Long-Term Visual Localization[END_REF][START_REF] Sattler | Benchmarking 6DOF outdoor visual localization in changing conditions[END_REF][START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF] or pose estimation [START_REF] Gao | Complete solution classification for the perspective-three-point problem[END_REF][START_REF] Grabner | 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild[END_REF][START_REF] Persson | Lambda twist: An accurate fast robust perspective three point (P3P) solver[END_REF]. The predominant approach currently consists of first detecting salient local features, by selecting the local extrema of some form of feature selection function, and then describing them by some form of feature descriptor [START_REF] Bay | SURF: Speeded up robust features[END_REF][START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF][START_REF] Rublee | ORB: An efficient alternative to SIFT or SURF[END_REF]. While hand-crafted features such as Hessian affine detectors [START_REF] Mikolajczyk | An affine invariant interest point detector[END_REF] with SIFT descriptors [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] have obtained impressive performance under strong viewpoint changes and constant illumination [START_REF] Mikolajczyk | A comparison of affine region detectors[END_REF], their robustness to illumination changes is limited [START_REF] Mikolajczyk | A comparison of affine region detectors[END_REF][START_REF] Zhou | Evaluating local features for day-night matching[END_REF]. More recently, a variety of trainable keypoint detectors [START_REF] Laguna | Key.Net: Keypoint detection by handcrafted and learned CNN filters[END_REF][START_REF] Lenc | Learning covariant feature detectors[END_REF][START_REF] Mishkin | Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability[END_REF][START_REF] Verdie | TILDE: A temporally invariant learned detector[END_REF] and descriptors [START_REF] Balntas | PN-Net: Conjoined triple deep network for learning local image descriptors[END_REF][START_REF] Balntas | Learning local feature descriptors with triplets and shallow convolutional neural networks[END_REF][START_REF] Han | MatchNet: Unifying feature and metric learning for patch-based matching[END_REF][START_REF] Mishchuk | Working hard to know your neighbor's margins: Local descriptor learning loss[END_REF][START_REF] Tian | L2-Net: Deep learning of discriminative patch descriptor in Euclidean space[END_REF][START_REF] Zagoruyko | Learning to compare image patches via convolutional neural networks[END_REF] have been proposed, with the purpose of obtaining increased robustness over hand-crafted methods. While this approach has achieved some success, extreme illumination changes such as day-to-night matching combined with changes in camera viewpoint remain a challenging open problem [START_REF] Balntas | Workshop in Long-Term Visual Localization under Changing Conditions[END_REF]13,[START_REF] Germain | Sparse-to-Dense Hypercolumn Matching for Long-Term Visual Localization[END_REF]. In particular, all local feature methods, whether handcrafted or trained, suffer from missing detections under these extreme appearance changes.

In order to overcome this issue, the detection stage can be avoided and, instead, features can be extracted on a dense grid across the image. This approach has been successfully used for both place recognition [START_REF] Arandjelović | NetVLAD: CNN architecture for weakly supervised place recognition[END_REF][START_REF] Germain | Sparse-to-Dense Hypercolumn Matching for Long-Term Visual Localization[END_REF][START_REF] Noh | Large-scale image retrieval with attentive deep local features[END_REF][START_REF] Torii | 24/7 place recognition by view synthesis[END_REF] and image matching [START_REF] Rocco | Neighbourhood consensus networks[END_REF][START_REF] Sattler | Benchmarking 6DOF outdoor visual localization in changing conditions[END_REF][START_REF] Widya | Structure from motion using dense cnn features with keypoint relocalization[END_REF]. However, extracting features densely comes with additional challenges: it is memory intensive and the localisation accuracy of the features is limited by the sampling interval of the grid used for the extraction.

In this work we adopt the dense feature extraction approach. In particular, we build on the recent Neighbourhood Consensus Networks (NCNet) [START_REF] Rocco | Neighbourhood consensus networks[END_REF], that allow for jointly trainable feature extraction, matching, and match-filtering to directly output a strong set of (mostly) correct correspondences. Our proposed approach, Sparse-NCNet, seeks to overcome the limitations of the original NCNet formulation, namely: large memory consumption, high execution time and poorly localised correspondences.

Our contributions are the following. First, we propose the efficient Sparse-NCNet model, which is based on a 4D convolutional neural network operating on a sparse correlation tensor, which is obtained by storing only the most promising correspondences, instead of the set of all possible correspondences. Sparse-NCNet processes this sparse correlation tensor with submanifold sparse convolutions [START_REF] Graham | 3D semantic segmentation with submanifold sparse convolutional networks[END_REF] and can obtain equivalent results to NCNet while being several times faster (up to 10×) and requiring much less memory (up to 20×) without decrease in performance compared to the original NCNet model. Second, we propose a two-stage relocalisation module to improve the localisation accuracy of the correspondences output by Sparse-NCNet. Finally, we show that the proposed model significantly outperforms state-of-the-art results on the HPatches Sequences [START_REF] Balntas | HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors[END_REF] benchmark for image matching with challenging viewpoint and illumination changes and the InLoc [START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF] benchmark for indoor localisation and camera pose estimation. Furthermore, we show our model obtains competitive results on the Aachen Day-Night benchmark [START_REF] Sattler | Benchmarking 6DOF outdoor visual localization in changing conditions[END_REF], which evaluates day-night feature matching for the task of camera localisation. An example of the correspondences produced by our method is presented in Fig. 1. Our code and models are available online 1 .

Related work

In this section, we review the relevant related work.

Matching with trainable local features. Most recent work in trainable local features has focused on learning more robust keypoint descriptors [START_REF] Balntas | PN-Net: Conjoined triple deep network for learning local image descriptors[END_REF][START_REF] Balntas | Learning local feature descriptors with triplets and shallow convolutional neural networks[END_REF][START_REF] Han | MatchNet: Unifying feature and metric learning for patch-based matching[END_REF][START_REF] Mishchuk | Working hard to know your neighbor's margins: Local descriptor learning loss[END_REF][START_REF] Tian | L2-Net: Deep learning of discriminative patch descriptor in Euclidean space[END_REF][START_REF] Zagoruyko | Learning to compare image patches via convolutional neural networks[END_REF]. Initially these descriptors were used in conjunction with classic hand-crafted keypoint detectors, such as DoG [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF]. Recently, trainable keypoint detectors where also proposed [START_REF] Laguna | Key.Net: Keypoint detection by handcrafted and learned CNN filters[END_REF][START_REF] Lenc | Learning covariant feature detectors[END_REF][START_REF] Mishkin | Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability[END_REF][START_REF] Verdie | TILDE: A temporally invariant learned detector[END_REF], as well as methods providing both detection and description [12,13,[START_REF] Ono | LF-Net: Learning local features from images[END_REF]42,[START_REF] Yi | LIFT: Learned invariant feature transform[END_REF]. From these, some adopt the classic approach of first performing detection on the whole image and then computing descriptors from local image patches, cropped around the detected keypoints [START_REF] Ono | LF-Net: Learning local features from images[END_REF][START_REF] Yi | LIFT: Learned invariant feature transform[END_REF], while the most recent methods compute a joint representation from which both detections and descriptors are computed [12,13,42]. In most cases, local features obtained by these methods are independently matched using nearest-neighbour search with the Euclidean distance [START_REF] Balntas | PN-Net: Conjoined triple deep network for learning local image descriptors[END_REF][START_REF] Balntas | Learning local feature descriptors with triplets and shallow convolutional neural networks[END_REF][START_REF] Mishchuk | Working hard to know your neighbor's margins: Local descriptor learning loss[END_REF][START_REF] Tian | L2-Net: Deep learning of discriminative patch descriptor in Euclidean space[END_REF], although some works have proposed to learn the distance function as well [START_REF] Han | MatchNet: Unifying feature and metric learning for patch-based matching[END_REF][START_REF] Zagoruyko | Learning to compare image patches via convolutional neural networks[END_REF]. As discussed in the previous section, local features are prone to loss of detections under extreme lighting changes [START_REF] Germain | Sparse-to-Dense Hypercolumn Matching for Long-Term Visual Localization[END_REF]. In order to alleviate this issue, in this work we adopt the usage of dense features, which are described next.

Matching with densely extracted features. Motivated by applications in largescale visual search, others have found that using densely extracted features provides additional robustness to illumination changes compared to local features extracted at detected keypoints, which suffer from low repeatability under strong illumination changes [START_REF] Torii | 24/7 place recognition by view synthesis[END_REF][START_REF] Zhao | Oriented pooling for dense and non-dense rotation-invariant features[END_REF]. This approach was also adopted by later work [START_REF] Arandjelović | NetVLAD: CNN architecture for weakly supervised place recognition[END_REF][START_REF] Noh | Large-scale image retrieval with attentive deep local features[END_REF]. Such densely extracted features used for image retrieval are typically computed on a coarse low resolution grid (e.g. 40 × 30). However, such coarse localisation of the dense features is not an issue for visual retrieval, as the dense features are 1 http://www.di.ens.fr/willow/research/sparse-ncnet/ not directly matched, but rather aggregated into a single image-level descriptor, which is used for retrieval. Recently, densely extracted features have been also employed directly for 3D computer vision tasks, such as 3D reconstruction [START_REF] Widya | Structure from motion using dense cnn features with keypoint relocalization[END_REF], indoor localisation and camera pose estimation [START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF], and outdoor localisation with night queries [START_REF] Germain | Sparse-to-Dense Hypercolumn Matching for Long-Term Visual Localization[END_REF][START_REF] Sattler | Benchmarking 6DOF outdoor visual localization in changing conditions[END_REF]. In these methods, correspondences are obtained by nearest-neighbour search performed on extracted descriptors, and filtered by the mutual nearest-neighbour criterion [START_REF] Oron | Best-buddies similarity-robust template matching using mutual nearest neighbors[END_REF]. In this work, we build on the NCNet method [START_REF] Rocco | Neighbourhood consensus networks[END_REF], where the match filtering function is learnt from data. Different recent methods for learning to filter matches are discussed next.

Learning to filter incorrect matches. When using both local features extracted at keypoints or densely extracted features, the obtained matches by nearestneighbour search contain a certain portion of incorrect matches. In the case of local features, a heuristic approach such as Lowe's ratio test [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] can be used to filter these matches. However the ratio threshold value needs to be manually tuned for each method. To avoid this issue, filtering by mutual nearest neighbours can be used instead [13]. Recently, trainable approaches have also been proposed for the task of filtering local feature correspondences [START_REF] Brachmann | Neural-guided RANSAC: Learning where to sample model hypotheses[END_REF][START_REF] Moo Yi | Learning to find good correspondences[END_REF][START_REF] Sarlin | Superglue: Learning feature matching with graph neural networks[END_REF][START_REF] Zhang | Learning two-view correspondences and geometry using order-aware network[END_REF]]. Yi et al . [START_REF] Moo Yi | Learning to find good correspondences[END_REF] propose a neural-network architecture that operates on 4D match coordinates and classifies each correspondence as either correct or incorrect. Brachmann et al . [START_REF] Brachmann | Neural-guided RANSAC: Learning where to sample model hypotheses[END_REF] propose the Neural-guided RANSAC, which extends the previous method to produce weights instead of classification labels, which are used to guide RANSAC sampling. Zhang et al . [START_REF] Zhang | Learning two-view correspondences and geometry using order-aware network[END_REF] also extend the work of Yi et al . in their proposed Order-Aware Networks, which capture local context by clustering 4D correspondences onto a set of ordered clusters, and global context by processing these clusters with a multi-layer perceptron. Finally, Sarlin et al . [START_REF] Sarlin | Superglue: Learning feature matching with graph neural networks[END_REF] describe a graph neural network followed by an optimisation procedure to estimate correspondences between two set of local features. These methods were specifically designed for filtering local features extracted at keypoint locations and not features extracted on a dense grid. Furthermore, these methods are focused only on learning match filtering, and are decoupled from the problem of learning how to detect and describe the local features.

In this paper we build on the NCNet method [START_REF] Rocco | Neighbourhood consensus networks[END_REF] for filtering incorrect matches, which was designed for dense features. Furthermore, contrary to the above described methods, our approach performs feature extraction, matching and match filtering in a single pipeline.

Improved feature localisation. Recent methods for local feature detection and description which use a joint representation [12,13] as well as methods for dense feature extraction [START_REF] Rocco | Neighbourhood consensus networks[END_REF][START_REF] Widya | Structure from motion using dense cnn features with keypoint relocalization[END_REF] suffer from poor feature localisation, as the features are extracted on a low-resolution grid. Different approaches have been proposed to deal with this issue. The D2-Net method [13] follows the approach used in SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] for refining the keypoint positions, which consists of locally fitting a quadratic function to the feature detection function around the feature position and solving for the extrema. The Superpoint method [12] uses a CNN decoder that produces a one-hot output for each 8 × 8 pixel cell of the input image (in case a keypoint is effectively detected in this region), therefore achieving pixel-level accuracy. Others [START_REF] Widya | Structure from motion using dense cnn features with keypoint relocalization[END_REF] use the intermediate higher resolution features from the CNN to improve the feature localisation, by assigning to each pooled feature the position of the feature with highest L2 norm from the preceding higher resolution map (and which participated in the pooling). This process can be repeated up to the input image resolution.

The relocalisation approach of NCNet [START_REF] Rocco | Neighbourhood consensus networks[END_REF] is based on a max-argmax operation on the 4D correlation tensor of exhaustive feature matches. This approach can only increase the resolution of the output matches by a factor of 2. In contrast, we describe a new two-stage relocalisation module that builds on the approach used in NCNet, by combining a hard relocalisation stage that has similar effects to NCNet's max-argmax operation, with a soft-relocalisation stage that obtains sub-feature-grid accuracy via interpolation.

Sparse Convolutional Neural Networks were recently introduced [START_REF] Graham | Sparse 3D convolutional neural networks[END_REF][START_REF] Graham | Spatially-sparse convolutional neural networks[END_REF] for the purpose of processing sparse 2D data, such as handwritten characters [START_REF] Graham | Spatially-sparse convolutional neural networks[END_REF]; 3D data, such as 3D point-clouds [START_REF] Graham | Sparse 3D convolutional neural networks[END_REF]; or even 4D data, such as temporal sequences of 3D point clouds [START_REF] Choy | 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks[END_REF]. These models have shown great success in 3D pointcloud processing tasks such as semantic segmentation [START_REF] Choy | 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks[END_REF][START_REF] Graham | 3D semantic segmentation with submanifold sparse convolutional networks[END_REF] and point-cloud registration [START_REF] Choy | Fully convolutional geometric features[END_REF][START_REF] Gojcic | Learning multiview 3D point cloud registration[END_REF]. In this work, we use networks with submanifold sparse convolutions [START_REF] Graham | 3D semantic segmentation with submanifold sparse convolutional networks[END_REF] for the task of filtering correspondences between images, which can be represented as a sparse set of points in a 4D space of image coordinates. In submanifold sparse convolutions, the active sites remain constant between the input and output of each convolutional layer. As a result, the sparsity level remains fixed and does not change after each convolution operation. To the best of our knowledge this is the first time these models are applied to the task of match filtering.

Sparse Neighbourhood Consensus Networks

In this section we detail the proposed Sparse Neighbourhood Consensus Networks. We start with a brief review of Neighbourhood Consensus Networks [START_REF] Rocco | Neighbourhood consensus networks[END_REF] identifying their main limitations. Next, we describe our approach which overcomes these limitations.

Review: Neighbourhood Consensus Networks

The Neighbourhood Consensus Network [START_REF] Rocco | Neighbourhood consensus networks[END_REF] is a method for feature extraction, matching and match filtering. Contrary to most methods, which operate on local features, NCNet operates on dense feature maps (f A , f B ) ∈ R h×w×c with c channels, which are extracted over a regular grid of h × w spatial resolution. These are obtained from the input image pair (I A , I B ) ∈ R H×W ×3 by a fully convolutional feature extraction network. The resolution h × w of the extracted dense features is typically 1/8 or 1/16 of the input image resolution H × W , depending on the particular feature extraction network architecture used.

Next, the exhaustive set of all possible matches between the dense feature maps f A and f B is computed and stored in a 4D correlation tensor c AB ∈ R h×w×h×w . Finally, the correspondences in c AB are filtered by a 4D CNN. This network can detect coherent spatial matching patterns and propagate information from the most certain matches to their neighbours, robustly identifying the correct correspondences. This last filtering step is inspired by the neighbourhood consensus procedure [START_REF] Bian | GMS: Grid-based motion statistics for fast, ultra-robust feature correspondence[END_REF][START_REF] Schaffalitzky | Automated scene matching in movies[END_REF][START_REF] Schmid | Local grayvalue invariants for image retrieval[END_REF][START_REF] Sivic | Video Google: A text retrieval approach to object matching in videos[END_REF][START_REF] Zhang | A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry[END_REF], where a particular match is verified by analysing the existence of other coherent matches in its spatial neighbourhood in both images.

Despite its promising results, the original formulation of Neighbourhood Consensus Networks has three main drawbacks that limit its practical application: it is (i) memory intensive, (ii) slow, and (iii) matches are poorly localised. These points are discussed in detail next.

High memory requirements. The high memory requirements are due to the computation of the correlation tensor c AB ∈ R h×w×h×w which stores all matches between the densely extracted image features (f A , f B ) ∈ R h×w×c . Note that the number of elements in the correlation tensor (h × w × h × w) grows quadratically with respect to the number of features (h × w) of the dense feature maps (f A , f B ), therefore limiting the ability to increase the feature resolution. For instance, for dense feature maps of resolution 200×150, the correlation tensor would require by itself 3.4GB of GPU memory in the standard 32-bit float precision. Furthermore, processing this correlation tensor using the subsequent 4D CNN would require more than 50GB of GPU memory, which is much more than what is currently available on most standard GPUs. While 16-bit half-float precision could be used to halve these memory requirements, they would still be prohibitively large.

Long processing time. In addition, Neighbourhood Consensus Networks are slow as the full dense correlation tensor must be processed. For instance, processing the 100 × 75 × 100 × 75 correlation tensor containing matches between a pair of dense feature maps of 100 × 75 resolution takes approximately 10 seconds on a standard Tesla T4 GPU.

Poor match localisation. Finally, the high-memory requirements limit the maximum feature map resolution that can be processed, which in turn limits the localisation accuracy of the estimated correspondences. For instance, for a pair images with 1600 × 1200px resolution, where correspondences are computed using a dense feature map with a resolution of 100 × 75, the output correspondences are localised within an error of 8 pixels. This can be problematic if correspondences are used for tasks such as pose estimation, where small errors in the localisation of correspondences in image-space can yield high camera pose errors in 3D space.

In this paper, we devise strategies to overcome the limitations of the original NCNet method, while keeping its main advantages, such as the usage of dense feature maps which avoids the issue of missing detections, and the processing of multiple matching hypotheses to avoid early matching errors. Our efficient Sparse-NCNet approach is described next.

Sparse-NCNet: Efficient Neighbourhood Consensus Networks

In this section, we describe the Sparse-NCNet approach in detail. An overview is presented in Fig. 2. Similar to NCNet, the first stage of our proposed method consists in dense feature extraction. Given a pair of RGB input images (I A , I B ) ∈ R H×W ×3 , L2-normalized dense features (f A , f B ) ∈ R h×w×c are extracted via a fully convolutional network F (•):

f A = F (I A ), f B = F (I B ). (1)
Then, these dense features are matched and stored into a sparse correlation tensor. Contrary to the original NCNet formulation, where all the pairwise matches between the dense features are stored and processed, we propose to keep only the top K matches for a given feature, measured by the cosine similarity. In detail, each feature f A ij: from image A at position (i, j) is matched with its K nearest-neighbours in f B , and vice versa. The one-sided sparse correlation tensor, matching from image A to image B (A → B) is then described as:

c A→B ijkl = f A ij: , f B kl: if f B kl: within K-NN of f A ij: 0 otherwise . (2)
To make the sparse correlation map invariant to the ordering of the input images, we also perform this in the reverse direction (B → A), and add the two one-sided correlation tensors together to obtain the final (symmetric) sparse correlation tensor :

c AB = c A→B + c B→A . (3)
This tensor uses a sparse representation, where only non-zero elements need to be stored. Note that the number of stored elements is, at most,

h × w × K × 2
which is in practice much less than the h × w × h × w elements of the dense correlation tensor, obtaining great memory savings in both the storage of this tensor and its subsequent processing. For example, for a feature map of size 100 × 75 and K = 10, the sparse representation takes 3.43MB vs. 215MB of the dense representation, resulting in a 12× reduction of the processing time. In the case of feature maps with 200 × 150 resolution, the sparse representation takes 13.7MB vs. 3433MB for the dense representation. This allows Sparse-NCNet to also process feature maps at this resolution, something that was not possible with NCNet due to the high memory requirements. The proposed sparse correlation tensor is a compromise between the common procedure of taking the best scoring match and the approach taken by NCNet, where all pairwise matches are stored.

In this way, we can keep sufficient information in order avoid early mistakes, while keeping low memory consumption and processing time.

Then the sparse correlation tensor is processed by a permutation-invariant CNN ( N (•)), to produce the output filtered correlation map cAB :

cAB = N (c AB ). ( 4
)
The permutation invariant CNN N (•) consists of applying the 4D CNN N (•) twice such that the same output matches are obtained regardless of the order of the input images:

N (c AB ) = N (c AB ) + N (c AB ) T T , (5)
where by transposition we mean exchanging the first two dimensions with the last two dimensions, which correspond to the coordinates of the two input images. The 4D CNN N (•) operates on the 4D space of correspondences, and is trained to perform the neighbourhood consensus filtering. Note that while N (•) is a sparse CNN using submanifold sparse convolutions [START_REF] Graham | 3D semantic segmentation with submanifold sparse convolutional networks[END_REF], where the active sites between the sparse input and output remain constant, the convolution kernel filters are dense (i.e. hypercubic). While in the original NCNet method, a soft mutual nearest-neighbour operation M (•) is also performed, we have removed it as we noticed its effect was not significant when operating on the sparse correlation tensor. From the output correlation tensor cAB , the output matches are computed by applying argmax at each coordinate:

(i, j), (k, l) a match if      (i, j) = argmax (a,b) cAB abkl , or (k, l) = argmax (c,d) cAB ijcd , (6)
where (i, j) is the match coordinate in the sampling grid of f A , and (k, l) is the match coordinate in the sampling grid of f B .

Match relocalisation by guided search

While the sparsification of the correlation tensor presented in the previous section allows processing higher resolution feature maps, these are still several times smaller in resolution than the input images. Hence, they are not suitable for applications that require (sub)pixel feature localisation such as camera pose estimation or 3D-reconstruction.

To address this issue, in this paper we propose a two-stage relocalisation module based on the idea of guided search. The intuition is that we search for accurately localised matches on 2h × 2w resolution dense feature maps, guided by the coarse matches output by Sparse-NCNet at h × w resolution. For this, dense features are first extracted at twice the normal resolution ( f A , f B ) ∈ R 2h×2w×c , which is done by upsampling the input image by 2× before feeding it into the feature extraction CNN F (•). Note that these higher resolution features are used for relocalisation only, i.e. they are not used to compute the correlation tensor or processed by the 4D CNN for match-filtering, which would be too expensive. Then, these dense features are downsampled back to the normal h × w resolution by applying a 2×2 max-pooling operation with a stride of 2, obtaining f A and f B . These low resolution features (f A , f B ) ∈ R h×w×c are processed by Sparse-NCNet, which outputs matches in the form m = (i, j), (k, l) , with the coordinates (i, j) and (k, l) indicating the position of the match in f A and f B , respectively, as described by [START_REF] Balntas | Learning local feature descriptors with triplets and shallow convolutional neural networks[END_REF].

Having obtained the output matches in h × w resolution, the first step (hard relocalisation) consists in finding the best equivalent match in the 2h × 2w resolution grid. This is done by analysing the matches between two local crops of the high resolution features f A and f B , and keeping the highest-scoring one. The second step (soft relocalisation) then refines this correspondence further, by obtaining a sub-feature accuracy in the 2h × 2w grid. These two relocalisation steps are illustrated in Fig. 3, and are now described in detail.

Hard relocalisation. The first step is hard relocalisation, which can improve localisation accuracy by 2×. For each match m = (i, j), (k, l) , the 2× upsampled coordinates (2i, 2j), (2k, 2l) are first computed, and 2 × 2 local feature crops f A,L , f B,L ∈ R 2×2×c are sampled around these coordinates from the high resolution feature maps f A and f B :

f A,L = ( f A ab: ) 2i≤a≤2i+1 2j≤b≤2j+1 , (7)
and similarly for f B,L . This is done using a ROI-pooling operation [START_REF] Girshick | Fast R-CNN[END_REF]. Finally, exhaustive matches between the local feature crops f A,L and f B,L are computed, and the output of the hard relocalisation module is the displacement associated with the maximal matching score:

∆m h = (δi, δj), (δk, δl) = argmax (a,b),(c,d) f A,L ab: , f B,L cd: . (8)
Then, the final match location from the hard relocalisation stage is computed as:

m h = 2m + ∆m h = (2i + δi, 2j + δj), (2k + δk, 2l + δl) . (9)
Note that the relocalised matches m h are defined in a 2h × 2w grid, therefore obtaining a 2× increase in localisation accuracy with respect to the initial matches m, which are defined in a h × w grid. Also note that while the implementation is different, the effect of the proposed hard relocalisation is similar to the maxargmax operation used in NCNet [START_REF] Rocco | Neighbourhood consensus networks[END_REF], while being more memory efficient as it avoids the computation of the a dense correlation tensor in high resolution.

Soft relocalisation. The second step consists of a soft relocalisation operation that obtains sub-feature localisation accuracy in the 2h × 2w grid of high resolution features f A and f B . For this, new 3 × 3 local feature crops ( f A,L , f B,L ) ∈ R 3×3×c are sampled around the coordinates of the estimated matches m h from the previous relocalisation stage. Note that no upsampling of the coordinates is done in this case, as the matches are already in the 2h × 2w range. Then, soft relocalisation displacements are computed by performing the softargmax operation [START_REF] Yi | LIFT: Learned invariant feature transform[END_REF] on the matching scores between the central feature of f A,L and the whole of f B,L , and vice versa:

∆m s = (δi, δj), (δk, δl) where      (δi, δj) = softargmax (a,b) f A,L ab: , f B,L 11:
(δk, δl) = softargmax

(c,d) f A,L 11: , f B,L cd:
.

The intuition of the softargmax operation is that it computes a weighted average of the candidate positions in the crop where the weights are given by the softmax of the matching scores. The final matches from soft relocalisation are obtained by applying the soft displacements to the matches from hard relocalisation:

m s = m h + ∆m s .

Experimental evaluation

We evaluate the proposed Sparse-NCNet method on three different benchmarks: (i) HPatches Sequences, which evaluates the matching task directly, (ii) InLoc, which targets the problem of indoor 6-dof camera localisation and (iii) Aachen Day-Night, which targets the problem of outdoor 6-dof camera localisation with challenging day-night illumination changes. We first present the implementation details followed by the results on these three benchmarks.

Implementation details. We train the Sparse-NCNet model following the training protocol from [START_REF] Rocco | Neighbourhood consensus networks[END_REF]. We use the IVD dataset with the weakly-supervised mean matching score loss for training [START_REF] Rocco | Neighbourhood consensus networks[END_REF]. The 4D CNN N (•) has two sparse convolution layers with 3 4 sized kernels, with 16 output channels in the hidden layer. A value of K = 10 is used for computing c AB (3). The model is implemented using PyTorch [START_REF] Paszke | Automatic differentiation in PyTorch[END_REF], MinkowskiEngine [START_REF] Choy | 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks[END_REF] and Faiss [START_REF] Johnson | Billion-scale similarity search with GPUs[END_REF], and trained for 5 epochs using Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] with a learning rate of 5 × 10 -4 . A pretrained ResNet-101 (up to conv 4 23) with no strided convolutions in the last block is used as the feature extractor F (•). This feature extraction model is not finetuned as the training dataset is small (3861 image pairs) and that would lead to overfitting and loss of generalisation. The softargmax operation in [START_REF] Choy | 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks[END_REF] uses a temperature value of 10.

HPatches Sequences

The HPatches Sequences [START_REF] Balntas | HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors[END_REF] benchmark assesses the matching accuracy under strong viewpoint and illumination variations. We follow the evaluation procedure from [13], where 108 image sequences are employed, each from a different planar scene, and each containing 6 images. The first image from each sequence is matched against the remaining 5 images. The benchmark employs 56 sequences with viewpoint changes, and constant illumination conditions, and 52 sequences with illumination changes and constant viewpoint. The metric used for evaluation is the mean matching accuracy (MMA) [13]. Further details about this metric are provided in Appendix B.

Sparse-NCNet vs. NCNet. In Fig. 4 we compare the matching quality of the proposed Sparse-NCNet model and the NCNet model. We first compare both methods under equal conditions, both without relocalisation (methods A1 vs. A2), and with hard relocalisation only (methods B1 vs. B2). The results in Fig. 4 show that Sparse-NCNet can obtain significant reductions in processing time and memory consumption, while keeping almost the same matching performance. In addition, our proposed two-stage relocalisation module can improve performance with a minor increase in processing time (methods C1 vs. B1). Finally, the reduced memory consumption allows for processing of higher resolution 200 × 150 feature maps, which is not possible for NCNet. Our proposed method in higher resolution (method C2) produces the best results while still being 30% faster and 3× more memory efficient than the best NCNet variant (method B2). Sparse-NCNet can obtain equivalent results to NCNet, both without relocalisation (c.f . A1 vs. A2), and with hard relocalisation (H) (c.f . B1 vs. B2), while greatly reducing execution time and memory consumption. The proposed two-stage relocalisation (H+S) brings an improvement in matching accuracy with a minor increase in execution time (c.f . C1 vs. B1). Finally, the reduced memory consumption in Sparse-NCNet allows for processing in higher resolution, which produces the best results, while still being faster and more memory efficient than NCNet (c.f . C2 vs. B2).

Sparse-NCNet vs. state-of-the-art methods. In addition, we compare the performance of Sparse-NCNet against several methods, including state-of-the-art trainable methods such as SuperPoint [12], D2-Net [13] or R2D2 [42]. The meanmatching accuracy results are presented in Fig. 5. For all other methods, the top 2000 features points where selected from each image, and matched enforcing mutual nearest-neighbours, yielding approximately 1000 correspondences per image pair. For Sparse-NCNet, the top 1000 correspondences where selected for each image pair, for a fair comparison. Sparse-NCNet obtains the best results for the illumination sequences for thresholds higher than 4 pixels, and in the viewpoint sequences for all threshold values. Sparse-NCNet obtains the best results overall, with a large margin over the state-of-the-art R2D2 method. We believe this could be attributed to the usage of dense descriptors (which avoid the loss of detections) together with an increased matching robustness from performing neighbourhood consensus. Qualitative examples and comparison with other methods are presented in Appendix B.

InLoc benchmark

The InLoc benchmark [START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF] targets the problem of indoor localisation. It contains a set of database images of a building, obtained with a 3D scanner, and a set of query images from the same building, captured with a cell-phone several months later. The task is then to obtain the 6-dof camera positions of the query images. We follow the DensePE approach proposed [START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF] to find the top 10 candidate database images for each query, and employ Sparse-NCNet to obtain matches between them. Then, we follow again the procedure in [START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF] to obtain the final [START_REF] Mishchuk | Working hard to know your neighbor's margins: Local descriptor learning loss[END_REF][START_REF] Mishkin | Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability[END_REF] 39.8 61.2 77.6 DELF [START_REF] Noh | Large-scale image retrieval with attentive deep local features[END_REF] 38.8 62.2 85.7 SuperPoint [12] 42.8 57.1 75.5 D2-Net [13] 44.9 66.3 88.8 D2-Net (Multi-scale) [13] 44.9 64.3 88.8 R2D2 (patch = 16) [42] 44.9 67.3 87.8 R2D2 (patch = 8) [42] 45.9 66.3 88.8 Sparse-NCNet (H, 200 × 150) 44.9

68.4 86.7 estimated 6-dof query pose, which consists of running PnP [START_REF] Gao | Complete solution classification for the perspective-three-point problem[END_REF] followed by dense pose verification [START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF].

The results are presented in Fig. 10. First, we observe that Sparse-NCNet with hard relocalisation (H) and a resolution of 100 × 75 obtains equivalent results to NCNet (methods B vs. C), while being almost 7× faster and requiring 6.5× less memory, confirming what was already observed in the HPatches benchmark (c.f . B1 vs. B2 in Fig. 4a). Moreover, our proposed Sparse-NCNet method with two-stage relocalisation (H+S) in the higher 200 × 150 resolution (method A) obtains the best results and sets a new state-of-the-art for this benchmark. Recall that it is impossible to use the original NCNet on the higher resolution due to its excessive memory requirements. More qualitative examples are included in Appendix C.

Aachen Day-Night

The Aachen Day-Night benchmark [START_REF] Sattler | Benchmarking 6DOF outdoor visual localization in changing conditions[END_REF] targets 6-dof outdoor camera localisation under challenging illumination conditions. It contains 98 night-time query images from the city of Aachen, and a shortlist of 20 day-time images for each night-time query. Sparse-NCNet is used to obtain matches between the query and images in the short-list. The resulting matches are then processed by the 3D reconstruction software COLMAP [START_REF] Schönberger | Structure-from-motion revisited[END_REF] to obtain the estimated query poses.

The results are presented in Table 1. Sparse-NCNet presents a similar performance to the state-of-the-art methods D2-Net [13] and R2D2 [42]. Note that the results of these three different methods differ by only a few percent, which represents only 1 or 2 additionally localised queries, from the 98 total night-time queries. The proposed Sparse-NCNet obtains state-of-the-art results for the 1m and 5 • threshold, being able to localise 68.4% of the queries (67 out of 98). One qualitative example from this benchmark is presented in Fig. 1, and more are included in Appendix D.

Conclusion

In this paper we have developed Sparse Neighbourhood Consensus Networks for efficiently estimating correspondences between images. Our approach overcomes the main limitations of the original Neighbourhood Consensus Networks that demonstrated promising results on challenging matching problems, making these models practical and widely applicable. The proposed model jointly performs feature extraction, matching and robust match filtering in a computationally efficient manner, outperforming state-of-the-art results on two challenging matching benchmarks. The entire pipeline is end-to-end trainable, which opens-up the possibility for including additional modules for specific downstream problems such as camera pose estimation or 3D reconstruction.

B HPatches Sequences benchmark

Mean matching accuracy. The Mean Matching Accuracy (MMA) metric is used in the HPatches Sequences benchmark to assess the fraction of correct matches under different tolerance thresholds. It is computed in the following way:

MMA {(p A i , p B i )} N i=1 ; t = N i=1 1 >0 t -T H (p A i ) -p B i N , (11)
where {(p A i , p B i )} N i=1 is the set of matches to be evaluated, T H (p A i ) is the warped point p A i using the ground-truth homography H, 1 >0 is the indicator function for positive numbers, and t is the chosen tolerance threshold (in pixels).

Additional qualitative results are presented in Figures 8 and9. We compare the MMA of Sparse-NCNet with the state-of-the-art methods SuperPoint [12], D2-Net [13] and R2D2 [42], which are trainable methods for joint detection and description on local features. The correctly matched points are shown in green, while the incorrectly matched ones are shown in red, for a threshold value t = 3 pixels. For the proposed Sparse-NCNet, results are presented for two different numbers of matches, 2000 and 6000. Results show that our method produces the largest fraction of correct matches, even when considering as many as 6000 correspondences. In particular, note that our method is able to produce a large amount of correct correspondences even under strong illumination changes, as shown in Fig. 9. Furthermore, note that the nature of the correspondences produced by Sparse-NCNet is different from those of local feature methods. While local feature methods can only produce correspondences on the detected points, which are the local extrema of a particular feature detection function, our method produces densely packed sets of correspondences. This results from Sparse-NCNet's propagation of information in local neighbourhoods, as discussed in Appendix. A.

SuperPoint [12] D2-Net [13] R2D2 [42] Sparse-NCNet, 2k Sparse-NCNet, 6k

C InLoc benchmark

We present additional qualitative results from the InLoc indoor localisation benchmark [START_REF] Taira | InLoc: Indoor visual localization with dense matching and view synthesis[END_REF] in Fig. 10, where the task is to estimate the 6-dof pose of a query image within a large university building. Each image pair is composed of a query image (top row) captured with a cell-phone and a database image (middle row), captured several months earlier with a 3D scanner. Note that the illumination conditions in the two types of images are different. Furthermore, because of the time difference between both images, some objects may have been displaced (e.g. furniture) and some aspects of the scene may have changed (e.g. wall decoration). For ease of visualisation, we overlay only the top 500 correspondences for each image pair, which appear in green. These correspondences have not been geometrically verified, and therefore contain a certain fraction of incorrect matches. Note however, that most matches are coherent and the few incorrect outliers are likely to be removed when running RANSAC [START_REF] Fischler | Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[END_REF] within the PnP pose solver [START_REF] Gao | Complete solution classification for the perspective-three-point problem[END_REF], therefore obtaining a good pose estimate. Also note how Sparse-NCNet is able to obtain correspondences in low textured areas such as walls or ceilings, or on repetitive patterns such as carpets.

D Aachen day-night benchmark

Additional qualitative results from the Aachen-day benchmark are shown in Fig. 11. We show several image pairs composed of night query images (top) and their top matching database images (bottom), according to the average matching score of Sparse-NCNet. For each image pair, we overlay the top 500 correspondences obtained with Sparse-NCNet. Note that these correspondences were not geometrically verified by any means. Nevertheless, as seen in Fig. 11, most correspondences are coherent and seem to be correct, despite the strong changes in illumination between night and day images.

  (a) Input images (b) Output matches (c) Match confidence

Fig. 1 :

 1 Fig. 1: Correspondence estimation with Sparse-NCNet. Given an input image pair (a), we show the raw output correspondences produced by Sparse-NCNet (b) which contain groups of spatially coherent matches. These groups tend to form around highly-confident matches, which are shown in yellow shades (c) (see Appendix A for a discussion on this behaviour and additional examples).

Fig. 2 :

 2 Fig. 2: Overview of Sparse-NCNet. From the dense feature maps f A and f B , their top K matches are computed and stored in the one-sided sparse 4D correlation tensors c A→B and c B→A , which are later combined to obtain the symmetric sparse correlation tensor c AB . The raw matching score values in c AB are processed by the 4D Sparse-NCNet N (•) producing the output tensor cAB of filtered matching scores.

Fig. 3 :

 3 Fig. 3: Two-stage relocalisation module. (a) The hard relocalisation step allows to increase by 2× the localisation accuracy of the matches m outputted by Sparse-NCNet, which are defined on the h × w feature maps f A and f B . This is done by keeping the most similar match m h between two 2 × 2 local features f A,L and f B,L , cropped from the 2h × 2w feature maps f A and f B . (b) The soft relocalisation step then refines the position of these matches in the 2h × 2w grid, by computing sub-feature-grid soft localisation displacements based on the softargmax operation.

Fig. 4 :

 4 Fig.4: Sparse-NCNet vs. NCNet on HPatches. Sparse-NCNet can obtain equivalent results to NCNet, both without relocalisation (c.f . A1 vs. A2), and with hard relocalisation (H) (c.f . B1 vs. B2), while greatly reducing execution time and memory consumption. The proposed two-stage relocalisation (H+S) brings an improvement in matching accuracy with a minor increase in execution time (c.f . C1 vs. B1). Finally, the reduced memory consumption in Sparse-NCNet allows for processing in higher resolution, which produces the best results, while still being faster and more memory efficient than NCNet (c.f . C2 vs. B2).

  ++[START_REF] Mishchuk | Working hard to know your neighbor's margins: Local descriptor learning loss[END_REF][START_REF] Mishkin | Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability[END_REF] Affine Det. + RootSIFT[START_REF] Mikolajczyk | An affine invariant interest point detector[END_REF][START_REF] Arandjelović | Three things everyone should know to improve object retrieval[END_REF]

Fig. 5 :

 5 Fig. 5: Sparse-NCNet vs. state-of-the-art on HPatches. The MMA of Sparse-NCNet and several state-of-the-art methods is shown. Sparse-NCNet obtains the best results overall with a large margin over the recent R2D2 method.

Fig. 6 :

 6 Fig. 6: Results on the InLoc benchmark for long-term indoor localization. (Left) Our proposed method (A) obtains state-of-the-art results on this benchmark. (Right) Our method obtains correspondences in challenging indoor scenes with repetitive patterns and low amount of texture. Top: query images. Bottom: matched database images captured from different viewpoints. Correspondences produced by our approach are overlaid in green. The query and database images were taken several months apart.

Fig. 7 :

 7 Fig. 7: Insights about Sparse-NCNet. We show the top N matches between each pair of images for different values of N . The strength of the match is shown by color (the more yellow the stronger). Please note how new matches tend to appear close to high scoring matches, demonstrating the propagation of information in Sparse-NCNet.

Fig. 8 :

 8 Fig. 8: HPatches qualitative results (viewpoint). We present the results of Sparse-NCNet, along with several state-of-the-art methods. The correct correspondences are shown in green, and the incorrect ones in red for a threshold t = 3px. Below each pair we indicate the fraction of correct matches (both in percentage and absolute values). Our method is presented for both the top 2K matches and the top 6K matches, and it obtains the largest fraction of correct matches for both cases. Examples are from the viewpoint sequences.

Fig. 10 :

 10 Fig. 10: InLoc qualitative results. For each image pair, we show the top 500 matches produced by Sparse-NCNet between the query image (top row) and database image (middle row). In addition we show the rendered scene from the estimated query 6-dof pose (bottom row), obtained by running RANSAC+PnP[14, 15] on our matches. Note these rendered images are well aligned with the query images, demonstrating that the estimated poses have low translation and rotation errors.

Fig. 11 :

 11 Fig. 11: Aachen day-night results. We show the top 500 correspondences obtained by Sparse-NCNet between the night query image (top) and the database day image (bottom). Note that the large majority of matches are correct, despite the strong illumination changes.

Table 1 :

 1 Results on Aachen Day-Night. Sparse-NCNet is able to localise a similar number of queries than R2D2 and D2-Net.

		Correctly localised queries (%)
	Method	0.5m, 2 •	1.0m, 5 •	5.0m, 10 •
	RootSIFT [29, 2]	36.7	54.1	72.5
	DenseSfM [46]	39.8	60.2	84.7
	HessAffNet + HN ++

  8% (727/1057) 45.7% (1170/2561) 64.8% (1567/2420) 85.6% (1712/2000) 77.6% (4656/6000)

41.6% (558/1342) 30.6% (424/1386) 72.7% (722/993) 99.7% (1994/2000) 99.2% (5952/6000) SuperPoint

[12]

D2-Net

[13]

R2D2

[42]

Sparse-NCNet, 2k Sparse-NCNet, 6k

68.

Acknowledgements. This work was partially supported by ERC grant LEAP No. 336845, the European Regional Development Fund under project IMPACT (reg. no. CZ.02.1.01/0.0/0.0/15 003/0000468), Louis Vuitton ENS Chair on Artificial Intelligence, and the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute).

Appendices

In this appendices we present insights about the way Sparse-NCNet operates (Appendix A) and additional qualitative results on the HPatches Sequences (Appendix B), InLoc (Appendix C) and Aachen Day-Night (Appendix D) benchmarks.

A Insights about Sparse-NCNet

In this section we provide additional insights about the way Sparse-NCNet operates, which differs from traditional local feature detection and matching methods. In Fig. 7 we plot the top N matches produced by Sparse-NCNet for different values of N : 100 (left column), 400 (middle column) and 1600 (right column). By comparing the middle column (showing the top 400 matches) with the left column (showing the top 100), we can observe that many of the additional 300 matches are close to the initial 100 matches. A similar effect is observed when comparing the right column (top 1600 matches) with the middle column (top 400 matches). This could be attributed to the fact that Sparse-NCNet propagates information from the strongest matches to their neighbours. In this sense, strong matches, which are typically non-ambiguous ones, can help in matching their neighbouring features, which might not be so discriminative.


Tanya E S Dahms

André Körning

Esther Ndlovu

Hélène Martin-Yken

Etienne Dague

Decon

Deconstructing and reconstructing the yeast cell wall: A top-down, bottom-up approach

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Deconstructing and reconstructing the yeast cell wall: A top-down, bottom-up approach

T. E. S. Dahms Models of the Saccharomyces cerevisiae cell wall include a complex interplay of polysaccharides and proteins, with -1,3-and --1,6-glucans representing major components in terms of dry mass. The cell wall network is decorated with proteins bound to the glucans through different types of molecular interactions, including modified GPI-anchors. Chitin is another key component, which appears at the inner surface and is concentrated in bud scars, while the outer surface is coated with mannoproteins.

There is a great body of evidence supporting such models; however, the precise contribution of each molecular component to the overall cell wall viscoelasticity and integrity is still an area of active investigation. The aim of this work was to examine the structural and physical properties of the S. cerevisiae cell wall during its enzymatic degradation and subsequent regeneration, using atomic force microscopy (AFM) quantitative (QI™) and high-speed (HS) imaging. We studied a wild-type strain and two well characterized cell wall mutants of S. cerevisiae, kre6, deleted in a gene required for -1,6glucan synthesis and knr4, deprived of a cell wall signaling protein notably involved in the transcriptional control of chitin synthase genes.

Clear differences between the mutant and wild type cell walls were already distinguishable by phase contrast microscopy. Spheroplasts of each strain were prepared using zymolyase and lyticase, but imaging of the perfectly round spheroplasts proved challenging, even when housed in the wells of plasma-treated PDMS stamps coated with Cell Tak. Here we present HS-and QI-AFM imaging of S. cerevisiae exposed to lyticase, showing changes to cell wall structure and physical properties in real time.