Samuel Felton

Élisa Fromont

Eric Marchand

Deep metric learning for visual servoing: when pose and image meet in latent space

We propose a new visual servoing method that controls a robot's motion in a latent space. We aim to extract the best properties of two previously proposed servoing methods: we seek to obtain the accuracy of photometric methods such as Direct Visual Servoing (DVS), as well as the behavior and convergence of pose-based visual servoing (PBVS). Photometric methods suffer from limited convergence area due to a highly non-linear cost function, while PBVS requires estimating the pose of the camera which may introduce some noise and incurs a loss of accuracy. Our approach relies on shaping (with metric learning) a latent space, in which the representations of camera poses and the embeddings of their respective images are tied together. By leveraging the multimodal aspect of this shared space, our control law minimizes the difference between latent image representations thanks to information obtained from a set of pose embeddings. Experiments in simulation and on a robot validate the strength of our approach, showing that the sought out benefits are effectively found.

I. INTRODUCTION

Visual servoing (VS) is the task of controlling the motion of a robot in order to reach a desired goal or a desired pose using only visual information extracted from an image stream [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF]. The camera can be mounted on the robot's end effector or directly observing the robot. Visual servoing usually requires the extraction and the tracking of visual information (usually geometric features) from the image in order to design the control law. The choice of features is a crucial aspect of VS, as it impacts the servoing behaviour (namely the convergence to the target pose and the trajectory in 3D space). Geometric features are usually split into two categories. The first one, Image-Based VS (IBVS), uses 2D primitives in the image space, such as points [START_REF] Weiss | Dynamic visual servo control of robots. an adaptive image based approach[END_REF], lines [START_REF] Andreff | Visual servoing from lines[END_REF] or moments [START_REF] Chaumette | Image moments: a general and useful set of features for visual servoing[END_REF], in order to create the control law. The second category, named Pose-Based VS (PBVS) [START_REF] Wilson | Relative end-effector control using cartesian position-based visual servoing[END_REF], [START_REF] Chaumette | Image moments: a general and useful set of features for visual servoing[END_REF], uses image information to estimate the camera pose, which can be used to directly control the robot's motion. The PBVS control law is often seen as the most practical and optimal one: if the camera pose is perfectly estimated, then the scheme is globally convergent and the trajectory is the shortest path to the goal pose, both in translation and rotation.

In all cases, geometric features must be extracted, as well as matched between current and desired images. As this is an error-prone process, another way of performing VS has developed. By using photometric features, such as raw pixel intensities, feature extraction is avoided [START_REF] Collewet | Visual servoing set free from image processing[END_REF]. In Direct VS (DVS), the difference between pixel intensities is minimized, which leads to very accurate positioning, but with Authors are with Univ Rennes, Inria, CNRS, Irisa, Rennes, France Email: {samuel.felton, elisa.fromont, eric.marchand}@irisa.fr an unpredictable trajectory and a small convergence domain, since the cost function to minimize is highly non-linear. To alleviate the latter problem, a solution is to represent images by lower dimensional features that better correlate to the pose. Multiple representations have been studied: [START_REF] Bakthavatchalam | A Direct Dense Visual Servoing Approach using Photometric Moments[END_REF] expresses an image with photometric moments. In [START_REF] Marchand | Direct visual servoing in the frequency domain[END_REF], servoing is performed in the frequency domain, where only the smoothly varying low-frequency information is preserved. A similar, learning-based approach was proposed in [START_REF] Marchand | Subspace-based visual servoing[END_REF], where principal component analysis is used to project on a subspace, which maximally preserves the information of an image set. Inspired by this work, we previously proposed AEVS [START_REF] Felton | Visual Servoing in Autoencoder Latent Space[END_REF], projecting images in the latent space of an autoencoder.

Between all these VS methods, a trade-off becomes apparent: the easier feature extraction is, the harder servoing becomes. On one end of the spectrum lies DVS, with no extraction but a highly non-linear cost function. On the other end, PBVS requires estimating the pose (from a 3D model of the scene/object or directly from data) but the cost function is smooth. Pose estimation also has the drawback of introducing some noise into the robot's trajectory. However, as camera relocalization is a fundamental task of many computer vision applications, it has become a topic of interest for the deep learning community, that seeks to replace or support classical geometric approaches with a neural network. One of the first works was PoseNet [START_REF] Kendall | Posenet: A convolutional network for real-time 6-dof camera relocalization[END_REF], that uses a Convolutional Neural Network (CNN) to regress the camera pose in a scene (position + orientation) from a single RGB image. This process was repurposed for VS, with multiple works such as [START_REF] Bateux | Training deep neural networks for visual servoing[END_REF], [START_REF] Saxena | Exploring convolutional networks for end-to-end visual servoing[END_REF], [START_REF] Yu | Siamese convolutional neural network for sub-millimeter-accurate camera pose estimation and visual servoing[END_REF] which employ a CNN to estimate the pose difference between current and desired images. This difference is then fed to the PBVS control law in order to move the robot closer to the target pose. Servoing is then fully dependent on this estimation, and [START_REF] Zhou | On the continuity of rotation representations in neural networks[END_REF] notes that in the case of orientation regression, trained networks may still yield a large error, as the rotation space is not smooth and continuous. Furthermore, In some cases, the difference between translational and rotational motions may be nonobservable.

Our work proposes to replace the standard pose regression approach with metric learning. In metric learning, we seek to learn a similarity function between samples that is dependent on the task to be accomplished. These approaches can be used to group images of the same concept (high similarity), while pushing away images of different concepts (low similarity). This principle can be applied to many tasks, including classification [START_REF] Hoffer | Deep metric learning using triplet network[END_REF], object tracking [START_REF] Dong | Triplet loss in siamese network for object tracking[END_REF] or camera relocalization [START_REF] Balntas | Relocnet: Continuous metric learning relocalisation using neural nets[END_REF].

We propose to use metric learning to shape the latent space in which we perform VS. We aim to learn a space in which the similarity between two learned representations correlates with the distance between their respective camera poses. This representation space can be seen as an intermediate manifold between poses and images on which we project both modalities. Doing so, we propose a new servoing control law that combines the optimal behavior of PBVS with the accuracy and simplicity of photometric methods. In Section II, we give an overview of VS, detailing the generic minimization framework, as well as presenting how PBVS and visual servoing in an autoencoder latent space are achieved. With this knowledge, we introduce our method in Section III, that constrains the latent space to have good properties for VS via metric learning. By leveraging the multimodal aspect of this shared space, our control law minimizes image error thanks to information obtained from a set of pose projections. Finally, In Section IV, we present simulated and real-world experiments that validate our approach and illustrate its properties.

II. RELATED WORKS A. Visual servoing framework

Visual servoing aims to reach a desired pose r * , from its arbitrary, current pose r. Many robotics tasks, such as navigation, tracking, object picking can be viewed through this prism, e.g. navigation is a succession of positioning tasks. VS thus seeks to solve an optimization problem, finding the pose closest to r * that minimizes an error function e: r = arg min r e(r).

(

) 1
If e is well designed and e = 0, then r = r * . Since the pose r may be unknown during servoing, e is defined as e = s(r)s * , which is the difference between what the camera sees at the current pose s(r) = s, and what it should see at the desired pose s * . As stated before, the choice of features s is important as it conditions the 3D trajectory, the convergence domain -how far can r be from r * before VS diverges -and the final accuracy -how close is r to r * . Features may lie in the image space (IBVS), in the 3D world (PBVS) or in photometric space (e.g. considering pixel intensities [START_REF] Collewet | Visual servoing set free from image processing[END_REF]). In all cases, the relationship between the variation (in time) of the features s and the camera velocity v must be established:

ṡ = L s v (2)
where L s = ∂s ∂r is called the interaction matrix. By inverting this equation, we can then define the control law that best minimizes the error e:

v = -λL + s e (3)
where λ is a gain parameter L + s the pseudo-inverse of L s . The computed velocity v can be used to move the robot's end-effector closer to the desired pose r * . VS operates in a closed loop, with the minimization of e being performed in an iterative manner.

B. Deep learning for pose-based VS

A pose r = t θu ⊤ is an element of SE(3) = R 3 × SO(3) the group that represents rigid transformations, combining translation t and rotation θu, where u is the axis around which to rotate and θ is the angle of the rotation. It may also be represented as a homogeneous matrix o T c , that expresses the pose of camera F c (or any other frame) in a reference frame F o (such as one given by the origin of a scene object o). The displacement between two poses ∆r = c * T c can be computed as c * T c = c * T o o T c . Deep Learning (DL) has previously been used to estimate the camera pose r, given a single image I. The first major work to accomplish this was PoseNet [START_REF] Kendall | Posenet: A convolutional network for real-time 6-dof camera relocalization[END_REF]. Given a dataset of images and their associated poses (expressed in a common frame), a CNN is trained to minimize the following loss function:

L pose (t, q) = ∥ t -t∥ 2 + β∥q -q∥ 2 (4)
where t, q are the predicted position and orientation of the camera and t, q is the ground truth. The orientation q can be expressed as a unit quaternion, or as an axis-angle θu. β is an important hyperparameter, that balances the learning of both translation and orientation. This weighting is required as the two quantities are on different scales, and it must carefully be tweaked in order to get sensible results. In [START_REF] Kendall | Geometric loss functions for camera pose regression with deep learning[END_REF], the authors introduced a multi-task loss that automatically learns the weighting, improving upon the manual tuning of β. Given two images, it is also possible to regress the pose difference ∆r [START_REF] Bateux | Training deep neural networks for visual servoing[END_REF], [START_REF] Saxena | Exploring convolutional networks for end-to-end visual servoing[END_REF], [START_REF] Yu | Siamese convolutional neural network for sub-millimeter-accurate camera pose estimation and visual servoing[END_REF]. ∆r can then be plugged into the PBVS control law [START_REF] Wilson | Relative end-effector control using cartesian position-based visual servoing[END_REF], [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF] to compute v. The interaction matrix of a pose expressed in a fixed frame F c * (also valid for F o ) is defined as [START_REF] Chaumette | Visual servo control, Part I: Basic approaches[END_REF]:

L r = c * R c 0 0 L θu (5)
with c * R c a rotation matrix and L θu the interaction matrix defined in [START_REF] Chaumette | 2 1/2 D visual servoing: a possible solution to improve image-based and position-based visual servoings[END_REF]. If ∆r is perfectly estimated, then the control law v = -λL -1 r ∆r ensures that the 3D trajectory is a geodesic both in translation and rotation.

C. Servoing in latent space

In [START_REF] Felton | Visual Servoing in Autoencoder Latent Space[END_REF], we introduced AEVS, a method to perform VS in the latent space of an autoencoder (AE). This AE is a neural network that learns a projection from an image to a lower dimensional representation (encoder), as well as the inverse mapping (decoder). This approach is similar to PCA-based VS [START_REF] Marchand | Subspace-based visual servoing[END_REF], except that AEs learn non-linear projections, which PCA cannot. The autoencoding objective is the minimization of the reconstruction error. Considering two embeddings z I , z I * , the control law is of the form:

v = -λL + z I (z I -z I * ) (6)
where L z I is computed analytically by applying the chain rule, finding L z I is the composition of the encoder Jacobian and the interaction matrix of the input image, detailed in [START_REF] Collewet | Visual servoing set free from image processing[END_REF], [START_REF] Marchand | Control camera and light source positions using image gradient information[END_REF]:

L z I = ∂z I ∂I ∂I ∂r .
Although this process is applied to images, the same reasoning can be applied to an encoder for any type of inputs, as long as the interaction matrix of the input can be defined. While this approach improves upon other photometric dimensionality reduction schemes, it has some drawbacks. First, the training objective is weakly correlated to pose estimation. This makes it hard to know whether the latent space of a trained AE will give good results when applied to VS. Second, the interaction matrix L z I depends on the interaction matrix of DVS, which is known to lead to unpredictable trajectories due to a highly non-linear cost function [START_REF] Collewet | Visual servoing set free from image processing[END_REF]. Finally, AEVS requires the camera's intrinsic calibration, as well as an estimate of the depth: a very coarse estimation works, but degrades the trajectory.

D. Metric learning

Metric learning aims to learn a similarity function between two compared inputs. The similarity measure is defined not in the input space (e.g. comparing pixel intensities) but rather from the factors that underlie the variations of the data. As an example, metric learning may learn to group images of the same subject (e.g. class, landmark or person) while enforcing a large margin between dissimilar concepts. Metric learning is often coupled to nearest neighbor search. A majority of the metric learning algorithms focuses on binary supervision to learn a meaningful metric: either two samples are similar or they are not. This is best seen in the common triplet loss [START_REF] Hoffer | Deep metric learning using triplet network[END_REF]

L triplet (x a , x p , x n ) = max(d(x a , x p ) -d(x a , x n ) + ϵ, 0)
that compares an anchor representation x a with similar and dissimilar samples x p , x n , where ϵ is a margin parameter that gives the separation threshold between positive and negative data points. It is also possible to learn smoothly varying similarity functions with continuous metric learning, as studied in [START_REF] Kim | Deep metric learning beyond binary supervision[END_REF], [START_REF] Balntas | Relocnet: Continuous metric learning relocalisation using neural nets[END_REF]. An application of continuous metric learning that is closely tied to our work can be found in [START_REF] Balntas | Relocnet: Continuous metric learning relocalisation using neural nets[END_REF]. This approach learns a feature space where the Euclidean distance between two representations is directly correlated with the overlap between two images, i.e. how much of the scene is visible in both cameras. An additional pose difference regressor is then trained to obtain a better estimate for the camera relocalization task. This network compares the query with its nearest neighbors. Another work closely related to ours is [START_REF] Jayaraman | Learning image representations tied to ego-motion[END_REF] which learns a feature space that is equivariant to camera motion. This approach uses discrete metric learning and a discrete number of motion patterns (e.g. move forward, rotate left/right) is learned.

Using latent representations also allows projecting multiple modalities of the same data to compare them with a single metric, in a common space. In [START_REF] Wang | Learning deep structure-preserving image-text embeddings[END_REF], authors transform text and images and map them to a shared space. Metric learning then allows for the retrieval of images that match a given textual query. Similarly, [START_REF] Sung | Deep multimodal embedding: Manipulating novel objects with point-clouds, language and trajectories[END_REF] embeds point clouds, textual tasks, and trajectories in the same space, so that the best fitting trajectory may be selected given a task. Multiple modalities can also be used in audio processing, by either creating a music sample-tag association [START_REF] Won | Multimodal metric learning for tag-based music retrieval[END_REF], or an acousticlinguistic relationship [START_REF] Wan | Google's next-generation real-time unit-selection synthesizer using sequence-tosequence lstm-based autoencoders[END_REF].

III. METRIC LEARNING FOR VISUAL SERVOING

This section presents our novel approach to Visual Servoing, that leverages deep metric learning in order to frame VS in a learned space. We first detail the reasoning behind our method, then present our training procedure, that shapes the latent space. Finally, we describe how VS is performed in this new latent space. In this paper, our goal is to propose a latent space servoing scheme with a behavior similar to PBVS, overcoming the limitations of other deep learning-based methods. To do so, we propose to create a multimodal latent space Z, in which both pose and image representations are mapped. A pose r ∈ SE(3) maps to an image I ∈ I via a camera. A pose r j maps to an embedding z rj , while the image acquired at r j is noted z Ij . The relationship between latent space and images/poses is illustrated in Figure [START_REF] Andreff | Visual servoing from lines[END_REF] We argue that for the best VS behavior, the distance between two embeddings should be equal to the distance between their underlying poses:

d Z (z j , z k ) = d SE(3) (r j , r k ), where d Z is the Euclidean distance: d Z (z j , z k ) = ∥z j -z k ∥ 2 (7)
The distance constraint holds true, whether z j , z k are image or pose projections, i.e. z j = z Ij or z j = z rj . If this property is perfectly met, it follows that:

• for a given image I j , acquired at pose r j , d Z (z Ij , z rj ) = 0. This is similar to an absolute pose regression objective; • for two images I j , I k acquired at r j , r k , the constraint

d Z (z Ij , z I k ) = d SE(3) (r j , r k
) is akin to estimating the relative pose difference between r j and r k from the images;

• finally, d Z (z Ij , z I k ) = d Z (z rj , z r k ).
As the two cost functions are the same, using the interaction matrix linked to a pose representation in a servoing context is valid for the minimization of d Z (z Ij , z I k ).

We thus seek to learn a space that is equivariant to 3D motion. This is ideal for VS, as we wish for features that have a strong and straightforward relation to pose. Moreover, we seek to explicitly learn a space that is invariant to perturbations P ∈ P, such as lighting changes or occlusions (i.e. z I+P = z I ).

B. Learning an SE(3)-equivariant space

To learn the space Z, we propose to use two distinct, parallel neural networks. The first is ϕ : SE(3) → Z, that maps a pose r to an embedding z r = ϕ(r). The second model ψ : I → Z, maps an image I to its latent representation z I = ψ(I).

In order to shape Z, we devise our loss function that is based on the distances between latent representations. While most metric learning approaches focus on clustering problems, we require our distances to be continuously meaningful as in [START_REF] Balntas | Relocnet: Continuous metric learning relocalisation using neural nets[END_REF], [START_REF] Kim | Deep metric learning beyond binary supervision[END_REF]. Instead of comparing the embeddings of specific tuples (r j , I j , r k , I k ), we adopt a full-batch approach, comparing a representation with every other in the batch. To do so, we leverage the distance matrices in SE(3) and in the latent space, viewing an embedding batch as a fully connected graph. By using this dense approach, we encourage the latent representations to position themselves with respect to every neighbor, providing a more stable training signal for the models ϕ and ψ.

To train the pose encoder ϕ, Our first loss seeks to enforce the equivariance with SE(3) when considering pose projections. As SE(3) has no true distance metric, a weighting between translation t and rotation θu distances must be introduced to create a pseudo-metric d SE(3) , as in [START_REF] Kendall | Geometric loss functions for camera pose regression with deep learning[END_REF]. We propose to define the weightings from the data and normalize the translational and rotational distances by their average value in the dataset. To compute the translation/rotation distances between two poses r j , r k , we first compute the pose difference ∆r = (∆t, ∆θu) we then define a single dataset-aware metric on SE(3), as

d SE(3) (r j , r k ) = 1 w t ∥∆t∥ 2 + 1 w θu ∆θ (8)
where w t , w θu are averages of translational and rotational distances, computed on a subset of the dataset. With d SE(3) defined, we introduce our first loss. Considering a batch of B samples, we seek to minimize

L ϕ,SE(3) = 1 B 2 B j=0 B k=0 (d SE(3) (r j , r k )-∥z rj -z r k ∥ 2 ) 2 (9)
This loss is fairly straightforward to minimize, as ϕ has access to the full pose information, and its main task is to transform the combination of translational and rotational metrics into a single euclidean distance. Of course, since poses are not available during VS, we require ψ to project an image to the same embedding as its associated pose, as well as match the distances with other poses. This is modeled by:

L ψ,ϕ = 1 B 2 B j=0 B k=0 (∥z Ij -z r k ∥ 2 -∥z rj -z r k ∥ 2 ) 2 (10)
By transitivity, L ψ,ϕ also minimizes

(d SE(3)(rj ,r k ) -∥z Ij - z I k ∥ 2 ) 2 .
To learn invariance to perturbations, we incorporate perturbed samples in the image batch. These noisy samples are exploited in L ψ,ϕ . Moreover, the image representations associated to a single pose r j (the original image and its P perturbed versions) are compared, and their distance to each other minimized:

L Pj = 1 P 2 P m=0 P n=0 ∥z Ij +Pm -z Ij +Pn ∥ 2 (11)
To obtain our final training objective, we sum up the losses

L = L ϕ,SE(3) + L ψ,ϕ + j L Pj ( 12
)
The impact of each loss is visualized in Figure [START_REF] Bakthavatchalam | A Direct Dense Visual Servoing Approach using Photometric Moments[END_REF]. It can be seen that the objectives designed above act as pushpull forces, moving the embeddings to respect the distance constraints, as well as minimize the influence of perturbations. By comparing a representation with every neighbor, we ensure that a single iteration forces an embedding towards a more stable location.

z Ij z rj z r k z r l d Z(z I j ,zr k ) d Z(zr j ,zr k ) P d SE(3) (rj, r k ) d SE(3) (rj, r l ) -∇L ψ,ϕ -∇L ϕ,SE (3)
-∇L P,j Fig. 2: Shaping the latent space with metric learning. We compare the representations of a sample j with their neighbors. While illustrating only the gradients for j, the loss is applied to every other sample.

C. Training policy

To constitute our batches of data, we randomly sample N = 64 poses from the dataset, and add their M = 3 closest neighbors found in a subset of the data to ensure that the constraints are met both globally and locally. For each image, we also generate P = 2 perturbations. Comparing to AEVS [START_REF] Felton | Visual Servoing in Autoencoder Latent Space[END_REF], not using a decoder (discarded for VS) network allows for larger batches. The perturbations consist in adding Gaussian noise, changing the brightness of the image and performing random erasing [START_REF] Zhong | Random erasing data augmentation[END_REF]. The image/pose generation process is the same as the one presented in [START_REF] Felton | Visual Servoing in Autoencoder Latent Space[END_REF], leveraging simulation to create data cheaply and efficiently.

We set the dimensionality of the latent space so that Z = R 32 . The image encoder ψ is a ResNet-34 [START_REF] He | Deep residual learning for image recognition[END_REF], with the modifications of [START_REF] Felton | Visual Servoing in Autoencoder Latent Space[END_REF], i.e. replacing batch normalization [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF] with weight normalization [START_REF] Salimans | Weight normalization: A simple reparameterization to accelerate training of deep neural networks[END_REF] and the end average pooling with a group convolution. The pose-embedding network ϕ is a 5-layer perceptron of dimensions 6 → 32 → 64 → 128 → 64 → 32, with ReLU activations after each hidden layer. The networks are jointly trained for 50 epochs, on a dataset of 100K samples. Training takes around 8h with a Quadro RTX 6000. Gradient descent is performed with the Adam optimizer [START_REF] Kingma | A method for stochastic optimization[END_REF] and a learning rate of 10 -4 . With our networks trained, we can finally perform VS in the latent space.

D. Visual servoing in latent space

Our approach to VS is two-pronged, based on the multimodal nature of Z. We first embed the current and desired images z I = ψ(I), z I * = ψ(I * ) in Z to obtain e = z Iz I * .

To minimize e, we require an interaction matrix, that gives us the control directions. Before VS starts, we project a set of N poses {z r1 , ...z r N } = {ϕ(r 1 ), ..., ϕ(r N )} in the latent space, along with their interaction matrix, thanks to the method described in Section II-C. The latent interaction matrix is then defined as L zr j = ∂zr j ∂rj L rj , with ∂zr j ∂rj the Jacobian of ϕ with respect to r j (computed via forward propagation) and L rj given by Equation ( 5). Because we minimize L ψ,ϕ (Equation ( 10)), We can approximate the interaction matrix at z I as an interpolation of its neighbors' interaction matrices. We thus have a K-Nearest Neighbors (KNN) regression problem, defined as:

L zr = K j=0 α j L zr j with α j = ∥z rj -z I ∥ 2 K k=0 ∥z r k -z I ∥ 2 (13)
To obtain the pose embeddings set, we generate poses on a 6D grid (displacements in translation and rotation), and oversample near z I * . In our experiments, we thus create a set of 1M pose representations that can be used for KNN. To be computationally tractable, we store them in a KD-tree [START_REF] Jl | Multidimensional binary search trees used for associative searching[END_REF]. Other, smarter, less memory demanding sampling strategies, based on the values of both z I and z I * , are possible. We however found that this straightforward approach works well in practice. Unlike AEVS, the interaction matrices of the pose representations (and thus the network Jacobian) are computed offline. Fitting the pieces together, the final control law is simply

v = -λL + zr (z I -z I * ) (14)
In the next section, we explore the behavior of this VS scheme, both in simulation and on a 6DOF robot.

IV. EXPERIMENTS AND RESULTS

A. Simulation validation

We start our experimental validation with a large scale experiment: we run 500 VS tasks with multiple methods.

The initial positions are chosen with a "look-at/look-from" scenario: we sample camera positions in a volume of dimensions 1.2m × 1.2m × 0.3m, centered on the desired position. we then sample the focal (look-at) points of the cameras on the planar scene, with a distance to the desired focal points between 8cm and 32cm. The scene is a poster of dimensions 80cm × 60cm, and we set the desired camera elevation to 60cm. From the camera position and focal point, we build the camera orientation, to which we add a rotation around the optical axis ∈ [-120 • , 120 • ]. The average initial pose error is 47cm ± 16cm, 74 • ± 28 • .

We experiment with multiple methods, the first one being a pure photometric scheme (DVS) [START_REF] Collewet | Visual servoing set free from image processing[END_REF]. We also compare with AEVS [START_REF] Felton | Visual Servoing in Autoencoder Latent Space[END_REF], as well as a PBVS visual servoing approach which uses a CNN-based pose regression approach, as developed in [START_REF] Saxena | Exploring convolutional networks for end-to-end visual servoing[END_REF], [START_REF] Bateux | Training deep neural networks for visual servoing[END_REF] (referred as PBVS-CNN). To ease training and improve results, we adopt the automatic weighting loss of [START_REF] Kendall | Geometric loss functions for camera pose regression with deep learning[END_REF] that balances translation and rotation errors in Equation ( 4). For both AEVS and PBVS-CNN, we use the network and data described in Section III-C. For our method, we explore different numbers of neighbors K. We first report the percentage of samples that convergence for each method. A sample is defined as having converged if the VS method significantly reduces (by at least 90%) the initial positioning error and the final velocities are close to 0. For those samples, we measure the end positioning error, as well as the Absolute Pose Error (APE), averaged over all iterations of a trajectory, which describes how far the method strays from the geodesic of PBVS. The other included metrics are the mean length ratios: the length of trajectory of the observed method divided by the length of the PBVS trajectory. As can be seen in Table I, our method is able to converge reliably and accurately, with a positioning error that is comparable to photometric methods in the case of clean target images I * (noted ✓in the table), while having a far larger domain of convergence. We can observe that a larger value for K leads to a more stable and accurate positioning. Looking at the results of the PBVS-CNN, it can be seen that the end positioning error is subpar (3.3cm, 1.71 • on average). The trajectory statistics (APE and length ratio) show that our method is far closer to the behavior of PBVS, compared to AEVS or DVS. This is made explicit in Figure (3b). The overall statistics when considering cases where I * is perturbed (noted ✗) highlight that our method better handles variations in lighting and occlusions. While both convergence and accuracy degrade, they remain above that of the pose estimator, even on clean images.

Next, we explore the servoing behavior in the latent space. We perform 8 trajectories, where the initial errors are displacements on the x, y axes, with an initial error of 20cm. We then visualize the trajectories in the latent space by projecting in a 2D-subspace with PCA (explained variance ≈ 96%). As can be seen in Figure (3a), the minimization of e in the latent space leads to nearly straight lines in the latent space. The error between pose embeddings also correlates well with the error from image representations.

B. Robot experiment

We also deploy our method on a 6DoF gantry robot and study its behavior on a large motion. For the experiment, shown in Figure [START_REF] Bateux | Training deep neural networks for visual servoing[END_REF], the initial displacement is ∆r 0 = (-8.16cm, 7.27cm, -29.98cm, 24.86 • , -10.16 • , 135.06 • ), with a large error in the image (Figure (4c)) and run our method for 1.2k iterations. Note that this motion (and thus the image) is far from what is seen during training. As servoing progresses, the error in the latent space (Figure (4f)) is quickly minimized. The final positioning error is ∆r f inal = (0.09cm, 0.08cm, -0.04cm, 0.08 • , -0.12 • , -0.01 • ), and the resulting error in image space is low (Figure (4d)). As shown in Figures (4e,4g,4h), the trajectory starts with some unwanted motion on the x, y axes (compensated by y/x rotations), probably due to the fact that I lies outside the training domain. However, our method recovers and exhibits a smooth decrease in positioning error.

V. CONCLUSION

In this paper, we proposed a method that allows visual servoing from both image and pose representations in a common latent space. This new visual servoing scheme combines the accuracy of photometric methods with the behavior of pose-based approaches. Our experiments show strong results, with a large convergence domain and accurate positioning. In future works, we plan to extend our method to deal with more than two modalities (i.e. adding depth or segmentation information), so that the visual features at the current and desired poses may be drawn from different domains. In addition, we believe that it is possible to reduce data requirements by including some form of self-supervised learning. For instance, small motions could be used to warp an image and generate new weakly labeled samples. Supervision would then be used to learn representations tied to large motions, while smaller displacements would be taken into account with self-supervision.

Fig. 1 :

 1 Fig.1: The proposed latent space for visual servoing. Both images and poses are projected in Z, where they can be compared.

c * tc 2 Fig. 3 :

 23 Fig. 3: (a) PCA projection of trajectories z I -z I * in the latent space, for 2D motions. Circles show the error for the pose embeddings z r -z r * for various distances. (b) 3D trajectories of different methods on a single example.

Fig. 4 :

 4 Fig. 4: First experiment: (a) Starting image I. (b) Desired image I * . (c) Starting image difference I-I * . (d) Final image difference. (e) Pose difference rr * . (f) Error in the latent space z Iz I * . (g) Camera velocities v. (h) 3D trajectory.

TABLE I :

 I Comparative results of different VS methods on 500 trajectories. Error and trajectory statistics are reported for cases that converge. For metrics marked with ↑, higher is better. Lower is better when marked ↓.


Theodor Luibrand

Adrien Bercher

Rodolfo Rocco

Farnaz Tahouni-Bonab

Lucia Varbaro

Willem Carl

Claribel Rischau

Yixi Domínguez

Weiwei Zhou

Carl Willem Rischau

Claribel Domínguez

Yixi Zhou

Weiwei Luo

Soumen Bag

Lorenzo Fratino

Reinhold Kleiner

Stefano Gariglio

Dieter Koelle

Jean-Marc Triscone

Marcelo J Rozenberg

Alexey B Kuzmenko

Stefan Guénon

Javier Del Valle
email: javier.delvalle@unige.ch

Carl Willem Rischau

Characteristic length scales of the electrically induced insulator-to-metal transition

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Many correlated materials, such as the vanadate and rare-earth nickelate families, are well-known for their insulator-to-metal transition (IMT) [1][2][3]. The transition into the metallic state can be induced by increasing temperature, adding dopants or applying high pressures [4][5][6], but it can also be triggered electrically [7][8][9][10][11]. A large enough applied voltage or current can create a percolative metallic filament due to Joule heating [12], drastically reducing the resistance of the system [13][14][15][START_REF] Kumar | Separation of current density and electric field domains caused by nonlinear electronic instabilities[END_REF][START_REF] Lange | Imaging of Electrothermal Filament Formation in a Mott Insulator[END_REF][START_REF] Rocco | Exponential Escape Rate of Filamentary Incubation in Mott Spiking Neurons[END_REF][START_REF] Adda | Direct Observation of the Electrically Triggered Insulator-Metal Transition in V3O5 far below the Transition Temperature[END_REF]. We must note that this filament is not caused by the diffusion of ions under strong electric fields as commonly observed in resistive RAMs (random access memories) [START_REF] Yang | Memristive devices for computing[END_REF], but rather by a local phase transition from insulator to metal. When the voltage (current) is removed, the filament disappears, resulting in volatile resistive switching [START_REF] Del Valle | Subthreshold firing in Mott nanodevices[END_REF]. This phenomenon has attracted a lot of attention due to promising applications in emerging technologies, such as emulating neuronal spiking for neuromorphic computing [START_REF] Pickett | A scalable neuristor built with Mott memristors[END_REF][START_REF] Ignatov | A memristive spiking neuron with firing rate coding[END_REF][START_REF] Stoliar | A Leaky-Integrate-and-Fire Neuron Analog Realized with a Mott Insulator[END_REF][START_REF] Del Valle | A caloritronics-based mott neuristor[END_REF][START_REF] Yi | Biological plausibility and stochasticity in scalable VO2 active memristor neurons[END_REF][START_REF] Bohaichuk | Fast Spiking of a Mott VO2-Carbon Nanotube Composite Device[END_REF][START_REF] Oh | Energy-efficient Mott activation neuron for full-hardware implementation of neural networks[END_REF], probabilistic bits for stochastic computing [START_REF] Kumar | Chaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing[END_REF][START_REF] Jerry | Stochastic Insulator-to-Metal Phase Transition-Based True Random Number Generator[END_REF][START_REF] Del Valle | Generation of Tunable Stochastic Sequences Using the Insulator-Metal Transition[END_REF], or serving as electrooptical switches for optoelectronics [START_REF] Cueff | Optical switching in hybrid VO2/Si waveguides thermally triggered by lateral microheaters[END_REF][START_REF] Li | Photo-induced non-volatile VO2 phase transition for neuromorphic ultraviolet sensors[END_REF][START_REF] Olivares | Experimental demonstration of a tunable transverse electric pass polarizer based on hybrid VO2/silicon technology[END_REF][START_REF] Miller | Silicon waveguide optical switch with embedded phase change material[END_REF][START_REF] Lee | Sharpened VO2 Phase Transition via Controlled Release of Epitaxial Strain[END_REF]. In spite of this, many fundamental aspects of the electrically induced IMT are poorly understood. One of the most salient issues is the typical lengthscale of this process: what sets the size of the metallic filaments, or the number of them? And similarly, how do these characteristic lengths affect the resistive switching properties i.e. the sharpness of the switch (𝜕𝑉 𝜕𝐼 ⁄ ) or the total resistive drop? Understanding this is not only of fundamental interest, but also key for designing device applications.

Here, we use a combination of wide-field optical microscopy [START_REF] Lange | A high-resolution combined scanning laser and widefield polarizing microscope for imaging at temperatures from 4 K to 300 K[END_REF] and scattering-type scanning near-field optical microscopy (s-SNOM) [START_REF] Post | Coexisting firstand second-order electronic phase transitions in a correlated oxide[END_REF] to characterize filament lengthscales during the electrically-induced IMT in NdNiO3 and SmNiO3 microdevices. These compounds are two wellknown members of the rare earth nickelate family [2,6]. They both display an IMT concomitant with a structural phase transition, but there are rather important differences between the two. NdNiO3 has a sharp IMT around 120 K (depending on epitaxial strain) with a resistivity drop of more than two orders of magnitude (Fig. 1a). SmNiO3 on the other hand, displays a smooth IMT around 400 K, with a one order of magnitude resistivity change [2,6]. Such different IMTs allow us to contrast the results from both materials and to determine which parameters govern filament lengthscales.

Methods

We fabricated two-terminal microdevices on top of our NdNiO3 and SmNiO3 films, as depicted in Fig. 1b. The nickelate films (40 nm thick, Fig. S1) were grown on LaAlO3 (001) substrates using off-axis magnetron sputtering. We used a combination of optical lithography and ion etching to define isolated 360 m x 130 m nickelate islands, on top of which we patterned two planar Pt electrodes using a second optical lithography and on-axis Pt sputtering. The electrodes are 20 m wide, with a 20 m gap between them (10 m x 10 m for the s-SNOM experiments). The IMT can be triggered by applying a large enough voltage or current across the gap. To image this phenomenon, we take advantage of the large reflectivity change across the IMT [10]. We use optical microscopy to capture the distribution of metallic/insulating domains in the gap between electrodes [START_REF] Lange | Imaging of Electrothermal Filament Formation in a Mott Insulator[END_REF]. We do so in operando i.e. while applying a variable bias current, which allows us to capture clear images of the percolating filament. More details about the fabrication process, device characterization, and the experimental methods can be found in the supplementary information.

Experimental results

Fig. 2a shows the voltage V as a function of the current I in a NdNiO3 microdevice at several temperatures below the IMT temperature i.e. the film is in the insulating state when no current is applied. As the current is ramped up, the voltage rises steeply until a threshold is reached, after which a steep voltage reduction takes place. Such drop marks the moment when the electricallyinduced IMT occurs and a filament percolates between the electrodes. This is a well-known phenomenon [13][14][15][START_REF] Kumar | Separation of current density and electric field domains caused by nonlinear electronic instabilities[END_REF][START_REF] Lange | Imaging of Electrothermal Filament Formation in a Mott Insulator[END_REF][START_REF] Rocco | Exponential Escape Rate of Filamentary Incubation in Mott Spiking Neurons[END_REF][START_REF] Adda | Direct Observation of the Electrically Triggered Insulator-Metal Transition in V3O5 far below the Transition Temperature[END_REF], and it can be readily observed with our imaging set-up. The filament widens when current is further increased, shrinks for decreasing current and disappears at low enough current values (Fig. S2 and supplementary videos), in accordance with the volatile V-I curves in Fig. 2a. We must note that some of the V-I curves feature two discontinuities. In the current range between them, the system is not stationary but rapidly oscillates between a high and a low resistance state, as further discussed in section 3 of the supplementary information. As expected, we find the filament width to be strongly dependent on the bias current, similarly to what has been reported in previous works [14,15,[START_REF] Lange | Imaging of Electrothermal Filament Formation in a Mott Insulator[END_REF]. The bulk of this paper focuses on how other factors, such as temperature or material properties, also play a key role at setting filament size. Fig. 2a shows a clear trend: the voltage drop becomes sharper and larger as the temperature is lowered. This feature is also observed in SmNiO3 microdevices (Fig. 2d), where resistive switching at room temperature is modest, very gradual and without discontinuities. As the temperature is lowered, the drop becomes steeper and discontinuous. Comparing NdNiO3 and SmNiO3, it is clear that resistive switching is sharper for the former. Figs. 2b and2e show optical microscopy images of filaments in NdNiO3 and SmNiO3, respectively, for the same applied current, I = 20 mA. Images at two different temperatures are displayed for each material, showing distinctively thinner filaments at lower temperatures in both cases. This is better appreciated in Figs. 2c and2f, where filament width (for I = 20 mA) is plotted as a function of temperature. Lower temperatures yield thinner filaments and, therefore, higher current densities. Moreover, comparing NdNiO3 and SmNiO3, we can see that filaments are much thinner in NdNiO3. Therefore, for a fixed bias current, the filament size is strongly dependent on the material and the base temperature. Fig. 2 as a whole establishes a strong connection between filament size and V-I characteristics: thinner filaments (higher current densities) lead to sharper and larger resistive switching.

While optical microscopy is a versatile tool to visualize metallic/insulating areas at multiple temperatures and currents, its spatial resolution is limited by diffraction. In order to get more detailed images, we performed in-operando cryogenic s-SNOM measurements in NdNiO3 devices, using a set-up such as the one depicted in Fig. 3a. The spatial resolution of this atomic force microscopy (AFM)-based technique is limited only by the tip radius (20 nm) [START_REF] Post | Coexisting firstand second-order electronic phase transitions in a correlated oxide[END_REF], allowing us to obtain high-resolution AFM and near-field images of the filaments. Figs. 3b and3c show topography and SNOM images at 18 K and 70 K, respectively. Images for 0, 10 and 20 mA are displayed. For similar currents, filaments are thinner at lower temperatures, in accordance to widefield optical images in Fig. 2. Moreover, s-SNOM allows us to resolve clear qualitative differences between both temperatures. Images taken at 18 K show a single, intense filament percolating between electrodes, while at higher temperature multiple filaments appear. Thus, lower temperatures favor a winner-takes-all situation in which a single filament carries all the current. ) is focused at a metal-coated AFM tip, which further focuses light into an area comparable to the tip radius (20 nm). The tip-scattered signal is determined by the optical conductivity of the area of the material directly underneath the tip, allowing to get high resolution images of metallic (high signal) and insulating (low signal) domains. The AFM is working in tapping mode and the s-SNOM signal is detected at the 3rd order harmonic to filter out the far-field component as described in the supplementary. (b), (c) Topography and s-SNOM amplitude images for NdNiO3 microdevices at T=18 K and T=70 K, respectively. The s-SNOM signal was normalized using the Pt electrode as reference. S-SNOM data for three different current values is shown.

Resistor network simulations

To understand these results, we performed numerical simulations in which we model our system as a two-dimensional resistor network (Fig. 4a) [10,[START_REF] Stoliar | A Leaky-Integrate-and-Fire Neuron Analog Realized with a Mott Insulator[END_REF]. Each node in the network can be either metallic or insulating, depending on the local temperature and a Landau free energy functional that mimics the IMT. The insulating state resistivity increases as the temperature is decreased, following a variable range hopping dependence (Inset Fig. 4b). Currents and voltages at each node are calculated by solving Kirchhoff's laws. Local temperature is updated in each simulation time step, considering Joule heating and heat conduction. A more detailed description of the simulations can be found in the methods section of the supplementary material. This simple model reproduces experimental results with only a few parameters (Fig. 4b), allowing us to identify which ones play a key role. Fig. 4c shows simulated 2-dimensional resistivity maps of the devices for three different values of current and base temperature. As expected, filaments are strongly dependent on the bias current. Also, similarly to the experiments, filaments become narrower as the base temperature is lowered. The material confines current flow into a smaller region at lower temperatures. This in turn induces higher current densities, increasing local Joule heating and greatly affecting the temperature distribution across the device, as can be seen in Fig. 4d. As the filament narrows, its inner temperature increases. Therefore, thinner filaments induce a stronger current-temperature feedback, which is a key factor controlling switching dynamics [10,[START_REF] Rocco | Exponential Escape Rate of Filamentary Incubation in Mott Spiking Neurons[END_REF][START_REF] Kumar | Chaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing[END_REF]. A strong feedback makes the device susceptible to runaway effects which manifest as discontinuities in the experimental V-I curves [START_REF] Kumar | Chaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing[END_REF]. As a result, the thinner the filament and the higher its currenttemperature feedback, the larger the discontinuities in the V-I curves. Experimentally, such discontinuities are observed to be bigger and more frequent at lower temperatures, and especially for NdNiO3. These are the same conditions in which the thinnest filaments are observed.

Discussion

Since filament size determines switching properties, it is key to identify which parameters control the material's ability to confine current into smaller or larger areas. Here we analyze two contributions: the resistivity difference across the IMT and the thermal conductivity of the substrate. The resistivity contrast between the insulating (ins) and metallic (met) phases is expected to play a major role, since it corresponds to the resistivities outside and inside the filament. When ins >> met, the current is strongly focused into the filament, reducing Joule heating outside. This keeps the insulating areas cold and confines the filament in a small region. But as ins decreases, the insulator becomes leaky, allowing current flow and power dissipation outside the filament and reducing its confinement. The temperature dependence of the resistivity implies that as temperature increases, so does power dissipation outside the filament, increasing its width. For instance, the voltage at I=5 a.u. is half as large for T=90 a.u. than for T=18 a.u. (Fig. 4b), but the insulating resistivity is nearly 8 times smaller. As a result, power dissipation outside the filament is increased by a factor of 2 at T=90 a.u. vs T=18 a.u. These differences in resistivity explain not only the temperature dependence of the filament size, but also the differences observed between NdNiO3 and SmNiO3. The former has a larger resistivity change across the IMT (Fig. 1a) [6], and is therefore expected to focus current into thinner filaments. Furthermore, the presence of single or multiple percolating filaments (Fig. 3) can be understood in a similar light. For ins >> met, the current will crowd through the first hotspot that metallizes, favoring a winner-takes-all scenario.

But the resistivity drop across the IMT is not the only mechanism that can explain the differences in filament size. Thermal properties are also expected to play a crucial role, and they can provide a similarly satisfactory explanation. The thermal conductivity of LaAlO3 is not constant, but decreases from 0.6-0.7 W/cm•K at 60 K to 0.15 W/cm•K at 300 K [START_REF] Schnelle | Specific heat capacity and thermal conductivity of NdGaO3 and LaAlO3 single crystals at low temperatures[END_REF]. This means that the substrate can more efficiently evacuate heat at lower temperatures, keeping cold the areas surrounding the filament. The filament is therefore confined to a smaller region, as simulations have recently shown [START_REF] Rocco | Exponential Escape Rate of Filamentary Incubation in Mott Spiking Neurons[END_REF]. Therefore, thermal properties can also explain the smaller filament size at lower temperatures, as well as accounting for the overall differences between NdNiO3 and SmNiO3, which have IMTs in very different temperature ranges.

Unfortunately, it is difficult to disentangle the contributions due to the resistivity change across the IMT and the substrate thermal conductivity. Both parameters decrease as temperature is increased. Within the nickelate family, it is observed that as the transition temperature increases (with the reduction of the rare-earth radius), the resistivity drop across the IMT decreases [2,6]. A similar trend is observed in the V2O3, VO2, V3O5 family [10]. Therefore, it is not feasible to compare a material featuring a high temperature IMT with a large resistivity change, with another system having a low temperature IMT with a small resistivity change.

A way to overcome this drawback is to compare, for the same material, samples with different IMT quality. We fabricated two NdNiO3 films: one with a high quality IMT, and a second one subjected to a 120C, 30 minutes annealing in vacuum. The annealing creates oxygen vacancies, reducing the resistivity change across the IMT, as seen in Fig. 5a. This has clear consequences in the resistive switching properties, which are smoother for the annealed sample (Fig. 5b). Furthermore, there are notable differences in the filament, as can be seen in Figs. 5c and5d. The annealed device shows much less contrast and homogeneity within the metallic area, perhaps due to the formation of multiple filaments. This differs from the well-defined, intense filament for the non-annealed sample, and points out to a smaller degree of metallization and current focusing. This confirms that the resistivity change across the IMT is a key parameter controlling resistive switching and filament characteristics, although does not rule out important contributions from thermal conductivity.

Conclusions

In summary, we have used a combination of in-operando standard and scanning near-field optical microscopies to study the characteristic lengths of filament formation during the electricallyinduced IMT. We found that, in addition to bias current, filament width is strongly dependent on base temperature and the specific material. Lower set-up temperatures yield thinner filaments, increasing current density and local temperature, leading to sharper resistive switching properties. With the aid of resistor network simulations, we discussed the material properties that control filament size, underlining the importance of the resistivity drop across the IMT as well as the substrate's thermal conductivity.

Our results support recent works concerning another fundamental aspect of the electricallyinduced IMT: switching dynamics [10,11]. It was proposed that a large resistivity ratio between insulator and metal would induce higher current focusing, increasing local Joule heating within the filament and explaining the different switching timescales observed in V2O3, VO2, V3O5, NdNiO3 and SmNiO3. However, direct evidence of this has been lacking so far. The present work provides a systematic study of the characteristic lengthscales of the electrically-induced IMT, unveiling a strong connection between resistivity, thermal properties, filament size and resistive switching characteristics. The mechanisms outlined here are simple and general, and could be applicable to other types of resistive switching, such as ReRAM. Considered together with recent developments in the field [10,11,[START_REF] Del Valle | Subthreshold firing in Mott nanodevices[END_REF], it completes a simple and unified picture of the length-and time-scales of filament nucleation, growth and relaxation, as well as underlining their importance for developing new technologies based on the IMT.

Methods

Thin film and device fabrication

We grew NdNiO3 and SmNiO3 oxide films on (001) oriented LaAlO3 substrates using off-axis magnetron sputtering in an Ar:O2 (3.5:1) mixture at a pressure of 180 mTorr and substrate temperature of 460º C. Films are 40-45 nm thick and grow epitaxially, as can be seen using Xray diffraction (Figure S1c andS1d).

For microdevice fabrication, a combination of techniques was used. First, we patterned isolated NdNiO3 and SmNiO3 islands using optical lithography and Ar ion beam milling. This allows us to measure each device independently, since it is electrically isolated from the others. After this we patterned Pt electrodes on top of these islands. For that we used optical lithography followed by on-axis Pt sputtering at room temperature and a lift-off in acetone. Pt thickness is around 40 nm and the gap size is 20 m x 20 m. For the s-SNOM measurements, a further lithographic step was used. Optical lithography does not create smooth electrode edges. This is very challenging for SNOM, since the tip is tripped by the electrode irregularities. To improve this, we used electron beam lithography and a second Pt evaporation to define 10 m x 10 m electrodes with smooth edges.

In operando standard optical microscopy

We used an optical wide-field microscope that facilitates simultaneous imaging and electrical transport measurements [2]. The device under investigation is mounted in vacuum, on the cold finger of a liquid Helium continuous flow cryostat with a temperature range between 4.2 K and 300 K. The microscope has a spatial resolution of 500 nm, the illumination is a monochromatic LED with a wavelength of 532 nm, and the maximum field of view is approximately 500 µm x 500 µm.

The electric transport properties were measured in a two-probe configuration. For NdNiO3, we used a Keithley 2400 SourceMeter configured as a current source, whereas we used a highly stable self-built current source for SmNiO3.

Image Processing: For NdNiO3, the grey values were normalized to a NdNiO3 area that is not influenced by the resistive switching (not in-between the electrodes). For SmNiO3, all the images are differentialthe image with zero bias current is subtractedd for each current.

In operando s-SNOM

A cryogenic s-SNOM system (cryo-neaSNOM from neaspec/attocube GbmH) was used for nanoscopic imaging of the filaments in the devices. Infrared radiation from a Quantum Cascade Laser (Daylight Solutions) was focused at a metal-coated AFM tip (ARROW-NCPt-50 from NanoAndMore GmbH), which was grounded in order to reduce the electrostatic interaction between the tip and the sample. Despite that, we had to avoid applying voltages higher than 10 V in the microdevices as it would disturb the AFM in tapping mode. The tip size determines the spatial resolution (20 nm in this case). Pseudoheterodyne detection allows separating far-field and near-field contributions to the signal by using higher-order tapping harmonics (the 3rd harmonic is used in the present paper). The detected near field signal has an excellent spatial contrast between the insulating and metallic phases because of the large change of the optical conductivity across the IMT. More information about the s-SNOM operation can be found in [3].

Resistor network simulations

In the simulations presented in this work we use a phenomenological mesoscopic model known as Mott Resistors Network [4,5]. The model describes the material as a matrix of cells, each containing four resistors, which connect the cell to its four nearest neighbors. Each cell corresponds to a small region of the material of the order of 10 nm. This scale is chosen in order to define a phase for the cell, which can be insulating or metallic. At first, all the resistors are initialized to a high insulating value, and all the cells are in the insulator phase. A voltage is applied to the mesh through the metallic electrodes that are situated at the top and the bottom and currents begin to circulate in the resistor network. These currents can be computed, knowing the initial resistance and the applied voltage, using Kirchhoff laws. When the current I flows through the resistors Rij, these generate heat according to Joule's law, wit power 𝑃 = 𝐼 2 𝑅. The heat generated by a cell is given by the sum of the contributions of its four resistors

𝑃 𝑖𝑗 (𝑡) = (𝐼 1 2 (𝑡) + 𝐼 2 2 (𝑡) + 𝐼 3 2 (𝑡) + 𝐼 4 2 (𝑡)) 𝑅 𝑖𝑗 (𝑡)
where 𝑡 indicates time in units of the simulation time-step, 𝑖𝑗 are the indexes which identify the cell, 𝑃 𝑖𝑗 is the power generated by the cell, 𝑅 𝑖𝑗 is the resistance of the four resistors (which are always assumed to share the same value) and 𝐼 1 , 𝐼 2 , 𝐼 3 , 𝐼 4 are the four currents flowing through them. Setting to unity the geometrical dimension of the cell, we can identify the resistivity of the cell with 𝑅 𝑖𝑗 . The temperature of the cell will be the result of two contributions: Joule heating and a dissipative term that includes the dissipation to the nearest neighbour cells and the dissipation to a substrate at a fixed temperature 𝑇 0 , with which all the cells are in contact. Therefore, using the heat transfer equation, we can write the temperature of the cell as follows where 𝐾 is the thermal conductivity, 𝐶 the thermal capacity and the sum with indexes 𝑘𝑙 runs over the nearest neighbour cells. We note that we have made the non-essential assumption of choosing the same thermal conductivity for the dissipation to the substrate and to the nearest neighbours, and that the time-step of the simulation 𝛥𝑡 is set to unity.

The first order transition of a cell from the insulator to the metal phase (and vice-versa) is described as a thermally activated behavior with a probability that depends on the temperature of the cell according to the following Arrhenius-like law where 𝑎 and 𝑏 are the states of cell (insulator or metal) and 𝐸 𝐵 is the energy barrier that separates the two corresponding local minima as described by the following Landau-type free energy (which is appropriate for a 1 st order thermally driven transition) 𝜂 is the order parameter and 𝑇 𝐶 , ℎ 1 , ℎ 2 , 𝑝 1 and 𝑝 2 are constants. The resistivity of the cell is then chosen according to the state of the cell: low and constant (𝜌 𝑚𝑒𝑡 ) in the metal state and high and temperature dependent in the insulator state (𝜌 𝑖𝑛𝑠 (𝑇)). In particular, we choose Mott's equation for variable range hopping [6] to describe the temperature dependence of the resistivity in the insulating state, since it has already been used to fit the resistivity of NdNiO3 samples [7,8] 𝜌 𝑖𝑛𝑠 (𝑇) = 𝜌 0 𝑒

𝛥( 1 𝑇 - 1 𝑇 𝐼𝑀𝑇 ) 1 4 ⁄
where 𝛥 is a constant, 𝑇 𝐼𝑀𝑇 is the metal-insulator transition temperature and 𝜌 0 = 𝜌(𝑇 𝐼𝑀𝑇 ) is the resistivity at the transition temperature. Nevertheless, the specific choice of the functional form does not change the main qualitative features of the results. Once the resistivity of the cell has been computed we can update the resistance of the resistors within it. When all the cells have been updated, the time-step is increased by one and the simulation continues as described above, starting again from the computation of Kirchhoff currents.

Filament size vs. Applied current

Apart from temperature and specific material choice (the focus of the paper), filament size depends strongly on the applied current. This is a well-known effect [9][10][11][12][13]. It can be best appreciated in the supplementary videos 1-10. Fig. S2 shows images of a NdNiO3 microdevice at different points of a V-I measurement cycle, for a temperature T = 60 K. As can be seen, the filament width grows with applied current, appearing and disappearing at the discontinuities of the ramp-up and rampdown V-I curves, respectively.

FIG. 1 .

 1 FIG. 1. Sample characteristics. (a) Resistivity vs temperature for 40 nm thick NdNiO3 (blue) and SmNiO3 (red) thin films. Inset: Resistivity plotted as a function of T-TIMT. TIMT was calculated finding the maximum of 𝜕log(𝜌) 𝜕𝑇 ⁄ , where  is the resistivity. Only the warm up branch is shown. (b) Schematic representation of the two-terminal devices. Nickelate islands (brown) were patterned on top of a LaAlO3 substrate (blue). Two platinum electrodes (grey) were used to electrically trigger the IMT. The schematic is not at scale.

FIG. 2 .

 2 FIG. 2. Connection between resistive switching properties and filament size. (a), (d) Voltage vs current curves for NdNiO3 and SmNiO3 microdevices, respectively. Several temperatures are plotted for each material. (b), (e), Wide-field optical microscopy images of filaments in NdNiO3 and SmNiO3, respectively. Current is 20 mA for all four images. Two different temperatures are shown for each material. For the NdNiO3 images, reflectivity was normalized using a region far from the gap. For SmNiO3, differential images are shown, where at each point, the reflectivity at I=0 mA is subtracted. (c), (f), Filament width vs temperature at I=20 mA. The width was determined using a gaussian fitting of linescans perpendicular to the filament direction, taking the full-width-half-maximum as filament width. The error bars show the standard deviation of the distribution of widths.

FIG. 4 .

 4 FIG. 4. Resistor network simulations and current focusing effect. (a) Schematic representation of the simulated resistor network (size W x L). Low resistance electrodes (yellow) define an oxide gap where individual nodes could be either metallic (orange) or insulating (dark brown), as described in the methods section of the supplementary material. (b) Simulated voltage vs. current curves for three different temperatures: 18 a.u. (black), 70 a.u. (red) and 90 a.u. (blue). Inset: Simulated resistance vs. temperature of the device. (c) Simulated, two-dimensional resistivity plots for all combinations of three currents (1, 3 and 5 a.u.), and three device base temperatures (18, 70 and 90 a.u.). Resistivity is plotted in logarithmic colorscale. (d) Simulated, two-dimensional temperature plots for all combinations of three currents (1, 3 and 5 a.u.), and three device base temperatures (18, 70 and 90 a.u.). Temperature is plotted in linear colorscale and normalized to the transition temperature (120 a.u.).

FIG. 5 .

 5 FIG. 5. Resistive switching and filament characteristic in pristine and annealed NdNiO3. (a) Twoprobe device resistance vs temperature on pristine (blue) and annealed (red) NdNiO3 films. (b) Voltage vs current at T = 50 K for a pristine (blue) and annealed (red) sample. (c) Wide-field microscopy image of filament formation in pristine (left) and annealed (right) NdNiO3. T = 50 K and I = 20 mA in both cases. Reflectivity was normalized using an area far from the gap region. (d) Zoomed image into the central part of the filament for pristine (top) and annealed (bottom) NdNiO3. T = 50 K and I = 20 mA in both cases.

  FIG. S1. Structural and transport properties of nickelate films and microdevices. (a) Two-probe resistance vs temperature characteristics of NdNiO3 and SmNiO3 microdevices, respectively. (b) Two-probe resistance plotted as a function of T-TIMT. TIMT was calculated finding the maximum of 𝜕log(𝜌) 𝜕𝑇 ⁄ , were  is the resistivity. Only the warm up branch is shown. NdNiO3 shows a much sharper IMT. (c), (d) -2 scans of the SmNiO3 and NdNiO3 films, respectively. Scans were performed around the (001) LaAlO3 peak. Finite size oscillations are very visible, allowing us to determine the film thickness. Data was manually fitted using the InteractiveXRDfit software [1], and is shown in the plots as continuous dark lines. XRD shows high quality epitaxial films with thicknesses around 40-45 nm.

  FIG. S2. Filament size dependence on applied current. Central panel: V-I characteristics of a NdNiO3 microdevice at T = 60 K. Blue asterisks mark currents at which the outer panels where captured. Outer panels: standard optical microscope images of the devices at different applied currents.

Acknowledgements

The authors thank Marco Lopes for his support during the fabrication and measurement of these samples. The sample fabrication and project coordination were funded by the Swiss National Science Foundation through an Ambizione Fellowship (#PZ00P2_185848). The oxide growth was supported by the European Research Council under the European Union's Seventh Framework Program (FP7/2007-2013)/ERC Grant Agreement 319286 Q-MAC and the Swiss National Science Foundation Project no. 200020-179155. W.R. was supported by the U.S. Office of Naval Research through the NICOP Grant N62909-21-1-2028. s-SNOM measurements were supported by the Swiss National Science Foundation through a Research Grant #200020_201096. Simulations were funded by the French ANR project "MoMA" ANR-19-CE30-0020. T. L. acknowledges support by the Cusanuswerk, Bischöfliche Studienförderung. J.d.V acknowledges support from the Spanish Ministry of Science through a Ramón y Cajal Fellowship (#RYC2021-030952-I).

Supplementary Information

Characteristic lengthscales of the electrically-induced insulator-to-metal transition

Self-oscillation of microdevices

Interpretation of filament size is not straightforward for all currents. As can be seen in Fig. 2a and 2d in the main text, for some temperatures there are two voltage discontinuities when current is ramped up (for instance, at T = 50 K in NdNiO3). In between the two, there is a range of currents where the V-I curve is smooth. In that range, the system is not stationary, but rather oscillates between two configurations, one with a percolating filament and one without it. These selfoscillations, which are in the 10 kHz range, are a well-known effect [14,15] and they can be observed with an oscilloscope. The parasitic capacitance and the slow reaction time of current source are the main factors determining the oscillation regime.

All the analysis about filament size at different temperatures and materials shown in the paper is done for I = 20 mA. This is well-above the self-oscillation range, where the system is stationary again, so it does not affect our conclusions.

Supplementary references