Pierre Marza
email: pierre.marza@insa-lyon.fr

Laetitia Matignon
email: laetitia.matignon@univ-lyon1.fr

Olivier Simonin
email: olivier.simonin@insa-lyon.fr

Christian Wolf
email: christian.wolf@naverlabs.com

Multi-Object Navigation with dynamically learned neural implicit representations

Understanding and mapping a new environment are core abilities of any autonomously navigating agent. While classical robotics usually estimates maps in a stand-alone manner with SLAM variants, which maintain a topological or metric representation, end-to-end learning of navigation keeps some form of memory in a neural network. Networks are typically imbued with inductive biases, which can range from vectorial representations to birds-eye metric tensors or topological structures. In this work, we propose to structure neural networks with two neural implicit representations, which are learned dynamically during each episode and map the content of the scene: (i) the Semantic Finder predicts the position of a previously seen queried object; (ii) the Occupancy and Exploration Implicit Representation encapsulates information about explored area and obstacles, and is queried with a novel global read mechanism which directly maps from function space to a usable embedding space. Both representations are leveraged by an agent trained with Reinforcement Learning (RL) and learned online during each episode. We evaluate the agent on Multi-Object Navigation and show the high impact of using neural implicit representations as a memory source.

Introduction

Autonomous navigation in complex unknown 3D environments from visual observations requires building a suitable representation of the environment, in particular when the targeted navigation task requires high-level reasoning. Whereas classical robotics builds these representations explicitly through reconstructions, possibly supported through machine learning, end-to-end training learns them automatically either from reward, by imitation learning or through self-supervised objectives.

While spatial representations can emerge even in unstructured agents, as shown in the form of grid-cells in artificial [START_REF] Cueva | Emergence of grid-like representations by training recurrent neural networks to perform spatial localization[END_REF][START_REF] Banino | Vector-based navigation using grid-like representations in artificial agents[END_REF] and biological agents [START_REF] Beeching | Egomap: Projective mapping and structured egocentric memory for deep RL[END_REF], spatial inductive biases can support learning actionable spatial representations and decrease sample complexity. Popular inductive biases are metric maps [44,5,[START_REF] João | Mapnet: An allocentric spatial memory for mapping environments[END_REF], topological maps [START_REF] Beeching | Learning to plan with uncertain topological maps[END_REF][START_REF] Singh Chaplot | Neural topological slam for visual navigation[END_REF] and recently, self-attention, adapting transformers [55] to sequential decision making and navigation [START_REF] Fang | Scene memory transformer for embodied agents in longhorizon tasks[END_REF][START_REF] Du | VTNet: Visual Transformer Network for Object Goal Navigation[END_REF][START_REF] Chen | Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation[END_REF][START_REF] Reed | A Generalist Agent[END_REF]. The chosen representation should support robust estimation of navigable space even in difficult conditions, mapping fea-tures and objects of interest, as well as querying and reusing this information at a later time. The representation should be as detailed as required, span the full (observed) scene, easy to query, and efficient to read and write to, in particular when training is done in large-scale simulations.

Our work builds on neural fields and implicit representations, a category of models which represent the scene geometry, and eventually the semantics, by the weights of a trained neural network [START_REF] Xie | Neural fields in visual computing and beyond[END_REF]. They have the advantage of avoiding the explicit choice of scene representation (e.g. volume, surface, point cloud etc.) and inherently benefit from the generalization abilities of deep networks to interpolate and complete unobserved information. Implicit representations have demonstrated impressive capabilities in novel view synthesis [40,[START_REF] Vincent Sitzmann | Implicit neural representations with periodic activation functions[END_REF], and have potential as a competitive representation for robotics [START_REF] Ortiz | iSDF: Real-Time Neural Signed Distance Fields for Robot Perception[END_REF][START_REF] Li | Learning Continuous Environment Fields via Implicit Functions[END_REF][START_REF] Sucar | imap: Implicit mapping and positioning in real-time[END_REF][START_REF] Adamkiewicz | Vision-only robot navigation in a neural radiance world[END_REF]. Their continuous nature allows them to handle level of detail efficiently through a budget given as the amount of trainable weights. This allows to span large environments without the need of discretizing the environment and handling growing maps.

We explore and study the potential of implicit representations as inductive biases for visual navigation. Similar to recent work in implicit SLAM [START_REF] Sucar | imap: Implicit mapping and positioning in real-time[END_REF], our representations are dynamically learned in each episode. Going beyond, we exploit the representation dynamically in one of the most challenging visual navigation tasks, Multi-Object Navigation [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF]. We introduce two complementary representations, namely a query-able Semantic Finder trained to predict the scene coordinates of an object of interest specified as input, and an Occupancy and Exploration Implicit Representation, which maps 2D coordinates to occupancy information, see Figure 1. We address the issue of the efficiency of querying an implicit representation globally by introducing a new global read mechanism, which directly maps from function space, represented through its trainable parameters, to an embedding summarizing the current status of occupancy and exploration information, useful for navigation. Invariance w.r.t. reparametrization of the queried network is favored (but not enforced) through a transformer based solution. Our method does not require previous rollouts on the scene for pre-training or building a representation.

Our work targets a fundamental aspect of visual and semantic navigation, the mapping of key objects of interest. MultiON is currently one of the few benchmarks which evaluates it. As argued in previous literature [5,[START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF], only sequential tasks, where objects have to be found in a given order, allow object level mapping to emerge directly from reward. This follows from the observation that an agent trained to find and retrieve a single object per episode (from reward) is not required to map seen target objects, as observing them directly leads to a reactive motion towards them.

Our contributions can be summarized as follows: (i) We propose two implicit representations for semantic, occupancy and exploration information, which are trained online during each episode; (ii) We introduce a new global read procedure which can extract summarizing context information directly from the function itself; (iii) We show that the representations obtain performance gains compared to classical neural agents; (iv) We evaluate and analyze key design choices, the representation's scaling laws and its capabilities of lifelong learning.

Related Work

Visual Navigation -is a rich problem that involves perception, mapping and decision making, with required capacities being highly dependent on the specific task. A summary of reasoning in navigation has been given in [START_REF] Anderson | On evaluation of embodied navigation agents[END_REF], differentiating, for instance, between waypoint navigation (Pointgoal) [START_REF] Anderson | On evaluation of embodied navigation agents[END_REF] or finding objects of semantic categories (ObjectGoal) [START_REF] Anderson | On evaluation of embodied navigation agents[END_REF]. More recent tasks have been explicitely designed to evaluate and encourage mapping objects of interest during navigation itself [4,[START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF]. They are of sequential nature and use external objects, which are not part of the scanned 3D scenes but randomly placed. In this work we address Multi-Object Navigation (MultiON) [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF].

Mapping and Representations -Classical methods often rely on SLAM [8,[START_REF] Lluvia | Active Mapping and Robot Exploration: A Survey[END_REF] which has been proposed in different variants (2D or 3D metric, topological, eventually with semantics) and observations (LIDAR, visual etc.). The objective is to integrate observations and odometry estimates over a trajectory and reconstruct the scene. Differentiable variants have been proposed recently [28,30]. Mapping can also be discovered through interactions by a blind agent [START_REF] Bono | Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction[END_REF]. Visual Navigation can also be framed as an end-to-end learning problem, where representations are learned automatically from different signals, in particular RL. Memory can take the form of vectorial representations in recurrent units [START_REF] Mirowski | Learning to navigate in complex environments[END_REF][START_REF] Jaderberg | Reinforcement learning with unsupervised auxiliary tasks[END_REF]62], with hybrid variants including mapping [START_REF] Singh Chaplot | Neural topological slam for visual navigation[END_REF][START_REF] Sadek | Multi-Object Navigation in real environments using hybrid policies[END_REF][START_REF] Dey | Learning whom to trust in navigation: dynamically switching between classical and neural planning[END_REF]. Recent work tends to augment agents with more structured memories. Examples are spatial metric tensors, which can contain occupancy [START_REF] Singh Chaplot | Learning to explore using active neural slam[END_REF], semantics [START_REF] Singh Chaplot | Object goal navigation using goal-oriented semantic exploration[END_REF] or be fully latent, effectively corresponding to inductive biases of the neural agents [44,5,[START_REF] João | Mapnet: An allocentric spatial memory for mapping environments[END_REF]. Other alternatives are topological maps [START_REF] Beeching | Learning to plan with uncertain topological maps[END_REF][START_REF] Singh Chaplot | Neural topological slam for visual navigation[END_REF] or self-attention and transformers [55] adapted to navigation [START_REF] Fang | Scene memory transformer for embodied agents in longhorizon tasks[END_REF][START_REF] Du | VTNet: Visual Transformer Network for Object Goal Navigation[END_REF][START_REF] Chen | Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation[END_REF][START_REF] Reed | A Generalist Agent[END_REF].

Implicit representations -were initially targeting 3D reconstruction [START_REF] Mescheder | Occupancy networks: Learning 3d reconstruction in function space[END_REF][START_REF] Park | Deepsdf: Learning continuous signed distance functions for shape representation[END_REF]14]. The core idea is to replace the need for discretizing 3D space into voxels [START_REF] Maturana | Voxnet: A 3d convolutional neural network for real-time object recognition[END_REF], 3D points [START_REF] Fan | A point set generation network for 3d object reconstruction from a single image[END_REF] or meshes [START_REF] Groueix | A papier-mâché approach to learning 3d surface generation[END_REF], by an implicit representation of the 3D structure of the scene through the parameters of a learned neural network. Recent work [40,[START_REF] Vincent Sitzmann | Implicit neural representations with periodic activation functions[END_REF] achieved state-of-the-art performance on novel view synthesis with neural implicit representations. The NeRF paper introduced a differentiable volume rendering loss allowing to supervise 3D scene reconstruction from only 2D supervision [40]. For a more detailed overview of recent advances in the rapidly growing field, we refer the reader to [START_REF] Xie | Neural fields in visual computing and beyond[END_REF].

Implicit representations in robotics -are a recent phenomenon, used to represent density [START_REF] Adamkiewicz | Vision-only robot navigation in a neural radiance world[END_REF] or to perform visuomotor control [START_REF] Li | 3d neural scene representations for visuomotor control[END_REF]. Related to goal-oriented navigation, some work targets SLAM with neural implicit representations [START_REF] Sucar | imap: Implicit mapping and positioning in real-time[END_REF], follow-up adding semantics [59], learned from sparse semantic annotations of the scene. [START_REF] Zhi | ilabel: Interactive neural scene labelling[END_REF] is also built on top of [START_REF] Sucar | imap: Implicit mapping and positioning in real-time[END_REF] and allows a user to interactively provide semantic annotation for the implicit representation to be trained on in real time. [START_REF] Zhu | Nice-slam: Neural implicit scalable encoding for slam[END_REF] proposes a hierarchical implicit representation of a scene to scale to larger environments and obtain a more detailed reconstruction. [START_REF] Chung | Orbeez-slam: A real-time monocular visual slam with orb features and nerf-realized mapping[END_REF] combines feature-based SLAM and NeRF. Our work goes beyond implicit SLAM and does not stop at reconstructing a scene. We not only build implicit representations dynamically during the episode, we also use them in a down-stream navigation task without requiring any initial rollout for pre-training or building a representation. We also combine two different implicit representations targeting semantics vs. scene structure.

Analyzing the neural network function space -implicit representations are instances of function spaces, which are represented through their trainable parameters. Previous work performed analyses by predicting accuracy from network weights [START_REF] Unterthiner | Predicting neural network accuracy from weights[END_REF][START_REF] Charles | Heavy-tailed universality predicts trends in test accuracies for very large pre-trained deep neural networks[END_REF][START_REF] Charles H Martin | Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data[END_REF] or the generality gap between train and test performance from hidden activations [29,[START_REF] Yak | Towards task and architecture-independent generalization gap predictors[END_REF]. A direction pioneered by Hypernetworks [START_REF] Ha | Hypernetworks[END_REF] directly predict the network weights. Recently, [61] generate the weights of a CNN from support samples in the context of few-shot learning. More related to our work, [START_REF] Pan | Neural implicit flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data[END_REF] learns to predict the weights of an implicit representation based on external factors in the context of spatio-temporal dynamics encoding. In this work, we learn a direct mapping between an implicit representation, represented by its weights, to an actionable embedding summarizing the scene globally.

Navigating with implicit representations

We target the Multi-Object Navigation task [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF], which requires an agent to navigate in a photo realistic 3D environment from RGB-D observations o t and reach a sequence of target objects (colored cylinders) in a particular order. Goal categories g t are given at each time step t. In an RL setting, the agent receives positive reward for each successfully reached object as well as when the geodesic distance towards the goal decreases, and a small negative reward for each step, favoring short paths.

We follow and augment a base end-to-end architecture used in many recent RL approaches, including [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF][START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF][START_REF] Mirowski | Learning to navigate in complex environments[END_REF], with the RGB-D observation, class of the current target and previous action as input to the agent. Temporal information is aggregated with a GRU unit whose output is fed to actor and critic heads. We equip this agent with two implicit representations, trained to hold and map essential information necessary for navigation: the positions of different objects of interest, and occupancy / exploration information, as shown in Figure 1, fs fo The goal of the Semantic Finder f s (.; θ s ) parameterized by trainable weights θ s is to predict the absolute position of an object as x = [x x x y x z ] = f s (q; θ s ) specified through an input query vector q. Uncertainty u is also estimated -see Section 3.1 for details. x is then converted into coordinates relative to the agent to be fed to the GRU. Compared to classical metric representations [START_REF] João | Mapnet: An allocentric spatial memory for mapping environments[END_REF]44,[START_REF] Singh Chaplot | Neural topological slam for visual navigation[END_REF]5], querying the location of an object can be done through a single forward pass. The Occupancy and Exploration Representation can in principle be queried directly for a single position, but reading out information over a large area directly this way would require multiple reads. We propose to compress this procedure by providing a trainable global read operation r(.; θ g ), which predicts an embedding e containing a global context about what has already been explored, and positions of navigable space. The prediction is done directly from the trainable parameters of the implicit representation, as e = r(θ o ; θ r ).

Here θ o is input to r, whereas θ r are its parameters.

Given representations f s and f o , a single forward pass of the agent at time step t and for a goal g t involves reading the representations and providing the input to the policy. The current RGB-D observation o t is also encoded by the convolutional network c (different from the projection module p used to generate samples for training the Semantic Finder). Previous action a t-1 and current goal g t are passed through embedding layers, named L(.) in the following equations. These different outputs are fed to the policy,

x t = f s (g t ; θ s,t ), e t = r(θ o,t ; θ r ), c t = c(o t ; θ c ), (1)
h t = GRU (h t-1 , x t , u t , e t , L(a t-1 ), L(g t ), c t ; θ G ), ( 2
)
a t = π(h t ; θ π ), (3)
where we added indices .   also a simplification, as we train the agent with PPO, an actor-critic method -see Section 3.4.

Mapping means training! -The implicit representations f s and f o maintain a compact and actionable representation of the observed scene, and as such need to be updated at each time step from the current observation o t . Given their implicit nature and implementation as neural networks, updates are gradient based and done with SGD. The implicit representations are therefore trained from scratch at each episode even after deployment.

Training a representation from observations obtained sequentially during an episode also raises a serious issue of catastrophic forgetting [START_REF] Goodfellow | An empirical investigation of catastrophic forgetting in gradient-based neural networks[END_REF], as places of the scene observed early might be forgotten later in the episode [START_REF] Sucar | imap: Implicit mapping and positioning in real-time[END_REF]59]. We solve this by maintaining two replay buffers throughout the episode, one for each representation. Training samples are generated from each new observation and added to the replay buffers at each time step. Both representations are then trained for a number of gradient steps (n s for the Semantic Finder and n o for the Exploration and Occupancy Representation). Details on the two representations and their training are given in Sections 3.1 and 3.2. The global reader r is not trained or finetuned online but rather trained once offline.

The Semantic Finder f s

While recent work on implicit representations for robotics focused on signed distance functions [START_REF] Ortiz | iSDF: Real-Time Neural Signed Distance Fields for Robot Perception[END_REF][START_REF] Li | Learning Continuous Environment Fields via Implicit Functions[END_REF], occupancy [START_REF] Sucar | imap: Implicit mapping and positioning in real-time[END_REF] or density, assuming light density approximates mass density [START_REF] Adamkiewicz | Vision-only robot navigation in a neural radiance world[END_REF], the aim of this model is to localize an object of interest within the scene, which can be seen as inverse operation to classical work. From a query vector given as input, the Semantic Finder predicts the position of the object, which is particularly useful for an agent interacting with an environment in the context of a goal conditioned task. It is implemented as a 3-layer MLP with ReLu activations in the intermediate layers and a sigmoid activation for the output. Hidden layers have 512 neurons. The query vector q corresponds to the 1-in-K encoding of the target object class, which during navigation is directly determined by the object goal g t provided by the task.

Mapping/Training -The implicit representation is updated minimizing the L1 loss between the prediction x i = f s (q i , θ s ) and the supervised coordinates x * i (we avoided the term "ground-truth" here on purpose),

L s = i ∥x * i -x i ∥ 1
, where the sum goes over the batch sampled from the scene replay buffer. Coordinates

x * i are normalized ∈ [0, 1]. The data pairs (x * i , q i )
for training are created from each observation o t during each time step, each data point corresponding to an observed point. Pixels in o t are inversely projected into 3D coordinates in the scene using the depth channel, the camera intrinsics, as well as agent's coordinates and heading that are assumed to be available, as in [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF]. The query vector q corresponds to a 9-dimensional vector encoding a distribution over object classes (8 target objects and the "background" class). Let us recall that while the training of the representation is supervised, this supervision cannot use "ground-truth" information available only during training. All supervision information is required to be predicted from the data available to the agent even after deployment. We predict object class information through a semantic segmentation model p, which is applied to each current RGB-D observation o t ∈ R h×w×4 , recovering the output segmentation map m t ∈ R k×l×9 . The model has been pre-trained on the segmentation of the different target objects, i.e. coloured cylinders, and is not fine-tuned during training of the agent itself.

Training data pairs (x * i , q i ) are sampled from this output. The supervised coordinates x * i are simply the mean 3D coordinates of each feature map cell, after inverse projection. The query vector q i is the distribution over semantic classes. After the replay buffer is updated, a training batch must be sampled to update the neural field. One fourth of the samples in the batch of size b correspond to the b/4 last steps. The rest are sampled from the previous steps in the replay buffer. Uniform sampling is also performed among pairs collected at a given time step.

Estimating uncertainty -is an essential component, as querying yet unseen objects will lead to wrong predictions, which the agent needs to recognize as such, and discard. The estimation of uncertainty in neural networks is an open problem, which has been previously addressed through different means, including drop out as a Bayesian approximation [START_REF] Gal | Dropout as a bayesian approximation: Representing model uncertainty in deep learning[END_REF], variational information bottlenecks [START_REF] Shah | Rapid Exploration for Open-World Navigation with Latent Goal Models[END_REF], density estimation [START_REF] Kobyzev | Normalizing flows: An introduction and review of current methods[END_REF], and others. In this work, we approximate a density estimate in the scene replay buffer by calculating the minimum Euclidean distance between the input query and all embeddings in the replay buffer at the current time step. The method is simple and efficient and does not require explicitly fitting a model to estimate the marginal distribution p(q), in particular as the uncertainty representation is latent, can be un-normalized as not required to be a probability.

Occupancy and Exploration Implicit Representation f o

Unlike f s , the occupancy representation f o is closer to classical implicit representations in robotics, e.g. [START_REF] Sucar | imap: Implicit mapping and positioning in real-time[END_REF][START_REF] Ortiz | iSDF: Real-Time Neural Signed Distance Fields for Robot Perception[END_REF][START_REF] Li | Learning Continuous Environment Fields via Implicit Functions[END_REF][START_REF] Adamkiewicz | Vision-only robot navigation in a neural radiance world[END_REF], which map spatial coordinates to variables encoding information on navigable area like occupancy or signed distances. Different to previous work, our representation also includes exploration information, which changes over time during the episode. Once explored, a position changes its class, which makes our neural field dynamic. Another difference with f s is that the latter deals with 3D coordinates while f o is a topdown 2D representation. Inspired by [START_REF] Xie | Neural fields in visual computing and beyond[END_REF][START_REF] Tancik | Fourier features let networks learn high frequency functions in low dimensional domains[END_REF], the model uses Fourier features ϕ extracted from the 2D coordinates x previously normalized ∈ [0, 1],

ϕ = (cos(x2 0 ), sin(x2 0 ), ..., cos(x2 p 4 ), sin(x2 p 4 )). (4)
The network f o is a 3-layer MLP with ReLu intermediate activations and a softmax function at the output layer. Hidden layers have 512 neurons, and p = 40.

Mapping/Training -The implicit representation is updated minimizing the Cross Entropy loss between the prediction s of the neural field and the supervised label s * of three classes {Obstacle, Navigable, Unexplored}, as

L o = - 3 c=1 s * c log s c .
As for the Semantic Finder, training data pairs (s * , s) are created through inverse perspective projection of the pixels of the observation o t into 3D scene coordinates. Thresholding the z (height) coordinate decides between Navigable and Obstacle classes. Points with a z coordinate higher than a certain threshold are discarded. The replay buffer is balanced between both classes, and only samples of the last 1000 steps are kept. Samples of the Unexplored class are not stored.

The replay buffer is sampled similarly to the one for the Semantic Finder. However, additional samples for the Unexplored class are created by sampling uniformly inside the scene, for speed reasons simply ignoring conflicts with explored areas and treating them as noisy labels.

Global Occupancy Read r -handling reparametrization invariance

The global Occupancy reader r allows to query the occupancy information of the scene globally, beyond point-wise information, and as such is a trainable mapping from the space of functions f o (.; .) to an embedding space e. In particular, two functions

f o and f ′ o s.t. f o (x) = f ′ o (x)
∀x should be mapped to identical or close embeddings. However, as the occupancy networks f o are implemented as MLPs, any given instance f o (.; θ o ) parameterized by trainable weights θ o can be reparametrized by any permutation of hidden units, which leads to permutations of the rows and columns, respectively, of two weight matrices, its own and the one of the preceding layer. This reparameterization keeps the represented functions identical, although their representations as weight vectors are different.

To favor learning a global occupancy reader which is invariant w.r.t these transformations, we implement it as a transformer model with self-attention [55] -this, however, does not enforce full invariance. The model takes as input a sequence of tokens (w 1 , ..., w N ), where w i ∈ R a is a learned linear embedding of the incoming weights of one neuron within the implicit representation f o , and N is the number of neurons of f o . Each token is summed with a positional encoding in the form of Fourier features. An additional "CLS" token with learned embedding is concatenated to the input sequence. The reader is composed of 4 self-attention layers, with 8 attention heads. The output representation of the "CLS" token is used as the global embedding of the implicit representation.

Training -The global reader r is trained with full supervision from a dataset of 25k trajectories composed of MLP weights θ o,i and absolute maps M i , i..1..25k. Each map is a metric tensor providing occupancy information extracted from the corresponding implicit representation, i.e. M i (x y , x x ) = f o (x, θ o,i ). The dataset also contains an egocentric version M ′ i of each map, which is centered on the agent and oriented depending on its current heading. The reader r is trained in an Encoder-Decoder fashion, where r plays the role of the encoder,

e i = r(θ o,i ), Mi = Dec(e i , p i ), (5)
where p i is the agent pose (position and heading), necessary to decode ego-centric information. We minimize a cross entropy loss on the prediction of ego-centric maps,

L g = -i k l 3 c=1 M ′ * i,c (k, l) log M ′ i,c (k, l) (6)
Directly training this prediction proved to be difficult. We propose a procedure involving several steps, detailed in section 3.5. After the training phase, the reader g is used in the perception + mapping module of the agent as given in equation ( 1), and kept frozen during agent RL training.

Training the Agent

The agent is trained with RL, more precisely Proximal Policy Optimization (PPO) [START_REF] Schulman | Proximal policy optimization algorithms[END_REF]. The inner training loops of the implicit representations are supervised (red arrows in Figure 2) and occur at each time step in the forward pass, whereas the RL-based outer training loop of the agent occur after N acting steps (black arrows in Figure 2). As the perception module used to generate training data from RGBD-observations for the Semantic Finder f s is independent of the visual encoder c in the agent (see Figure 2), and as its query q t is fixed to the navigation goal g t , there is no need to track the weights θ s at each time step in order to backpropagate the PPO loss (outer training). This is a key design choice of our method.

Training assumptions -We do not rely on the existence of global GT maps for occupancy, as f o () and r() were trained on observation data from agent trajectories only. However, similar to [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF][START_REF] Pashevich | Episodic transformer for vision-and-language navigation[END_REF], we exploit object positions in simulation, during training only; Moreover, we require pixel-wise segmentation masks during training. We believe that this does not change requirements, as the goal is to fully exploit 3D photo-realistic simulators as a data source and see how far the field can go with this. Generalization requirements are unchanged: we require our agent to be able to generalize to new unseen scenes. Generalization to unseen object categories is not targeted, and in MultiON task setup not possible for any agent, as object positions and labels are required for reward calculation.

Training the global reader

The proposed training procedure for the global reader r (performed before training the agent) can be split into 3 phases.

The architecture for the convolutional decoder Dec() is kept the same in all of them. The hyperparameters of its different layers are detailed in the Supplementary Material. It is composed of 6 transpose convolution layers along with batch norm layers and ReLU activations, except for the last layer with a softmax activation. Figure 3 Global reader finetuning on egocentric maps -The global reader, whose weights are initialized from the weights obtained in the previous step, is now trained along with the same decoder from the first step (also used in the second step) on the set of egocentric maps M ′ i , i..1..25k. Both global reader and decoder are finetuned. The output of the global reader is not directly fed to the decoder, but is passed through linear layers in order to fuse information about first the position of the agent, and then its heading because this time the right operations of shift and rotation must be applied in order to reconstruct egocentric maps.

Integrating the pre-trained global reader into the agent -After this pre-training in 3 steps, the weights of the global reader are frozen and not updated during the RL training. However, a linear layer is learnt to project the 576-dim embedding from the global reader into a 256-dim representation fed to the GRU. This linear layer is trained from reward signals.

Experiments

MultiON task -we target the 3-ON version of the MultiON task [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF], where the agent deals with sequences of 3 objects, each belonging to one of 8 classes (cylinders of different colors). At each step, the observation o t is an RGB-D image of size 256×256×4 and the target class is a one-in-K (K=8) vector. The action space is discrete: {Move forward 0.25m, Turn left 30 • , Turn right 30 • , Found}. An episode is considered successful if the agent finds all of the targets before the time limit (2, 500 environment steps), and chooses the Found action for each one at a distance closer to 1.5m. Calling Found incorrectly terminates episode as a failure. An access to perfect odometry information (localization and heading) was assumed in [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF] as the standard protocol.

Dataset and metrics -The agent is trained on the Mat-terport3d [9] dataset. We followed the standard train/val/test split over scenes (denoted MultiON train, MultiON val, Mul-tiON test): 61 training, 11 validation and 18 test scenes. The train, val and test splits are respectively composed of 50, 000, 12, 500 and 12, 500 episodes per scene. Reported results on the val and test sets (Tables 1 and2) were computed on a subset of 1, 000 randomly sampled episodes. We report standard metrics as used in the navigation literature (and in [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF]): Success: percentage of successful episodes -all objects are reached respecting order, time; Progress: percentage of objects successfully found (respecting order, time); SPL: Success weighted by Path Length, extending the original SPL metric [START_REF] Anderson | On evaluation of embodied navigation agents[END_REF] to MultiON; PPL: Progress weighted By Path Length (the official MultiON challenge metric).

Global reader dataset -The Global reader r was trained on a dataset of 25k trajectories obtained from rollouts performed by a baseline agent [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF]. 95% was used for training and the rest for validation. On these trajectories we first trained the occupancy representation f o "in-situ", i.e. as if it were deployed on the agent, and we recorded training samples i for training the reader r: pairs of network weights θ o,i and associated maps M i obtained by iteratively querying the implicit representation. Ego-centric maps were generated from the absolute ones and both were cropped around their center.

Perception module dataset -The perception module [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF]. "AUX" = auxiliary losses using privileged information [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF]. "ORC"=non-comparable, uses oracle information. ρ = pre-training of input encoders from [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF]. Training details -We use the reward function given in [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF] for RL/PPO training (see supplementary material), and train all agents for 70M steps as in [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF]. For all agents in Table 1 and some in Table 2 (w/ pre-train: ✓ in ρ column), the encoders (visual encoder c, as well as goal and previous action embedding layers, see Figure 2) are pre-trained with a baseline, which corresponds to the ProjNeuralMap agent trained with auxiliary losses [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF]. This is done to faster training, as it will be shown later (in Table 2) that the same final performance can be reached without this initial pre-training of encoders. Training and evaluation hyper-parameters, as well as architecture details have been taken from [START_REF] Wani | Multion: Benchmarking semantic map memory using multi-object navigation[END_REF]. All reported quantitative results are obtained after 3 training runs for each model.

Impact of the implicit representations -Table 1 shows the impact of the two implicit representations on navigation (top: means over 3 runs; bottom: best validation seeds over 3 runs). To keep compute requirements limited and decrease sample complexity, in these ablations we do not train the full agent from scratch, in particular since the early stages of training are spent on learning basic interactions. We decompose training into three phases: 0-30M steps (no implicit representations, i.e. all entries to the agent related to f s and f o are set to 0); 30M -50M steps (training includes the Semantic Finder f s ) and finally 50M -70M steps (full model). This 3-steps approach is used to train all agents in Table 1, and will be denoted as curriculum (See Table 2, w/ curriculum:in γ column). All metrics on both val and test sets are improved, with the biggest impact provided by the Semantic Finder, which was expected. We conjecture that mapping object positions is a more difficult task, which is less easily delegated to the vectorial GRU representation, than occupancy. We also see an impact of the occupancy representation, which not only confirms the choice of the implicit representation f o itself, but also its global read through r(θ o ). Training curves are given in the supplementary material.

Uncertainty -has an impact on agent performance, as we show in the ablation in Table 3. Indeed, when training an agent with the semantic input since the beginning of training (w/o curriculum) and no occupancy input (as the uncertainty is only related to semantic information), feeding the agent with the computed uncertainty about the output of the Semantic Finder brings a boost in performance.

Comparison with previous SOTA methods -is done in Table 2. The performance entries of these baselines are taken from [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF], which describes the winning entry of the CVPR 2021 MultiON competition. Our method outperforms the different competing representations, even when they benefit from the same pre-training scheme and are thus completely comparable. NoMap with pre-training corresponds to the first row of Table 1. The difference between (g) and (i) is the use of the auxiliary tasks in [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF], but also that the implicit representations are available to the agent during the whole training period for (i), i.e. no decomposition into 3 phases as in the ablations in Table 1 (w/o curriculum). Moreover, compared to (g), in (i) the weights of the pre-trained encoders are finetuned. We see that the gains of our representations are complementary to the auxiliary losses in [START_REF] Marza | Teaching agents how to map: Spatial reasoning for multi-object navigation[END_REF]. (j) and (k) confirm this finding, showing that most of the gain compared with (h) comes from the implicit representations, with the auxiliary losses bringing an additional boost. (j) and (k) also show that, even though pre-training can help speed up RL training, similar test performance is achieved without it. Reconstruction performance of the Global Reader r -although the task of reconstructing egocentric maps from occupancy functions f o (represented by θ o ) is only used as a proxy task to train the Global reader r, we see it is a reliable proxy for the quality of extracting the global latent vector e fed to the agent.

Accuracy Jaccard Index 83.4 56.5

Table 4. Performance of the global reader r: We report accuracy and Jaccard index.

In Table 4 we report reconstruction performance measured as accuracy and mean Jaccard Index on the validation split of the dataset used to train the reader. We judge that an accuracy of 83.4% is surprisingly high, given that the global reader needs to reconstruct the content of the representation directly from its parameters θ o , that each implicit representation has been initialized randomly, and that the reader is required to be invariant w.r.t. to reparameterization (see Section 4). The task is even made harder as neural weights can be considered as an absolute representation of the env. and the reader must combine it with information about the agent pose to reconstruct an egocentric map.

Capacity of the Semantic Finder -Unlike all other results presented in this paper, this experiment is performed independently of the official MultiON benchmark. We construct a synthetic dataset to evaluate the capacity of the Semantic Finder. More details about the generated data can be found in the supplementary material. As the granularity of the implicit representations is handled through the budget in terms of trainable parameters, we evaluate the capacity of the Semantic Finder f s to store large numbers of objects in Figure 4: In (a), we can see that increasing the number of objects up to 1000 does not increase error with sufficient gradient updates. However, since inputs are 1-in-K, increasing the number of objects also increases capacity. In (c), we see that this does not hold for random objects (not 1-in-K), whose less structured storage requires more capacity. In (b), we keep the input dimension fixed for random objects, which further increases error.

Evaluating catastrophic forgetting -we evaluate the capacity of the Semantic Finder f s to hold the learned information over the full length of the sequence in spite of the fact that it is continuously trained. Figure 5 shows the evolution of the mean error in distance for the predicted position of queried target objects as a function of time. The error quickly goes below 1.5m once the object has been seen the first time (t=0 in the plot), which is distance threshold required by the MultiON task, and stays there, providing evidence that the model does not suffer from catastrophic forgetting.

Runtime performance -inspite of requiring to continously train the representations, we achieve 45 fps during parallelized RL training, including the environment steps (simulator rendering), forward passes, representation training and RL training on a single V100 GPU. The average time of one agent forward pass, including updates of implicit representations is 20ms on a V100 GPU, which is equivalent to 50 fps, enough for real-time performance.

Conclusion

We introduce two implicit representations to map semantic, occupancy and exploration information. The first estimates the position of an object of interest from a vector query, while the second encapsulates information about occupancy and explored area in the current environment. We also introduce a global read directly from the trainable weights of this representation. Our experiments show that both implicit representations have a positive impact on the navigation performance of the agent. We also studied the scaling laws of the semantic representation and its behavior in the targeted lifelong learning problem. Future work will target differentiating through the implicit representations all the way back to the perception modules.

Acknowledgement -We thank ANR for support through AI-chair grant "Remember" (ANR-20-CHIA-0018). Figure 7 provides an example for an episode rollout from the minival set of the MultiON CVPR 2021 Challenge. The agent is equipped with both implicit representations. For each line, from left to right, we first see the RGB-D egocentric view of the agent, then a topdown map with the three goals (white, pink and yellow squares) and their estimated location by the Semantic Finder (white, pink and yellow dots, with a shaded region to denote uncertainty). The radius illustrating uncertainty is unit-less and only given for visualization purposes -is not available to the agent in this particular form. The third illustration shows the implicit map obtained by querying the Occupancy and Exploration Implicit Representation, and on the right, there is the reconstructed output when feeding the embedding of the global reader to the convolutional decoder it was trained with. The last element is a curve showing the evolution of the uncertainty estimation of the Semantic Finder on the currently provided target object.

In this episode, the agent starts with the white object within its field of view, but the first target to reach is the pink cylinder (Row 1). As we can see, the estimation of goal positions from the Semantic Finder are wrong, which is expected as the episode has not yet started. However, the associated uncertainty is high, allowing the agent to discard this information. The agent then explores until it observes the pink object (Row 2). At that point, the uncertainty about the object to find drops. The estimate of the position of the pink object will be updated as training samples will be added to the semantic replay buffer. Also note that at that point the estimate of the position of the white object from the Semantic Finder is accurate as the object has already been seen previously. The agent then goes towards the pink target object and calls the found action (Row 3). Estimation of the positions of pink and white objects are accurate. The next target to find is the white object. The uncertainty about the current target is low as the white object has already been observed. The agent backtracks (Row 4) and goes towards the white object to call the found action (Row 5). The next goal is the yellow cylinder. At that point, the uncertainty about the current target increases as the yellow cylinder has not yet been within the agent's field of view. The agent explores (Row 6) and when the target is within its field of view (Row 7) the uncertainty related to the target to find drops. The agent goes towards the yellow object and calls the found action (Row 8). At the end of the episode, the Semantic Finder is able to localize the 3 objects, and the associated uncertainties are low. All objects have been successfully found, so this episode is considered as a success.

B.2. Failure case

Figure 8 shows an example of unsuccessful episode. The agent and the setup are the same as described in the previous subsection. The agent can see the white target at the beginning of the episode, but it is far and largely occluded.The first target to find is the blue cylinder. It thus explores the scene until seeing the object. The uncertainty thus drops, the prediction of the target position on the map is now correct. It is interesting to see that the prediction of the location of the white target is also pretty accurate even though it was hard to detect at the beginning of the episode. The agent reaches the blue target and calls the Found action. The next target to find is the white object. The uncertainty is not 0 but relatively small as the agent has already seen the white target before. It goes towards the object, and when it is within its field of view, the uncertainty drops to 0 and the location prediction on the map is accurate. The agent calls the Found action. It has thus succeeded in finding the first two objects that were quite close to its initial position. However, as shown in the last 3 rows, the agent does not succeed in exploring the environment enough and never finds the last target which is the green cylinder. After a few steps, it calls the Found action at the wrong location.

C. Perception modules

Two different modules are used in this work. The first one, responsible for representation perception, extracts representations from the RGB-D observation to populate the training replay buffer of the Semantic Finder. The second one, tackling reactive perception, encodes the observation into a vector fed to the GRU. This representation of the observation is thus more directly used in the decision making process -the name Reactive is certainly not 100% accurate, since the output of this module is still used to update agent memory, but this concerns only the hidden GRU memory and not the main implicit representations.

Reactive perception We use the encoder module in [START_REF] Vincent Sitzmann | Implicit neural representations with periodic activation functions[END_REF] (ref from the main paper), which encodes visual observations at each step. Table 5 (Enc) presents the hyperparameters of the convolutional layers in this visual encoder. Is is composed of 3 convolutional layers follower by a linear layer. ReLU activations are used. The embedding produced by this visual encoder is fed to the GRU module. In our work, the reactive perception module has been pre-trained with auxiliary losses, which corresponds to the method in [START_REF] Lluvia | Active Mapping and Robot Exploration: A Survey[END_REF] (ref from the main paper). It is then frozen and not updated during RL training. Representation perception The goal of the representation perception module is to extract vectors to be added to the Semantic Finder training replay buffer. The backbone encoder is the same as the reactive perception module (see Enc in Table 5) also pre-trained from the agent in [START_REF] Lluvia | Active Mapping and Robot Exploration: A Survey[END_REF] (ref from the main paper). This network is augmented with a segmentation head and is fine-tuned end-to-end on the task of segmenting Mul-tiON target objects.

D. Algorithmitic description of an agent forward pass

Algorithm 1 gives a high-level overview of the different steps happening after receiving the current observation from the environment to take the most suitable action.

D.1. Lines 1 -5 -Adding training samples to the semantic replay buffer

The segmentation map mt is obtained by passing the RGB-D observation ot through the representation perception module, i.e. a segmentation model pre-trained to segment the target objects (Line 2). An inverse projection operation, denoted invP roj() is used to project pixels from ot into their 3D coordinates nt using the depth channel of ot and the known camera intrinsics K (Line 3). A meanpooling operation, denoted meanP ooling() is then applied to nt in order to obtain the mean 3D coordinates of all pixels in each cell of the segmentation map mt (Line 4). Finally, pairs of softmax distribution over classes from mt and mean 3D coordinates are added to the training replay buffer of the Semantic Finder. This is implemented as the addSemSamples() in the algorithm (Line 5).

D.2. Lines 6 -8 -Adding training samples to the occupancy replay buffer

The 3D coordinates of projected pixels in nt are compared with threshold values along their vertical y coordinate to be either labelled as navigable space or obstacle. Only 3D points with their vertical coordinate below than another threshold value are kept.

D.4. Lines 14 -22 -Updating the Occupancy and Exploration Implicit Representation

The implicit representation is iteratively updated for a maximum of no steps while the error of the model (loss, initialized to 0 in line 15) is higher than a threshold. Same as for the Semantic Finder, a training batch bo is first sampled (getOccBatch(), line 18). The model is then evaluated on samples from bo (Line 19). The SGD() function is then applied to update the implicit representation. In this work, we chose no = 20 and thresh = 0.3.

E. Capacity of the Semantic Finder

This section provides further details about the study conducted to evaluate the impact of number of objects and the nature of their representation on the capacity of the Semantic Finder to memorize their position, and is complementary to the paragraph "Capacity of the Semantic Finder" in Section 4 and Figure 3 of the main paper.

To be flexible in the amount of objects we can use, we perform these experiments independently of the official MultiON benchmark. We consider three new scenarios, and for each one, a dataset is generated and used to train the Semantic Finder. All datasets are made of (query, position) pairs with positions being uniformly sampled between arbitrary scene bounds (between 0 and 1 along each axis). For each dataset, we also create variants varying the sample size. To reduce the amount of hyper-parameters (e.g. batch size), we ressort to gradient descent as opposed to stochastic gradient descent, i.e. each gradient step is computed over the whole dataset. The considered metrics is the mean L1 error on the prediction of positions as a function of the number of objects to memorize. The difference between the three scenarios is in the nature of queries associated with positions, and each scenario corresponds to a sub figure of Figure 3 in the main paper.

In Figure 3a, for a given size of the dataset, i.e. for given number of objects, each query is a 1-in-K encoded vector of the object category, which means that the query dimensions grows with growing numbers of objects. This evaluates the representation in situations where objects are identified by a unique class index. Provided a sufficient number of gradient steps, we can see that the error stays low even with an increasing number of objects. We conjecture that the good performance of this setting is due to the growth in, both, query size and thus capacity of the model (as the query is the input to the model) as the number of objects grows.

In Figure 3b, the query vectors have a fixed dimension of 9, equivalent to the dimension in the MultiON benchmark. Queries are not 1-in-K encoded, but composed of randomly sampled values. Even though more gradient steps are helpful, the conclusion here is that increasing the number of objects has a negative impact on the mean error of the model. Unlike in (a), the number of parameters does not increase here with number of objects as queries have a fixed size. This is thus an illustration of the challenge to memorize an increasingly high number of objects with a fixed model capacity.

Figure 3c is a combination of (a) and (b) with query size increasing with number of objects and queries composed of random values (no one-hot vectors). The increase in model capacity with more objects seems again to be beneficial provided enough gradient steps. However, its is clear that the positions associated with random queries are more difficult to memorize than for one-hot queries. This emphasizes the importance of the chosen query representation when building query-able semantic implicit representations.

F. Agent training details

All agents evaluated in this work are trained with Proximal Policy Optimization (PPO) [44], following settings from previous work [START_REF] Vincent Sitzmann | Implicit neural representations with periodic activation functions[END_REF][START_REF] Lluvia | Active Mapping and Robot Exploration: A Survey[END_REF]. We provide a formulation of the reward function and the PPO hyper-parameters in the next sub-sections.

where 1 reached t is the indicator function that equals 1 if the Found action was called at time t while being close enough to the target, and 0 otherwise. Rcloser is a reward shaping term taking as value the decrease in geodesic distance to the next goal compared to previous timestep. Finally, Rtime-penalty is a negative slack reward to encourage taken paths to be as short as possible.

PPO hyper-parameters -Table 6 presents the PPO hyperparameter values used to train all agents in this work.

G. Amount of compute

Table 7 shows the compute resources used to train, validate and test the different models involved in results presented in Table 1 and 2 in the main paper.

H. Limitations of the work

The proposed approach has the following limitations • The Semantic Finder can not deal with several instances of the same object type. This is not a problem when considering the MultiON task, but will be addressed in future work.

• The Semantic Finder only provides the position of an object of interest. Thus, it does not provide the agent with any information about how to reach the given target. Outputting a geodesic distance, and even a shortest path to the object would be interesting as future work.

• The current formulation of the uncertainty necessitates access to the full replay buffer of the episode. Future work will target estimating the uncertainty directly from the weights of the implicit representation.

I. Leveraging environment regularities and semantic priors in implicit representations

In this work, the weights of neural implicit representations are initialized from scratch at the beginning of each new episode and optimized in real time as the agent interacts with the environment. This is done in the same way as an explicit map would be updated on the fly during navigation.

K. Importance of Occupancy and Exploration information

In an instance based post-hoc analysis, we attempt to visualize the way how the implicit Occupancy and Exploration Representation is used by the agent with two different examples. Without the Occupancy and Exploration Representation -the agent fails to find a target that was already observed in the past: at the beginning of the episode, the agent observes the white cylinder, while the current target is the pink object. The agent thus explores the environment, finds it and properly calls the Found action. The next target is then the white object. As can be seen, the Semantic Finder properly locates the target, since it had been previously observed. However, the agent fails to backtrack, first entering a room without finding a path to the goal and then going in the wrong direction. It finally calls the Found action while not being close to the white target.

K.1. Occupancy information

The full agent -also having access to the Occupancy and Exploration Implicit Representation succeeds in reaching the white Table 7. Amount of compute: GPU days for runs involved in Tables 1 and2 in the main paper.

target (after finding the pink target, not shown on the Figure). Moreover, in visualizations of, both, the implicit representation fo and the reconstruction from the latent representation extracted by the Global Reader r, we can see the path to the goal (visualized as a red arrow) marked as explored area. All the information for the agent to correctly navigate towards the white object is thus contained in the implicit map and its global summary vector. This analysis cannot corroborate that the agent indeed used the representation as explained; however, we can at least provide evidence that the (successful) full agent had access to information, which was crucial to solve a task, on which the baseline agent failed. Without the Occupancy and Exploration Representationthe agent fails to explore the scene to find a target: the first target object is black. To this end, the agent must explore the environment. It starts to explore but misses a part of the scene (containing the black target), which is never explored for the rest of the episode. It finally calls the Found action far away from the black target.

K.2. Exploration information

The full agent -also having access to the Occupancy and Exploration Implicit Representation succeeds in finding the black target. It successfully explores the part of the scene that contains the object. In visualizations of, both, the implicit representation fo and the reconstruction from the latent vector extracted by the Global Reader r, we can see that the area to observe (visualized by a red shape) is at the frontier between explored and unexplored parts of the scene. This information can thus guide the agent to move towards the unexplored area.

Without Global Reader

Target:

Target:

Target:

Target:

With Global Reader

Target:

Without Global Reader

Target:

With Global Reader

Target:

Target:

Target:

Target:

Target:

Figure 1 .

 1 Figure 1. We propose two implicit representations as inductive biases for autonomous agents -both are learned online during each episode. ➁ a semantic representation fs predicts positions x from goals g given as semantic codes. We show Ground Truth object positions (rectangles) and predictions (round; radius shows uncertainty, unit-less, as an illustration). Blue and pink objects have been observed, but not the yellow target. ➂ A structural representation fo predicts occupancy and exploration s from positions x; we provide a global read which directly maps from function space fo (represented by trainable weights θo) to a context embedding e used by the agent. ➃ shows the reconstruction produced by a decoder Dec during training. Orange=navigable, Green=Obstacles, Blue=Unexplored. ➀ a ground-truth map is shown for reference, simulating a fully explored scene.

  provides an overview of the 3 steps involved in the training of the global reader.Fully convolutional autoencoder -The first step is to train a fully convolutional autoencoder on the set of absolute maps M i , i..1..25k. Only the decoder weights are kept. Global reader training on absolute maps -The second step consists in training the global reader to output embeddings fed to the frozen decoder from the previous step. The objective is to reconstruct absolute maps from the weights of the implicit representation. The global reader weights are kept after this training phase.

Figure 3 .

 3 Figure 3. Training the Global Reader in 3 steps: ➀ training a convolutional auto-encoder on absolute maps; ➁ extension to inputs equal to weights θo = the gobal reader; ➂ extention to predicting ego-centric maps, better suited to navigation problems. 0 -30 30 -50 50 -70 -Val --Test -S O S O S O Success Progress SPL PPL Success Progress SPL PPL

Figure 4 .

 4 Figure 4. Capacity of the semantic representation: we report mean distance prediction error (normalized ∈ [0, 1]) as a function of the number of stored objects. Replay buffers are composed of dummy queries: (a) one-hot queries with same dimension as number of objects; (b) random query with dimension 9; (c) random query with same dimension as number of objects.

Figure 5 .

 5 Figure 5. Lifelong learning of the semantic repr. fs: we report mean error in meters, test set, 600 episodes, as a function of the number of time steps since the object was first seen in the episode (t=0). The error falls immediately and stays low over the episode.

Figure 6 .

 6 Figure 6. Training stability: (a) Evolution of the collected reward on training episodes for the 3 models presented in Table 1 in the main paper. (b) Evolution of PPL, the official ranking metrics in the MultiON Challenge leaderboard, on val episodes for model checkpoints from the last 10M training frames.

21 j

 21 15 loss = 0 16 j = 0 17 while loss > thresh and j < n o do 18 b o = getOccBatch(r o ) 19 loss = eval(b o , θ o,t ) 20 θ o,t = SGD(b o , θ o,t ) = j + 1 22 end while These comparisons are done in the labelOccP os() function (Line 7). Pairs of label and 3D coordinates are then sampled in order to keep the balance between the two classes and added to the training replay buffer of the Occupancy and Exploration Implicit Representation. This is the addOccSamples() function (Line 8). D.3. Lines 9 -13 -Updating the Semantic Finder Two operations are repeated ns times. First a batch of training examples bs is sampled (getSemBatch(), line 11). Then, the SGD() function encapsulates the forward pass of fs on the sampled batch, the L1 loss computation, gradient computation and finally backpropagation. In this work, we fixed ns = 1.

Figure 10 shows

 10 Figure 10 shows an example episode from the MultiON 2021 challenge minival set targeted by two different agent variants.Without the Occupancy and Exploration Representation -the agent fails to find a target that was already observed in the past: at the beginning of the episode, the agent observes the white cylinder, while the current target is the pink object. The agent thus explores the environment, finds it and properly calls the Found action. The next target is then the white object. As can be seen, the Semantic Finder properly locates the target, since it had been previously observed. However, the agent fails to backtrack, first entering a room without finding a path to the goal and then going in the wrong direction. It finally calls the Found action while not being close to the white target.The full agent -also having access to the Occupancy and Exploration Implicit Representation succeeds in reaching the white

Type 0 -

 0 70 30 -70 50 -70 S O Nb episodes Nb GPUs GPU GPU time Nb runs Tot.

Figure 11

 11 Figure 11 shows another example episode taken from the MultiON 2021 challenge minival set.Without the Occupancy and Exploration Representationthe agent fails to explore the scene to find a target: the first target object is black. To this end, the agent must explore the environment. It starts to explore but misses a part of the scene (containing the black target), which is never explored for the rest of the episode. It finally calls the Found action far away from the black target.The full agent -also having access to the Occupancy and Exploration Implicit Representation succeeds in finding the black target. It successfully explores the part of the scene that contains the object. In visualizations of, both, the implicit representation fo and the reconstruction from the latent vector extracted by the Global Reader r, we can see that the area to observe (visualized by a red shape) is at the frontier between explored and unexplored parts of the scene. This information can thus guide the agent to move towards the unexplored area.

  object seen: Drop in uncertainty about current target White object to find: Uncertainty still low as white object already seen Yellow object to find: High uncertainty as never seen before Yellow object seen: Drop in uncertainty about current target

Figure 7 .Figure 8 .

 78 Figure 7. Agent rollout on an example successful episode from the MultiON CVPR 2021 Challenge minival set. From left to right: RGB-D ego view, topdown map (viz only) with targets (squares) and their estimated location by the Semantic Finder (dots, shaded region for uncertainty), map from the Occupancy and Exploration Implicit Representation, reconstructed egomap from the global reader and CNN Decoder trained end-to-end, uncertainty of the Semantic Finder on the currently selected target.

Figure 9 .

 9 Figure 9. Comparison of top-down maps obtained by querying the Occupancy and Exploration Implicit Representation trained with (left) and without (right) Fourier features.

Figure 10 .

 10 Figure 10. Importance of occupancy information: Episode example from the MultiON 2021 challenge minival set where an agent without the proposed Occupancy and Exploration Implicit Representation fails to find the white target. Another agent equipped with the representation containing occupancy information finds the target. Some information about the path to reach it (indicated with the red arrow) is indeed contained in the map. The green cone corresponds to the agent's position.

Figure 11 .

 11 Figure 11. Importance of exploration information: Episode example from the MultiON 2021 challenge minival set where an agent without the proposed Occupancy and Exploration Implicit Representation fails to find the black target. Another agent equipped with the representation containing occupancy information finds the target. Some information about the area to explore (indicated with the red shape) is indeed contained in the map. The green cone corresponds to the agent's position.

t is Mapping = training representations Query Global read

		p
	o t Goal RGB-D observation g t	Segmentation	Inverse projection	• t to relevant variables to indicate time. Please note that the trainable parameters θ s,t and θ o,t of the two implicit representations are time dependent, as they depend on the observed scene and are updated dynamically, whereas the parameters of the policy π and the global reader r are not. Here, GRU corresponds to the update equations of a GRU network, where we omitted gates for ease of f o θ o θ s x x * L s s * L e x r notation. The notation a f s h t a t h t-1

t = π(.) predicting action a

Representation perception (pre-trained) Implicit representations Reactive Perception (pre-trained)

	c
	a t-1
	Vectorial memory	Action
	Figure 2. Navigating with implicit representations. Red connections	indicate the training process of the two implicit representations
	(=mapping), which is also done during agent deployment. Black connections	show the forward pass of the agent. are discrete learned
	embeddings (LUT). Policy training is not shown in this figure

Table 1 .

 1 Impact

		------	32.1	47.7	21.6	32.6	41.0	55.9	28.9	39.0
	↑	--✓ -✓ -	38.1	51.9	27.3	37.1	48.6	61.5	35.2	44.2
		--✓ -✓ ✓	43.1	56.8	30.5	40.1	49.7	63.4	36.4	45.9

1.8 --✓ -✓ -37.8± 1.6 52.3± 0.9 26.35± 1.5 36.5± 0.8 47.0± 1.7 60.5± 1.6 34.5± 0.8 44.2± 1.0 --✓ -✓ ✓ 38.5± 4.6 52.5± 4.8 28.2± 2.1 38.3± 1.7 46.7 ± 3.0 60.1 ± 3.1 35.1 ± 1.4 44.8 ± 1.0 of the implicit representations: navigation performance on MultiON val and MultiON test. S=fs activated, O=fo activated in the corresponding training period (see text). Top/µ: means over 3 runs; Bottom/↑: best validation seeds over 3 runs.

Table 2 .

 2 ✓ N/A 57.7 ± 3.7 70.2 ± 2.7 37.5 ± 2.0 45.9 ± 1.9 ✓ -(i) Implicit (Ours) w/o curriculum w/ pre-train + AUX ✓ ✓ ✓ 58.3± 0.8 69.4± 1.1 43.8± 1.0 52.1± 1.6 ✓ Implicit (Ours) w/o curriculum w/o pre-train + AUX -✓ ✓ 57.9± 2.0 69.5± 0.6 43.3± 2.2 51.9± 3.7 ✓ -Comparison with SOTA methods on MultiON test. †=performance taken from

		Agent	ρ α γ Success Progress SPL	PPL AUX ORC
	(a)	OracleMap † [56]	-✓	50.4± 3.5 60.5± 3.1 40.7± 2.2 48.8± 1.9 -	✓
	(b)	OracleEgoMap † [56]	-✓	32.8± 5.2 47.7± 5.2 26.1± 4.5 37.6± 4.7 -	✓
	(c)	NoMap † [56]	-✓	16.7± 3.6 33.7± 3.3 13.1± 2.4 26.0± 1.7 -	-
	(d)	ProjNMap † [26]	-✓	25.9± 1.1 43.4± 1.0 18.3± 0.6 30.9± 0.7 -	-
	(e)	NoMap	✓ -	42.3± 1.5 56.7± 0.9 28.1± 1.0 37.8± 1.8 -	-
	(f)	ProjNMap [26]	✓ -	39.7± 2.3 55.4± 1.4 28.7± 1.1 40.1± 1.9 -	-
	(g)	Implicit (Ours) w/ curriculum w/ pre-train	✓ --46.7± 3.0 60.1± 3.1 35.1± 1.4 44.8± 1.0 -	-
	(h)	ProjNMap + AUX [37]	N/A -
	(j)	Implicit (Ours) w/o curriculum w/o pre-train	-✓ ✓ 54.8± 3.6 68.0± 3.4 41.7± 1.9 51.3± 1.6 -	-
	(k)

Table 3 .

 3 α = finetuning of input encoders with RL. γ = implicit representations are accessible to the agent since the beginning of RL training (w/o curriculum). Uncertainty: comparing training w/ semantic input only, no occupancy, from the beg. of training, w/ and w/o uncertainty.

	Uncertainty Success Progress SPL	PPL
	-	35.4± 3.0 49.7± 3.3 29.4± 2.0 40.9± 2.4
	✓	43.4± 3.1 58.0± 3.0 35.1± 0.8 46.4± 1.0

p was trained to segment the different target objects. The generated dataset is composed of 132k pairs of RGB-D observations and segmentation masks. Samples for 4 scenes were kept as a validation set.

Table 5 (

 5 Seg) details the architecture

Table 1

 1

		0	TransposeConv2D	64	32	3	2	0	0
		1	TransposeConv2D	32	32	3	2	0	0
	Dec	2 3	TransposeConv2D TransposeConv2D	32 16	16 8	3 3	2 2	0 0	0 0
		4	TransposeConv2D	8	8	3	2	0	0
		5	TransposeConv2D	8	3	3	2	0	1
		0	Conv2D	4	32	8	4	0	-
	Enc	1	Conv2D	32	64	4	2	0	-
		2	Conv2D	64	32	3	1	0	-
		0	Conv2D	32	32	5	1	2	-
	Seg	1	Conv2D	32	32	5	1	2	-
		2	Conv2D	32	9	3	1	1	-

in the main paper. (b) Evolution of PPL, the official ranking metrics in the MultiON Challenge leaderboard, on val episodes for model checkpoints from the last 10M training frames. Model layer id type in channels out channels kernel size stride in padding out padding

Table 5 .

 5 Convolutional layers: hyperparameter values in the different presented CNN architectures. Dec is the CNN decoder trained with the global reader, Enc is the visual encoder used in both the representation and reactive perceptions, Seg is the segmentation head combined with Enc in the representation perception module c. of the segmentation head. It is composed of 3 convolutional layers with ReLU activations, except for the last layer where a softmax activation is applied. After this training phase, the weights of the representation perception module are frozen and not updated during RL training.

  Algorithm 1: Different steps necessary to update implicit representations at each agent step. Input :Observation o t , camera intrinsics K, goal g t , replay buffers r s and r o , weights θ s,t , θ o,t 1 // Adding training samples to the semantic replay buffer 2 m t = p(o t ) 3 n t = invProj(o t , K) 4 k t = meanPooling(n t ) 5 r s = addSemSamples(r s , m t , k t ) 6 // Adding training samples to the occupancy replay buffer 7 l t = labelOccPos(n t ) 8 r o = addOccSamples(r o , n t , l t ) 9 // Updating the Semantic Finder 10 for i ← 0 to n s -1 do 11 b s = getSemBatch(r s )

	12	θ s,t = SGD(b s , θ s,t )
	13 end for
	14 // Updating the Occupancy and
		Exploration Implicit
		Representation

Table 6 .

 6 PPO hyper-parameters: Values for hyper-parameters used when training all agents in this work.

	Reward

function --The reward function at time-step t is composed of three terms,

Rt = 1 reached t • Rgoal + Rcloser + Rtime-penalty

•

  Slower RL training compared to baseline agents. Even if reaching an average of 45 fps with 2 implicit representations updated with backpropagation at each agent step is already quite satisfying, we are still far from the 150 fps when training P rojN euralM ap or 200 fps for N oM ap.

  Training efficiency and the quality of the learnt representations are thus two important factors. Leveraging the knowledge about scene layout and semantic priors that was gained by each implicit representation to speed up the training of others and improving the quality of the provided mapping is thus a relevant future direction. Meta-learning better weight initializations, as was done in previous work (Sitzmann et al. MetaSDF: Metalearning Signed Distance Functions, NeurIPS 2020), or having a common backbone followed by randomly initialized layers for new episodes are two promising directions.Figure9compares the top-down map obtained after querying the Occupancy and Exploration Implicit Representation trained with and without Fourier features. On each plot, the left and right maps respectively show the impact of using and not using Fourier features. Without the latter, no detail about the environment layout can be reconstructed. This corroborates findings also reported in other literature on implicit representations and coordinate networks, e.g.

J. Importance of Fourier features

(Mildenhall et

al., NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis, ECCV 2020)

Appendix A. Training stability and curves

Figure 6 shows the training curves of the 3 different agents we compared in Table 1 of the main paper: the recurrent baseline agent (blue), with the Semantic Finder (orange) and with both implicit representations (green). Left, (a), we see the evolution of the training reward as a mean and standard deviation over 3 runs. Right, (b) shows PPL, the main metric chosen for ranking the agents in the MultiON learderboard, which we show for different checkpoints during training and evaluated on the validation set. As can be seen, training is quite stable over runs, and adding the two representations provides a boost in performance (as already reported in Table 1 in the main paper).

B. Visualization of the agent behavior


Marie Anne Chevallier

E Fantuzzi

M Hajek

S Mayer

R Tanner

J Thomas

F Vanhavere

D J Thomas

Analysis of EURADOS neutron intercomparison results according to new ISO standards criteria

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

One of the most important actions undertaken within EURADOS Working Group (WG) 2 is the organization of self-sustained programs of personal dosemeter intercomparisons (ICs). Indeed, WG 2 promotes actions to harmonize individual monitoring throughout Europe. Actions started in 2008, concerning whole-body photon dosemeters, are performed on a regular two-year basis. Some extremity IC exercises have been performed in the meanwhile. As far as neutron dosimetry is concerned, the first intercomparison exercise held by WG 2 was in 2012 (2)(3) [START_REF] Chevallier | EURADOS IC2012n: Further Information Derived from a EURADOS International Comparison of Neutron Personal Dosemeters[END_REF] . It was decided to perform such exercise on a five-year basis to reach a compromise between the need, imposed by national regulation, for the IMSs to participate in such exercises, and financial considerations since neutron irradiations are expensive. There is also the practical issue of the non-negligible impact of the mobilization of the few available neutron facilities. Consequently, the second EURADOS ICn exercise occurred in 2017 [START_REF] Mayer | EURADOS intercomparison 2017 for neutron dosemeters[END_REF] [START_REF] Mayer | Results of the EURADOS 2017 intercomparison for whole body neutron dosemeters (IC2017n)[END_REF] and another one is in progress in 2022.

Overview of the IC2012 and IC2017n exercises

These two exercises had very similar scopes. They focused on the neutron personal dosemeters, either passive or active, routinely used in individual monitoring of exposed workers, as provided by IMSs, and intended to measure neutron personal dose equivalent Hp (10). 31 IMSs participated in 2012 (vs 32 in 2017), with full results for 32 systems (vs 33 in 2017).

The main difference between the two exercises was prompted by the fact, raised during the process of IC2012n, that some IMSs could report results only if they had some a priori information about the neutron fields. This is part of their routine process. So, in 2012, a two-step procedure was used. Initially, very little information on the radiation fields was provided by the Organization Group (OG), and all IMSs submitted their results. In a second step, additional information on the radiation field was given so that the IMSs could choose the proper calibration factor to be applied. All IMSs were allowed to modify their results with justification to be agreed by the OG. In order to cope with problems introduced by this approach, the IC2017n exercise was performed via another process (one-step procedure): the participants had to state clearly during the registration process whether they required additional a priori field information according to their system and routine procedure of analysis. This fact was clearly stated in the participation certificate. To all the other participants, only very general information about the fields was provided. Another difference between the two exercises was the irradiation plan. Table 1 gives the similarities and differences of both irradiation plans. One can note that the lowest dose tested, in both exercises, was 0.3 mSv with a bare Cf-252 source.

Choice of the performance criteria

The basic principle of a dosimetry intercomparison is to expose dosemeters to accurately known doses in reference fields and to evaluate the responses. To evaluate the intrinsic quality of the response of a dosimetry system and to quantify the difference between systems, criteria are needed to determine what can be considered an acceptable under-response or an acceptable over-response. To perform a fair and accurate analysis of the results it is more appropriate to conduct it on the basis of procedures and criteria agreed by the scientific community.

During the IC2012n exercise, there were no applicable performance standards for the proficiency test for neutron personal dosemeters. The standard covering intercomparison exercises is ISO 14146 which gives test procedures, and also criteria, to be used for the periodic verification of the performance of dosimetry services supplying personal dosemeters [START_REF]Radiation protection -Criteria and performance limits for the periodic evaluation of dosimetry services 2[END_REF] . Unfortunately, in 2012, the scope of this document did not include neutron dosemeters, but only photon dosemeters. The need for criteria for neutron intercomparisons was emphasized by the EURADOS IC2012n organization group in their conclusions (2) .

There exist several organizations setting up procedures, type tests and criteria at an international level such as ISO (International Organization for Standardization) and IEC (International Electro technical Commission). At a national level, standards are available from organizations such as: DIN (Deutsches Institut für Normung, D) or ANSI (American National Standard Institute). Other organizations such as ICRU (International Commission on Radiation Units and Measurements) and ICRP (International Commission on Radiological Protection) also give guidelines and recommendations. These documents are used for reference by IMSs. Moreover, in many countries, the regulatory body relies on these international or national documents to fix the requirements for the performance of individual dosemeters. Nevertheless, standards are in constant evolution, being regularly revised and updated. At an international level, the standards which are relevant for personal dosimetry are of two kinds: those related to the realization and use of reference radiation fields, and standards giving the requirements and recommendations for testing the performance of personal dosemeters. There are not many standards dealing specifically with personal neutron dosimetry. Two exist at a national level:

 American standard ANSI/HPS N13.11-2009 [START_REF]11-2009[END_REF] defines the general criteria and testing requirements for establishing personal dosimetry performance, and ANSI/HPS N13.52-1999 (8) gives specific requirements and recommendations for neutron dosemeters. However, only tests with un-moderated

Cf-252 and D2O-moderated Cf-252 neutron sources are considered in these standards.

 The German standard for neutron dosimetry, DIN 6802 (9)(10)(11) is specific for dosimetry systems using the albedo technique and does not provide criteria for the performance of personal dosemeters, which are instead provided in guidelines by the German authority SSK (12) . International guidelines such as ICRU Report 66 [START_REF]Determination of Operational Dose Equivalent Quantities for Neutrons[END_REF] , cover the operational dose equivalent quantities for neutrons. However, none of these documents provides criteria for the performance of individual monitoring measurements for neutrons. They focus on describing concepts, neutron fields' characterization, experimental measurements, etc. That is the same for ICRP Publication 75 [START_REF]ICRP Publication n°75, General Principles for the Radiation Protection of Workers[END_REF] which gives recommendations for radiation protection of workers for any radiation field.

Only two documents at international level deal specifically with personal neutron dosimetry: IEC Standard 61526 (15) dealing with active personal dosemeters, and ISO Standard 21909 (16) concerning passive neutron dosemeters. They give performance limits depending on the irradiation quality (energy and angle) in IEC 61526 and depending on the considered techniques in ISO 21909 (thermoluminescence albedo dosemeters / solid state nuclear track etch detectors for instance) in the 2005 publication (available for IC2012n before being revised for a publication in 2021 (18) [START_REF] Chevallier | Revision of the 21909 standard on passive personal neutron dosemeters[END_REF] ). Moreover, both documents give type tests and performance criteria for the characterization of neutron personal dosemeter, but not performance criteria for proficiency testing, which is a very different way to test a dosimetry system. While standards on dosimeter type testing aim to test a mean behaviour of the dosimetry system with several dosemeters per irradiation configuration and cover a large range of irradiation qualities, the philosophy of proficiency testing is to check, on a regular basis, the performance of a dosemeter by sampling configurations, usually simulating the most commonly occurring conditions in routine applications (sometimes referred as to "simulated workplace conditions"). Moreover, in the ISO 14146 standard there is an approval criterion, added to the performance limit, that is fulfilled if the number of the outliers upon the whole number of the irradiated dosemeters does not exceed a defined limit (3) .

Chosen limits for the IC2012n exercise

For IC2012n the criterion for deciding on under-and over-reading was a factor of 2 limit as shown in eq. (1):

0.5 ≤ ≤ 2 (1)
where Href is the irradiated dose, defined as the conventional quantity value of the dose to which the dosemeters are irradiated, and Hmeas is the measured personal dose equivalent.

This slightly arbitrary choice was considered by the OG as a reasonable approach to deciding whether the neutron dosimetry was sufficiently reliable. It is in agreement with ICRP Publication 75, which reads under §251: "The Commission has noted that [….] in the workplace, where the energy spectrum and orientation of the radiation field are generally not well known, […] the overall uncertainty at the 95% confidence level in the estimation of the effective dose around the relevant dose limit may well be a factor of 1.5 in either direction for photons and may be substantially greater for neutrons of uncertain energy and for electrons."

The limits, based on ICRP Publication 75 recommendations, are in accordance with the criteria chosen in former intercomparison exercises (cf. Figure 1) and are similar, in terms of order of magnitude, to the criteria of IEC 61526 and ISO 21909.

Chosen limits for the IC2017n exercise

Fortunately, ISO Standard 14146 was updated in 2018 (3) , to include neutron dosimetry. The IC2017n exercise could thus be analysed in accordance with this new document. The performance criterion for the neutron dosemeter response, R, is expressed in terms of limits defined by a trumpet curve given by eq. ( 2):

0,5 • 1 - . / , / , ≤ 𝑅 = ≤ 2 (2)
where H0 is defined as the "lower dose limit", which is the "dose below which irradiations should not be performed", according to the definition in the ISO document.

Comparison of the criteria

Equation (2) shows that the higher acceptance level for the performance limit is always the same whatever the considered dose for the different tested configurations during an intercomparison exercise. An over-estimation of maximum +100% of the reference value is accepted. This upper limit is the same as the one chosen for IC2012n.

But the lower limit given by ISO 14146 depends on the values of both Href and H0, which was chosen by the OG to be 100 µSv.

The lower limits are then the ones shown in Table 2. Moreover, in ISO 14146, an approval criterion is associated with the performance limits: "The criteria stated in 7.1 are valid for a coverage interval with a coverage probability of 95 %. A maximum of one tenth of the dosemeters irradiated may exceed the limits." This approval criterion has implications for the results of IC2012n. It is thus of interest to reanalyse the results of IC2012n and IC2017n to quantify any modifications in the number of systems that would satisfy the approval criteria following these changes in the lower limit. The expected impact is a priori foreseen for the irradiation conditions with the lowest level of dose in both exercises: Cf-252 at 0° of incidence at 300 µSv.

RE-ANALYSIS OF IC2012N AND IC2017N

Table 3 gives the percentage of systems satisfying the chosen criteria for IC2012n (maximum of two outliers) according to the chosen limits (factor 2) versus the ISO 14146 (2018) ones. The main difference is for albedo dosemeters. This change is not due to results at the low dose configuration of 0.3 mSv, as initially expected. The change in the criteria induce changes to the number of outliers for the 250 keV irradiation quality. Table 4 gives the percentage of systems satisfying the criteria for IC2017n (maximum of two outliers) according to the chosen limits (or approval for ISO 14146). The main change is for the track etch dosemeters: only one system changed its status in regard of this performance criterion. And again, it is not for the low dose configuration at 0.3 mSv, but for the quite challenging workplace field: D2O-moderated Cf-252 behind a shadow block, at 1 mSv (cf. Figure 2).

CONCLUSION

In 2012, for the first neutron intercomparison organized for many years, there was no available performance criterion for proficiency testing for neutron individual dosimeters. ISO Standard 14146 "Radiological protection -Criteria and performance limits for the periodic evaluation of dosimetry services" was revised in 2018 so that criteria were available for IC1017n. The impact of the change of the criteria on the analysis of the two intercomparison exercises was expected to be for the configuration at the lowest dose tested (300 µSv) where there is the higher modification in the criteria chosen for IC2012n and IC2017n. However, looking in detail the data reveals that there was no change in the analysis for this configuration. Indeed, this low dose irradiation was performed with a bare Cf-252 source, which is a source commonly used by IMSs as a calibration neutron field. The impact in the change of the criteria appears to be for more challenging configurations such as 250 keV or simulated-workplace fields. This conclusion underlines the fact that the key issue for neutron dosemeters remains the energy and angle dependence of the response. This issue has also had to be addressed in the revision of the ISO Standard 21909. Indeed, notwithstanding the revision of ISO Standard 14146, ISO Standard 21909 has been further revised since 2012. This document gives type-tests and performance criteria for passive neutron dosemeters (17)(18) . During this revision process, it was agreed among the ISO experts that some systems will not be able to fulfil the requirements in term of energy and angular responses, as some systems present a wide variation in their energy response. The fact that some types of dosemeters need a priori information of the neutron field at which they are exposed to is specific to neutron dosimetry. To cope with this problem, a second part was written giving some methodologies and criteria for the qualification of personal dosimetry systems in workplaces (18) [START_REF] Chevallier | Revision of the 21909 standard on passive personal neutron dosemeters[END_REF] . As far as intercomparison exercises are concerned, this issue was raised during the first intercomparison exercise in 2012 and explains the need for the specific two-step procedure. Finally, the energy and angle dependence of the response is definitely the most challenging issue for neutron dosimetry systems. The hardest configurations to measure are the workplace fields. This is quite a paradox as workplace fields are the most representative of the ones the workers are exposed to routinely. This underlines one main difficulty of neutron dosimetry i.e., that the neutron calibration fields are not representative of the field encountered in routine at workplaces. Neutron dosimetry remains then the most challenging personal dosimetry.

Table 1. Irradiation plans for IC2012n and IC2017n exercises (2) [START_REF] Mayer | EURADOS intercomparison 2017 for neutron dosemeters[END_REF] .

Irradiation quality IC2012n IC2017n

Bare Figure 2. Responses of all dosimetry systems (Albedos on the top, and track etch detectors on the bottom) for the workplace field: 252-Cf (D2O) behind shadow block @ 1 mSv, tested during the IC2017n exercise [START_REF] Mayer | EURADOS intercomparison 2017 for neutron dosemeters[END_REF] .

Figure 1 .

 1 Figure 1. Synthesis of criteria considered to quantify neutron personal dosimetry reliability (2) .

Table 2 .

 2 Low limit criteria according to the ISO 14146 for the doses for the IC2012n and IC2017n.

	252 Cf source at 0°	0.3 mSv	0.3 mSv
		3 mSv	1.5 mSv
		15 mSv	12 mSv
	250 keV mono-energetic neutrons, at 0° 1 mSv	-
	Bare 252 Cf source at 45°	2 mSv	-
	Bare 252 Cf source at 0° + additional photons	-	1.5 mSv
	(+ 1 mSv photons)
	241 Am-Be at 0°	-	1.5 mSv
	252 Cf(D2O) moderated source, at 0°	3 mSv 1.2 mSv
	Bare 252 Cf source + shadow cone, at 0°	2 mSv	-
	252 Cf (D2O) moderated source behind a	-	1 mSv
	shadow block, at 0°

Table 3 .

 3 Percentage of systems satisfying the approval criteria for IC2012n. The category 'other' consists in neutron dosimetry systems based on fission track detection or electronic ones based on silicon diodes.

		All	Track etch	Albedo Others
	Limits [0,5;2]	34%	47%	17%	33%
	ISO 14146 Limits (H0=100µSv)	44%	53%	33%	33%

Table 4 .

 4 Percentage of systems satisfying the approval criteria for IC2017n.

		All	Track	Albedo
			etch
	Limits [0,5;2]	42%	44%	40%
	ISO 14146 Limits	45%	50%	40%
	(H0=100µSv)