Shizhe Chen

Pierre-Louis Guhur

Makarand Tapaswi

Cordelia Schmid

Ivan Laptev

Learning from Unlabeled 3D Environments for Vision-and-Language Navigation

Keywords: Vision-and-Language, Navigation, 3D Environments

. We generate a navigation graph for each building and transfer object predictions from 2D to generate pseudo 3D object labels by cross-view consistency. We then fine-tune a pretrained language model using pseudo object labels as prompts to alleviate the cross-modal gap in instruction generation. Our resulting HM3D-AutoVLN dataset is an order of magnitude larger than existing VLN datasets in terms of navigation environments and instructions. We experimentally demonstrate that HM3D-AutoVLN significantly increases the generalization ability of resulting VLN models. On the SPL metric, our approach improves over state of the art by 7.1% and 8.1% on the unseen validation splits of REVERIE and SOON datasets respectively.

Introduction

Having a robot carry out various chores has been a common vision from science fiction. Such a long-term goal requires an embodied agent to understand our human language, navigate in the physical environment and interact with objects. As an initial step towards this goal, the vision-and-language navigation task (VLN) [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF] has emerged and attracted growing research attention. Early VLN approaches [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF][START_REF] Ku | Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding[END_REF] provide agents with step-by-step navigation instructions to arrive at a target location, such as "Walk out of the bedroom. Turn right and walk down the hallway. At the end of the hallway turn left. Walk in front of the couch and stop". While these detailed instructions reduce the difficulty of the task, they lower the practical value for people when commanding robots in real Fig. 1: HM3D-AutoVLN dataset. We use 900 unlabeled 3D buildings from the HM3D [START_REF] Ramakrishnan | Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai[END_REF] dataset. We improve labels obtained with a 2D segmentation model based on 3D cross-view consistency (middle, see Fig. 2). Then we rely on these pseudo labels to generate instructions via a speaker model (right, see Fig. 3). We automatically create over 200K realistic training samples for VLN. life. Thus, more recent VLN methods [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF][START_REF] Zhu | Soon: Scenario oriented object navigation with graph-based exploration[END_REF] focus on high-level instructions and request an agent to find a specific object at the target location, e.g., "Go to the living room and bring me the white cushion on the sofa closest to the lamp". The agent needs to explore 3D environments on its own to find the "cushion".

Compared to step-by-step instructions, following such high-level goal-driven instructions is more challenging. As there is no detailed guidance, an agent needs to learn about the structure of the environment for effective exploration. However, most existing VLN tasks such as REVERIE [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF] or SOON [START_REF] Zhu | Soon: Scenario oriented object navigation with graph-based exploration[END_REF] are based on 3D scans from the Matterport3D (MP3D) dataset [START_REF] Chang | Matterport3d: Learning from rgb-d data in indoor environments[END_REF], and contain less than 60 buildings and approximately 10K training trajectories. The limited amount of training data makes VLN models overfit to seen environments, resulting in less generalizable navigation policies in unseen environments. Manually collecting more VLN data is however expensive and not scalable. To address this data scarcity issue, previous works have investigated various data augmentation methods, such as synthesizing more instructions and trajectories in seen environments by a speaker model [START_REF] Fried | Speaker-follower models for vision-and-language navigation[END_REF], environment dropout [START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF] or editing [START_REF] Li | Envedit: Environment editing for vision-and-language navigation[END_REF], and mixing up seen environments [START_REF] Liu | Vision-language navigation with random environmental mixup[END_REF]. Nevertheless, these approaches are still based on a small amount of 3D environments that cannot cover a wide range of objects and scenes. To address visual diversity, VLN-BERT [START_REF] Majumdar | Improving vision-and-language navigation with image-text pairs from the web[END_REF] utilizes image-caption pairs from the web to improve generalization ability, while Airbert [START_REF] Guhur | Airbert: In-domain pretraining for vision-and-language navigation[END_REF] shows that pairs from indoor environments, the BnB dataset, are more beneficial for VLN tasks. However, it is hard for image-caption pairs to mimic the real navigation experience of an agent in 3D environments making it challenging to learn a navigation policy with action prediction.

In this work, we propose a new data generation approach to improve the model's generalization ability to unseen environments by learning from largescale unlabeled 3D buildings (see Fig. 1). We take advantage of the recent HM3D dataset [START_REF] Ramakrishnan | Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai[END_REF] consisting of 900 buildings in 3D. However, this data comes without any labels. In order to generate high-quality instruction-trajectory pairs on a diverse set of unseen environments, we use large-scale pretrained vision and language models. We first use an image segmentation model [START_REF] Cheng | Masked-attention mask transformer for universal image segmentation[END_REF] to detect 2D objects for images in the environment, and utilize cross-view consistency in 3D to increase the accuracy of pseudo 3D object annotations. Then, we fine-tune a language model, GPT-2 [START_REF] Radford | Language models are unsupervised multitask learners[END_REF], with pseudo object labels as prompts to generate high-level navigation instructions to this object. In this way, we construct the HM3D-AutoVLN dataset that uses 900 3D buildings, and consists of 36,562 navigable nodes, 172,000 3D objects and 217,703 object-instruction-trajectory triplets for training -an order of magnitude larger than prior VLN datasets. We train multiple state-of-the-art VLN models [8, [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF][START_REF] Hong | Vln BERT: A recurrent vision-and-language BERT for navigation[END_REF][START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF] with the generated HM3D-AutoVLN data and show significant gains. Specifically, we improve over the state-of-the-art DUET model [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF] on REVERIE and SOON datasets by 7.1% and 8.1% respectively. In summary, our contributions are as follows:

-We introduce an automatic approach to construct a large-scale VLN dataset, HM3D-AutoVLN, from unlabeled 3D buildings. We rely on 2D image models to obtain pseudo 3D object labels and on pretrained language models to generate instructions. -We carry out extensive experiments on two challenging VLN tasks, REVERIE and SOON. The training on HM3D-AutoVLN dataset significantly improves the performance for multiple state-of-the-art VLN models. -We provide insights on data collection and challenges inherent to leveraging unlabeled environments. It suggests that the diversity of environments is more important than the number of training samples alone.

Related Work

Vision-and-language navigation. The VLN tasks have recently been popularized with the emergence of various supportive datasets [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF]6,[START_REF] Irshad | Hierarchical cross-modal agent for robotics visionand-language navigation[END_REF][START_REF] Krantz | Beyond the nav-graph: Vision-and-language navigation in continuous environments[END_REF][START_REF] Ku | Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding[END_REF][START_REF] Padmakumar | Teach: Task-driven embodied agents that chat[END_REF]48,[START_REF] Thomason | Vision-and-dialog navigation[END_REF]. Different instructions define the variations of VLN tasks.

Step-by-step instructions [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF]6,[START_REF] Ku | Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding[END_REF] require an agent to strictly follow the path, whereas goal-driven high-level instructions [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF][START_REF] Zhu | Soon: Scenario oriented object navigation with graph-based exploration[END_REF] mainly describe the target place and object and command the agent to retrieve a particular remote object. The embodied question answering task asks an agent to navigate and answer a question [START_REF] Das | Embodied question answering[END_REF][START_REF] Yu | Multi-target embodied question answering[END_REF], and vision-and-language dialog [START_REF] De Vries | Talk the walk: Navigating new york city through grounded dialogue[END_REF]38,[START_REF] Padmakumar | Teach: Task-driven embodied agents that chat[END_REF][START_REF] Thomason | Vision-and-dialog navigation[END_REF] tasks use interactive communications with agents to guide the navigation. Due to the inherent multimodal nature, works on VLN tasks provide appealing and creative model architectures, such as cross-modal attention mechanism [START_REF] Fried | Speaker-follower models for vision-and-language navigation[END_REF][START_REF] Ma | Selfmonitoring navigation agent via auxiliary progress estimation[END_REF][START_REF] Ma | The regretful agent: Heuristicaided navigation through progress estimation[END_REF][START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF][START_REF] Wang | Active visual information gathering for vision-language navigation[END_REF], awareness of objects [START_REF] Hong | Language and visual entity relationship graph for agent navigation[END_REF]37,[START_REF] Qi | The road to know-where: An object-and-room informed sequential bert for indoor visionlanguage navigation[END_REF], sequence modeling using transformers [8, [START_REF] Hong | Vln BERT: A recurrent vision-and-language BERT for navigation[END_REF][START_REF] Pashevich | Episodic transformer for vision-and-language navigation[END_REF], Bayesian state tracking [START_REF] Anderson | Chasing ghosts: Instruction following as bayesian state tracking[END_REF], and graph-based structures for better exploration [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF][START_REF] Deng | Evolving graphical planner: Contextual global planning for vision-and-language navigation[END_REF][START_REF] Wang | Structured scene memory for vision-language navigation[END_REF]. The models are usually pretrained in a teacher forcing manner [8,[START_REF] Hao | Towards learning a generic agent for vision-and-language navigation via pre-training[END_REF] and then finetuned using student forcing [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF][START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF], stochastic sampling [START_REF] Li | Robust navigation with language pretraining and stochastic sampling[END_REF], or reinforcement learning [8,[START_REF] Hong | Vln BERT: A recurrent vision-and-language BERT for navigation[END_REF][START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF]53]. While most existing VLN works focus on discrete environment with predefined navigation graphs, VLN in continuous environments [START_REF] Krantz | Waypoint models for instruction-guided navigation in continuous environments[END_REF][START_REF] Krantz | Beyond the nav-graph: Vision-and-language navigation in continuous environments[END_REF] is more practical in real world [START_REF] Anderson | Sim-to-real transfer for vision-and-language navigation[END_REF]. The discrete environments also prove to be beneficial for continuous VLN such as providing waypoint supervision [START_REF] Raychaudhuri | Language-aligned waypoint (law) supervision for vision-and-language navigation in continuous environments[END_REF] and enabling hierarchical modeling of low-level and high-level actions [7,[START_REF] Hong | Bridging the gap between learning in discrete and continuous environments for vision-and-language navigation[END_REF].

Data-centric VLN approaches. One of the major challenges of VLN remains the scarcity of training data, leading to a large gap between seen and unseen environments. Data augmentation is one effective approach to address over-fitting, such as to dropout environment features [START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF], change image styles [START_REF] Li | Envedit: Environment editing for vision-and-language navigation[END_REF], mixup rooms in different environments [START_REF] Liu | Vision-language navigation with random environmental mixup[END_REF], generate more path-instruction pairs from speaker models [START_REF] Fried | Speaker-follower models for vision-and-language navigation[END_REF][START_REF] Fu | Counterfactual vision-and-language navigation via adversarial path sampler[END_REF][START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF] and new images from GANs [START_REF] Koh | Pathdreamer: A world model for indoor navigation[END_REF]. However, those data augmentation methods are still based on a limited number of environments, abating generalization ability on unseen environments. Contrary to the above, VLNBert [START_REF] Majumdar | Improving vision-and-language navigation with image-text pairs from the web[END_REF] and Airbert [START_REF] Guhur | Airbert: In-domain pretraining for vision-and-language navigation[END_REF] exploit abundant web image-captions to improve generalization. Nevertheless, such data do not fully resemble real navigation experience and thus they can only be used to train a compatibility model to measure path-instruction similarity instead of learning a navigation policy. In our work, photo-realistic 3D environments are used to learn a navigation policy.

3D environments. 3D environments and simulation platforms [START_REF] Kolve | AI2-THOR: An interactive 3d environment for visual ai[END_REF][START_REF] Li | igibson 2.0: Object-centric simulation for robot learning of everyday household tasks[END_REF][START_REF] Puig | Virtualhome: Simulating household activities via programs[END_REF][START_REF] Savva | Habitat: A platform for embodied ai research[END_REF] are a basic foundation to promote research in embodied intelligence. There are artificial environments based on video game engines such as iThor [START_REF] Kolve | AI2-THOR: An interactive 3d environment for visual ai[END_REF] and Vir-tualHome [START_REF] Puig | Virtualhome: Simulating household activities via programs[END_REF], which utilize synthetic scenes and allow interactions with objects. However, synthetic scenes have limited visual diversity and do not fully reflect the real world. Thus, we focus on photo-realistic 3D environments for VLN. The MP3D [START_REF] Chang | Matterport3d: Learning from rgb-d data in indoor environments[END_REF] dataset is a collection of 90 labeled environments and is most widely adopted in existing VLN datasets such as R2R [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF], RxR [START_REF] Ku | Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding[END_REF], REVERIE [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF] and SOON [START_REF] Zhu | Soon: Scenario oriented object navigation with graph-based exploration[END_REF]. Though possessing high-quality reconstructions, the number of houses in MP3D is limited. The Gibson [START_REF] Xia | Gibson env: Realworld perception for embodied agents[END_REF] dataset contains 571 scenes, however they have poor 3D reconstruction quality. The recent HM3D dataset [START_REF] Ramakrishnan | Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai[END_REF] has the largest number of 3D environments though with no labels. It contains 1,000 high-quality and diverse building-scale reconstructions (900 publicly released) around the world such as multi-floor residences, stores, offices, etc. In this work, we employ the unlabeled HM3D dataset to scale up VLN datasets.

3 Generating VLN Dataset from Unlabeled 3D Buildings

In this section, we present our approach to automatically generate large-scale VLN data from unlabeled 3D environments, specifically HM3D [START_REF] Ramakrishnan | Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai[END_REF]. We consider a discrete navigation environment [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF][START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF][START_REF] Zhu | Soon: Scenario oriented object navigation with graph-based exploration[END_REF], which is the mainstream setting for VLN. The discrete setup treats each environment as an undirected graph G = {V, E}, where V denotes navigable nodes, and E denotes connectivity edges. Each node V i ∈ V corresponds to a panoramic RGB-D image captured at its location (x, y, z). An agent is equipped with a camera and GPS sensors and moves between connected nodes on this navigation graph.

Fig. 2: We automatically construct the navigation graph and label 3D objects for each unlabeled 3D building.

We aim to generate VLN data with high-level instructions (similar to [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF][START_REF] Zhu | Soon: Scenario oriented object navigation with graph-based exploration[END_REF]), which mainly describe the target scene and specific object instance. The success of the agent is defined as arriving at a node where the target object is close and visible and correctly localizing the object. Therefore, the training data should consist of a language instruction, an initial position and orientation of the agent, a ground-truth trajectory to the target location and the position of the target object. In order to generate such training data, it is essential to know object locations in the 3D space instead of bounding boxes in 2D images. Otherwise, it can be hard to infer relationships between objects in different views leading to unreliable target locations. Moreover, a sentence that refers to a specific object should be generated since the object name alone is not specific. Generation of such data is challenging because the original 3D buildings come without any labels. Therefore, we propose to use image and text models pretrained on large-scale external datasets for pseudo labeling. This involves two stages: (i) building-level pre-processing to obtain navigation graphs and 3D object annotations (Sec. 3.1); and (ii) object-trajectory-instruction triplet generation (Sec. 3.2).

Building-level Pre-processing

We utilize the Habitat simulator [START_REF] Savva | Habitat: A platform for embodied ai research[END_REF] to load 3D buildings in HM3D [START_REF] Ramakrishnan | Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai[END_REF] such that agents can navigate. This is a continuous navigation environment, thus we first convert each environment into a discrete navigation graph and then extract 3D pseudo object annotations for each building as illustrated in Fig. 2.

Constructing navigation graphs. We follow the characteristics of navigation graphs in the MP3D simulator [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF] to create new graphs for continuous HM3D. For example, the average distance between connected nodes is around 2 meters and connectivity is defined by whether two nodes are visible and navigable from each other. However, the MP3D simulator relies on human efforts to create such graphs where the nodes are selected and navigability is inspected manually. Instead, we build the graph in a fully automatic way. We first randomly sample 20,000 navigable locations in the 3D environment and then greedily add locations as new nodes into the graph. The newly added location is the nearest one to existing nodes in the graph among all remaining candidates that are more than 2 meters away from the existing nodes. After collecting all the nodes, we use two criteria to connect different nodes: (i) the geodesic distance between nodes is below 3 meters; and (ii) the average distance of the depth image captured from one point to another should be larger than 2 meters. Fig. 2 (left) presents an example of navigation graph generation on one floor of the environment. For each node, we extract 36 RGB-D images from different orientations to represent the panoramic view, following the VLN setting [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF].

Labeling 3D objects from 2D predictions. Since existing 3D datasets such as Scannet [START_REF] Dai | Scannet: Richly-annotated 3d reconstructions of indoor scenes[END_REF] are small and contain a limited number of object classes, pretrained 3D object detectors are incapable of providing high-quality 3D object labels for HM3D which covers a diverse set of scenes and objects. Therefore, we propose to use an existing 2D image model to label 3D objects. Specifically, we use a panoptic segmentation model Mask2Former [START_REF] Cheng | Masked-attention mask transformer for universal image segmentation[END_REF] trained on ADE20K [START_REF] Zhou | Scene parsing through ade20k dataset[END_REF] to generate instance masks for all images in the constructed graph. We project 2D pixel-wise semantic predictions into 3D point clouds using the camera intrinsic parameters and depth information, and thus obtain 3D bounding boxes. However, due to the domain gap between existing 2D labeled images and images from HM3D, the predictions of 2D models can be noisy as shown in Fig. 5. Even worse, a 3D object is often partially observed from one view, making the estimated 3D bounding box from a single location less accurate. In order to further improve 3D object labeling, we take advantage of cross-view consistency of 3D objects and merge 2D predictions from multiple views. To ease computation and reduce label noise, we downsample the extracted point clouds with class probabilities into voxels of size 0.1 × 0.1 × 0.1m 3 in a similar way as semantic map construction [START_REF] Chaplot | Object goal navigation using goal-oriented semantic exploration[END_REF]. Hence, 2D predictions from different views of an object can be merged together. We average class probabilities of all points inside each voxel and take the label with maximum probability to refine semantic prediction. The neighboring voxels of the same class are grouped together as a complete 3D object. In this way, we generate 3D objects for the whole building and also map 2D objects of different views into the unified 3D objects. This enables us to create reliable goal locations for target objects in the VLN task. Fig. 2 (right) shows example predictions of 3D semantic objects from a top-down view.

Data Generation: VLN Training Triplets

Based on the pseudo 3D object labels of the building, we generate objecttrajectory-instruction triplets as shown in Fig. 3. For each 3D object, we obtain the corresponding goal locations in the graph where the object is visible and located within d o meters. Then we randomly select a start node that is 4 to 9 steps away from the target location, and use the navigation graph to compute the shortest path as the expert trajectory. The final panoramic image in the trajectory is used to generate high-level VLN instructions. Generating instructions with object prompts. Previous works [START_REF] Fried | Speaker-follower models for vision-and-language navigation[END_REF][START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF] train speaker models from scratch on VLN datasets to generate instructions from an image trajectory. Due to the small amount of training examples in VLN datasets, such speaker models may suffer from inaccurate classification and are also unable to generate novel words beyond the training vocabulary. As a result, they even perform worse than template-based instructions on unseen images with diverse objects [START_REF] Guhur | Airbert: In-domain pretraining for vision-and-language navigation[END_REF]. We alleviate these problems by (i) using object labels obtained from the 3D object labeling as prompts for the speaker model; and (ii) fine-tuning a pretrained large language model (GPT-2 [START_REF] Radford | Language models are unsupervised multitask learners[END_REF]).

To be specific, our speaker model consists of a transformer encoder that produces prompt embeddings and a language decoder that generates sentences conditioning on the prompts. The encoder is fed with three types of tokens: target object token, other object tokens in the panorama, and view image tokens. The object token embedding encodes the object label, location, size, and visual representation, while the view image embedding is obtained from the visual representation and orientation embedding. A multi-layer transformer is adopted to learn relations of different tokens in order to generate more accurate referring expression for the target object. We average the output embeddings for each type of tokens separately, resulting in three tokens (target object, other objects, and view image) as prompts to the decoder. The decoder is initialized with pretrained GPT-2 [START_REF] Radford | Language models are unsupervised multitask learners[END_REF], a state-of-the-art language model. It takes the above prompts as the first three tokens and sequentially predicts words. We finetune the speaker model end-to-end on the REVERIE dataset [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF].

HM3D-AutoVLN: A Large-scale VLN Dataset

We employ the above procedure to process all unlabeled buildings in HM3D and generate a large-scale VLN dataset with high-level instructions, named HM3D-

VLN Model and Training

In this section, we briefly introduce a state-of-the-art VLN model DUET [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF] and then describe its training procedure.

DUET VLN Model

The main idea of the DUal scalE graph Transformer (DUET) [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF] model is to build a topological map during navigation, which allows the model to make efficient long-term navigation plans over all navigable nodes in the graph instead of being limited to the neighboring nodes. The model consists of two modules: topological mapping and global action planning. The topological mapping module gradually adds newly observed locations to the map and updates node representations. The global action planning module is a dual-scale graph transformer to predict a next location in the map or a stop action at each step. The dual-scale architecture enables the model to jointly use fine-grained language grounding with fine-scale representations of the current location and global graph reasoning with coarsescale representation of the map. The DUET model is the winning entry in ICCV 2021 REVERIE and SOON navigation challenge. We use the publicly released code4 in our experiments.

Training

Stage I: Pretraining on HM3D-AutoVLN. We use three proxy tasks to pretrain DUET [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF], including Masked Language Modeling (MLM), Single-step Action Prediction (SAP) and Object Grounding (OG). For MLM, the inputs are a masked instruction and a full expert trajectory (

V 1 , • • • , V T ).
The goal is to recover the masked words from the cross-modal visual representation. For SAP, we ask the model to predict the next action given the instruction and its navigation history. We randomly select the navigation history as:

(i) (V 1 , • • • , V T ) with target action STOP ; (ii) (V 1 , • • • , V t )
where 1 ≤ t < T with target action V t+1 ; and (iii) a random node V i in the graph, and choose a target node V i+1 with shortest overall distance from V i to V i+1 and from V i+1 to V T among all nodes in the constructed map. Finally, for OG, the model is fed with an instruction and expert trajectory

(V 1 , • • • , V T ) to predict the ground-truth object in V T .
Stage II: Fine-tuning on downstream VLN datasets. We fine-tune a pretrained DUET on each downstream task by using the pseudo interactive demonstrator algorithm [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF] -a special case of student forcing that addresses exposure bias. We suggest two strategies for fine-tuning: (i) fine-tuning on the downstream dataset only; or (ii) balancing the size of HM3D-AutoVLN and the downstream dataset, and combining the two datasets for joint training. The latter strategy is successful at preventing catastrophic forgetting of the previously learned policy as shown in our experiments in Table 2.

Experiments

Experiment Setup

Datasets. We evaluate models on two VLN tasks with high-level instructions, REVERIE [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF] and SOON [START_REF] Zhu | Soon: Scenario oriented object navigation with graph-based exploration[END_REF]. Each dataset is divided into training, val seen, val unseen and a hidden test unseen split. The statistics of training splits are presented in Table 1. The REVERIE dataset provides groundtruth object bounding boxes at each location and only requires an agent to select one of the bounding boxes. The shortest path from the agent's initial location to the target location is between 4 to 7 steps. The SOON dataset instead does not give groundtruth bounding boxes, and asks an agent to directly predict the orientation of the object's center. The path lengths are also longer than REVERIE with 9.5 steps on average. Due to the increased task difficulty and smaller training dataset size, it is more challenging to achieve high performance on SOON than on REVERIE.

Evaluation metrics. We measure two types of metrics for the VLN task, navigation-only and remote object grounding. The navigation-only metrics focus solely on whether an agent arrives at the target location. We use standard navigation metrics [START_REF] Anderson | Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments[END_REF] including Success Rate (SR), the percentage of paths with the final location near any target locations within 3 meters; Oracle Success Rate (OSR), the percentage of paths with any location close to target within 3 meters; and SR weighted by Path Length (SPL) which multiplies SR with the ratio between the length of shortest path and the agent's predicted path. As SPL takes into account navigation accuracy and efficiency, it is the primary metric in navigation. The remote object grounding metrics [START_REF] Qi | Reverie: Remote embodied visual referring expression in real indoor environments[END_REF] consider both navigation and object grounding performance. The standard metrics are Remote Grounding Success (RGS) and RGS weighted by Path Length (RGSPL). RGS in REVERIE is defined as correctly selecting the object among groundtruth bounding boxes, while in SOON as predicting a center point that is inside of the groundtruth polygon. RGSPL penalizes RGS by path length similar to SPL. For all these metrics, higher is better.

Implementation details. We adopt ViT-B/16 [START_REF] Dosovitskiy | An image is worth 16× 16 words: Transformers for image recognition at scale[END_REF] pretrained on ImageNet to extract view and object features. For SOON dataset, we use the Mask2Former trained on ADE20K [START_REF] Cheng | Masked-attention mask transformer for universal image segmentation[END_REF] to extract object bounding boxes and transfer the prediction in the same way as in REVERIE. We use the same hyper-parameters in modeling as the DUET model [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF]. All experiments were run on a single Nvidia RTX8000 GPU. The best epoch is selected based on SPL on the val unseen split.

Ablation Studies

The main objective of our work is to explore how much VLN agents can benefit from large-scale synthesized dataset. In this section, we carry out extensive ablations on the REVERIE dataset to study the effectiveness of our HM3D-AutoVLN dataset and key design choices.

Contributions from HM3D-AutoVLN dataset. In Table 2, we compare the impact of VLN data generated from different sources for training DUET. Row 1 (R1) only relies on manually annotated instructions on 60 buildings in MP3D. Row 2 (R2) utilizes groundtruth room and object annotations to generate instructions for all objects in the 60 seen buildings. Such data is of high-quality and significantly improves VLN performance, e.g. 3% on SPL. In R3, we only use our generated HM3D-AutoVLN for pretraining, while still finetuning on the REVERIE train split. Due to the visual diversity of the 900 additional buildings, it brings a larger boost than R2. Compared to R1, the improvement of the SPL metric is 8.6%, whereas other metrics like SR improve by 6.5%. This suggests that the pretraining on large-scale environments enables the model to learn efficient exploration, even without any additional manual annotations. Moreover, when the model is fine-tuned jointly on the HM3D-AutoVLN and REVERIE datasets (R4), the SR metric is further improved by 5% over R3. This indicates that fine-tuning on the downstream dataset alone may suffer from forgetting and lead to worse generalization performance. Compared to the navigation metrics, the remote object grounding metrics see modest improvements. We hypothesize that though our generated instructions often describe the object and scene accurately, they are not discriminative enough to refer to a specific object in the environment (e.g. unable to discriminate between multiple pillows placed on a sofa) or confuse predicting relationships between objects (e.g. second pillow from the left). We see improving RGS and RGSPL as a promising future direction that would need to take into account relations between objects.

Impact of the number of training environments. Here, we evaluate the impact of the number of training environments. As pretraining on HM3D-AutoVLN mostly improves navigation performance, we mainly show navigation metrics SR and SPL. The full result table is provided in the supplementary material. As shown in Figure 4a, more environments continuously improve the navigation performance. We observe that even with the full 900 environments in HM3D, the gains are not saturated but increase gradually. We also evaluate the impact of the number of environments given a fixed budget of VLN training examples. Figure 4b shows that using more environments improves the performance. On the left, the red bar uses 200 environments with 51,018 instructions, and the blue bar use the same amount of instructions but covers all 900 environments in HM3D. The right part is the same, but uses 400 environments for the red bar.

The results suggest that it is beneficial to increase the number of environments rather than just increasing the trajectories for a limited number of environments. Finetuning with a small amount of supervised data. A large-scale automatically generated dataset improves pretraining and hence can reduce the supervised data requirement in downstream tasks. We verify the effectiveness of HM3D-AutoVLN on a few-shot learning setting [START_REF] Guhur | Airbert: In-domain pretraining for vision-and-language navigation[END_REF] that compares the impact of finetuning on a variable number of REVERIE environments, see Table 3.

Even without finetuning on any supervised instructions from REVERIE, our pretrained model achieves fair performance in comparison to a baseline that is only finetuned on all supervised data. On the SPL metric, we outperform the baseline DUET model when finetuning on 10 environments (1/6 of the supervised data). Moreover, finetuning with half the original data (30 environments) achieves significant boosts on all metrics compared to the baseline model. A similar trend can be observed on the SOON dataset, see supplementary material.

Comparing distance to objects. As mentioned in Sec. 3.2, we select some of the visible objects that are close to the agent, within d o meters, to generate instructions. In Table 4, we compare the influence of different distances to the objects on the VLN performance. Larger d o allows us to generate more instructions. We can see that while including additional remote objects increases the number of instructions, it leads to a small drop in performance as the model struggles to identify objects that are small and far away. Evaluating quality of the HM3D-AutoVLN dataset. We validate each annotation procedure in our automatic dataset construction. For navigation graph 2D window, 3D mirror 2D door, 3D wardrobe 2D table, 3D desk 2D chair, 3D swivel chair Fig. 5: Qualitative examples of pseudo labelling generation, we measure whether the graph covers the whole building. Assuming each navigation node covers a circle with a radius of 2 meters, the graph achieves a high coverage rate of 93.4% on average. For 3D object labeling, we randomly select 300 bounding boxes and manually annotate semantic labels for them. We observe that 37.4% of them are correctly predicted with 2D predictions, whereas 58.3% of them were correctly predicted when applying cross-view consistency, showing an absolute improvement of 21%. Examples in Figure 5 highlight the advantages of using cross-view consistency. Due to the distorted view, it's reasonable that the 2D models wrongly predict the mirror and wardrobe to be a window and door respectively. Cross-view consistency also helps to integrate scene context, swapping a table to a desk and a normal chair to a swivel chair. For instruction generation, we further compare our GPT2based speaker model with template-based methods and a LSTM baseline [START_REF] Chen | Think global, act local: Dual-scale graph transformer for vision-and-language navigation[END_REF]. We manually evaluate 100 randomly selected instructions by measuring whether the instruction correctly mentions the target room, object class and object instance (with correct relations). We also measure the VLN performance using the generated instructions. Our model performs best on manual evaluation and on downstream VLN tasks as shown in Table 5.

Comparison with State of the Art

In Table 6, we compare with the state of the art on the REVERIE dataset. To demonstrate the contributions of our automatically constructed dataset, we further train additional VLN agents with the augmentation of the HM3D-AutoVLN dataset, including EnvDrop [START_REF] Tan | Learning to navigate unseen environments: Back translation with environmental dropout[END_REF], RecBert [START_REF] Hong | Vln BERT: A recurrent vision-and-language BERT for navigation[END_REF] and HAMT [8]. Our proposed dataset improves results of all methods and gives a particularly large boost to high-capacity models. When pretraining DUET on HM3D-AutoVLN, the increase in performance over DUET with pretraining is significant for all metrics on both the val unseen and the test unseen splits. For example, the SPL measure increases by 7.1% and 2.8% on val unseen and test unseen splits of REVERIE.

Table 7 provides results for the SOON dataset. Note that instructions from the SOON dataset are not used to train our speaker model (c.f. Sec. 3.2) and are somewhat different from the REVERIE instructions. Nevertheless, the large performance increase demonstrates cross-domain benefits of pretraining on an automatically collected large-scale dataset. Finally, we can also observe that the gain for object grounding is less significant, as discussed before this can be explained by the confusion between objects of the same categories and erroneous spatial relations.

Conclusion

This work addresses the lack of training data for VLN tasks. We propose to automatically generate pseudo 3D object labels and VLN instructions for a collection of large-scale unlabeled 3D environments. Training on our new dataset HM3D-AutoVLN significantly improves VLN performance due to the large-scale pretraining or co-training. It also provides insights on the importance of dataset collection and the challenges inherent to leveraging unlabeled environments. In particular, it shows that the diversity of navigation environments is more important than the number of training samples alone.

We first present additional automatically generated examples from our HM3D-AutoVLN dataset in Sec. A. Then, we present further ablation analysis of our methods and also discuss results on the SOON dataset (equivalent analysis for the REVERIE dataset was in the main paper) in Sec. B. Finally, some qualitative visualizations of predicted navigation results are presented in Sec. C.

A HM3D-AutoVLN Dataset

In Fig. 6, we provide more examples of generated navigation graphs and objecttrajectory-instruction triplets in the HM3D-AutoVLN dataset. Each trajectory consists of a sequence of panoramas. We only visualize the oriented view image at each step in Fig. 6 for simplicity.

B Additional Ablations

B.1 Comparison of Different Generated Instructions

We compare our generated instructions from the proposed speaker model (Sec. 3.2 of the main paper) with two types of template-based instructions. The first type of template instructions (Template.Obj) only describes the target object name, e.g., "open the window", "clean the mirror". The second type (Template.Sent) uses sentence structures in REVERIE training split as templates [START_REF] Guhur | Airbert: In-domain pretraining for vision-and-language navigation[END_REF] and replaces the noun phrases in the template with detected objects in the scene, e.g., "go to bedroom and clean the chandelier". As these template instructions are less accurate and contain less details, we generate three instructions per target object to alleviate the impact of noisy instructions. As shown in Table 8, even with fewer number of instructions per object, the generated instructions from our speaker model outperform the two template-based instructions. 10 present the results for all evaluation metrics corresponding to the SR/SPL results in Fig. 4 of the main paper. We can see that more training environments are beneficial for the performance on unseen environments in Table 9 and that this result is reflected across all metrics. When using a fixed number of instructions, it is better to collect training examples from more environments as shown in Table 10.

B.3 Few-shot Results

In Table 11, we show the performance of using different amounts of training data from the SOON dataset to fine-tune the DUET model pretrained on our HM3D-AutoVLN dataset. This is comparable to Table 3 of the main paper that presented results on the REVERIE dataset. To be noted, instructions in SOON dataset are not used to train our speaker model, and the style of instructions in HM3D-AutoVLN is quite different from the style in the SOON dataset. Despite the style differences, the pretraining allows our model to achieve better performance with a small amount of supervised data.

C Qualitative Results

Fig. 7 compares some predicted trajectories from DUET models with and without pretraining on our HM3D-AutoVLN dataset. We can see that the pretraining allows the navigation model to explore the unseen environment more efficiently.

Fig. 3 :

 3 Fig. 3: To generate VLN training triplets, we first select a target object using the pseudo 3D object labels and then sample a trajectory using the navigation graph. Finally, we use a trained speaker model to generate high-level VLN instructions.

  (a) Navigation performance with respect to the number of training environments in HM3D-AutoVLN dataset. (b) Given the same number of training examples, collecting data from more environments (blue) performs better than fewer environments.

Fig. 4 :

 4 Fig. 4: Influence of the number of environments on DUET performance.

Fig. 6 :

 6 Fig. 6: Examples of generated navigation graphs and object-trajectoryinstruction triplets in HM3D-AutoVLN dataset.

Fig. 8 Fig. 7 :Fig. 8 :

 878 Fig.8further presents some failure cases. The main problem is to correctly decide the stop location aligned to the fine-grained instruction. For example, in the last two examples in Fig.8, the agent has navigated close to the target location but failed to stop and continued the exploration.

Table 1 :

 1 Statistics of training data on different VLN datasets. (gt.obj denotes whether the data relies on groundtruth object annotations) AutoVLN. Table1compares HM3D-AutoVLN with existing VLN datasets that also use high-level instructions. REVERIE and SOON are manually annotated datasets and REVERIE-Speaker is an augmented dataset generated by a speaker model trained with groundtruth room and object annotations. Our HM3D-AutoVLN is an order of magnitude larger than these datasets, in terms of environments, objects, and instructions. Due to the pretrained language model, our dataset also features a diverse vocabulary 3 in the generated instructions.

	dataset	gt.obj #env. #objs #instr #vocab inst. length
	REVERIE [43]	✓	60	12,029 10,466 1,140	18.64
	REVERIE-Speaker [9]	✓	64	12,062 19,636	399	15.20
	SOON [57]	✓	34	4,358	2,780	735	44.09
	HM3D-AutoVLN	×	900 172,000 217,703 1,696	20.52

Table 2 :

 2 DUET performance on REVERIE (RVR) val unseen split using different training data. † denotes manual object annotations are used to synthesize data

		Training Data		Navigation		Grounding
		Pretrain	Finetune	OSR	SR	SPL	RGS RGSPL
	R1	RVR	RVR	48.74 44.36 30.79 30.30	21.08
	R2 RVR+Speaker †	RVR	51.07 46.98 33.73 32.15	23.03
	R3	HM3D	RVR	54.81 50.87 39.36 34.65 26.79
	R4	HM3D	RVR+HM3D 62.14 55.89 40.85 36.58 26.76

Table 3 :

 3 DUET performance when using a fraction of the supervised data

	HM3D Pretrain	#environments #instructions	Navigation OSR SR SPL RGS RGSPL Grounding
	×	60	10,466	48.74 44.36 30.79 30.30 21.08
	✓	0	0	43.08 36.81 25.28 20.82 13.74
	✓	1	449	50.78 42.12 29.55 25.02 17.26
	✓	10	1,404	50.47 43.79 33.61 26.30 20.24
	✓	30	5,244	60.81 53.71 39.26 34.42 25.11
	✓	60	10,466	62.14 55.89 40.85 36.58 26.76

Table 4 :

 4 DUET performance on instructions where the visible objects are at a different distance d o from the agent location

	do #instrs	Navigation OSR SR SPL RGS RGSPL Grounding
	2 217,703 62.14 55.89 40.85 36.58 26.76
	3 396,401 57.37 53.25 40.38 34.28 25.65
	∞ 544,606 59.98 53.37 38.03 35.70 25.50

Table 5

 5

	: Comparison of different
	speaker models in terms of manual
	captioning evaluation and the followup
	navigation performance
		Captioning	Navigation
		Room Obj Rel SR RGS
	Template	0.13 0.76 0.05 52.20 32.75
	LSTM	0.58 0.65 0.27 49.59 32.29
	GPT2 (Ours) 0.73 0.78 0.35 55.89 36.58

Table 6 :

 6 Comparison with the state-of-the-art methods on REVERIE dataset AutoVLN 62.14 55.89 40.85 36.58 26.76 62.3 55.17 38.88 32.23 22.68

				Val Unseen				Test Unseen
	Methods	Navigation	Grounding	Navigation	Grounding
		OSR	SR	SPL RGS RGSPL OSR	SR	SPL RGS RGSPL
	Human	-	-	-	-	-	86.83 81.51 53.66 77.84 51.44
	Seq2Seq [3]	8.07	4.20	2.84	2.16	1.63	6.88 3.99	3.09	2.00	1.58
	RCM [53]	14.23 9.29	6.97	4.89	3.89	11.68 7.84	6.67	3.67	3.14
	SMNA [34]	11.28 8.15	6.44	4.54	3.61	8.39 5.80	4.53	3.10	2.39
	FAST-MAttNet [43]	28.20 14.40 7.19	7.84	4.67	30.63 19.88 11.61 11.28	6.08
	SIA [32]	44.67 31.53 16.28 22.41 11.56 44.56 30.80 14.85 19.02	9.20
	Airbert [18]	34.51 27.89 21.88 18.23 14.18 34.20 30.28 23.61 16.83 13.28
	EnvDrop [49]	26.3	23.0	19.9	-	-	-	-	-	-	-
	+HM3D-AutoVLN	29.3	25.2	20.6	-	-	-	-	-	-	-
	RecBERT (oscar) [22] 27.66 25.53 21.06 14.20 12.00 26.67 24.62 19.48 12.65 10.00
	+HM3D-AutoVLN 33.23 29.20 23.12 18.12 14.18	-	-	-	-	-
	HAMT [8]	36.84 32.95 30.20 18.92 17.28 33.41 30.40 26.67 14.88 13.08
	+HM3D-AutoVLN 42.09 37.80 31.35 23.03 18.88	-	-	-	-	-
	DUET [9]	51.07 46.98 33.73 32.15 23.03 56.91 52.51 36.06 31.88 22.06
	+HM3D-

Table 7 :

 7 Comparison with the state-of-the-art methods on the SOON dataset

	Methods	OSR	Val Unseen SR SPL RGSPL OSR	Test Unseen SR SPL RGSPL
	GBE [57]	28.54 19.52 13.34	1.16	21.45 12.90 9.23	0.45
	DUET [9]	50.91 36.28 22.58	3.75	43.00 33.44 21.42	4.17
	DUET (+HM3D) 53.19 41.00 30.69	4.06	48.74 40.36 27.83	5.11

Table 8 :

 8 DUET performance on the REVERIE val unseen split using different methods for generating instructions.

	Method	#Instrs inst.len	OSR	Navigation SR	SPL	Grounding RGS RGSPL
	Template.Obj 653,109	3.88	54.33 49.28 37.66 29.85	23.07
	Template.Sent 653,109 11.50	56.16 52.20 39.90 32.75	24.86
	GPT2 (Ours) 217,703 20.52 62.14 55.89 40.85 36.58 26.76
	B.2 Influence of the Number of Training Environments

Table 9 and

 9 Table

Table 9 :

 9 DUET performance on the REVERIE val unseen split using different number of training environments from the HM3D-AutoVLN dataset.

	#Envs	OSR	Navigation SR	SPL	Grounding RGS RGSPL
	200	55.61 50.13 37.65 32.92	24.68
	400	55.04 51.09 36.79 32.43	23.16
	600	60.61 53.99 40.40 33.85	25.52
	900	62.14 55.89 40.85 36.58 26.76

Table 10 :

 10 DUET performance on the REVERIE val unseen split using the same number of instructions but different number of training environments in HM3D-AutoVLN dataset (51,018 is the total number of instructions contained in the 200 training environments, and so is 99,224 in 400 training environments)

	#Instrs #Envs	Navigation OSR SR SPL RGS RGSPL Grounding
	51,018	200 900	55.61 50.13 37.65 32.92 24.68 54.05 50.75 39.52 32.35 25.22
	99,224	400 900	55.04 51.09 36.79 32.43 23.16 60.44 53.96 38.35 36.01 25.56

Table 11 :

 11 DUET performance on SOON val unseen split using different number of SOON training environments in fine-tuning

	Pretrain #Envs #Instrs	Navigation OSR SR SPL	Grounding RGSPL
	×	34	2,780 50.91 36.28 22.58	3.75
	✓	0	0	14.75 11.21 7.92	0.38
	✓	1	67	37.32 25.22 17.97	2.17
	✓	10	848	45.58 34.51 24.38	2.75
	✓	34	2,780 53.19 41.00 30.69	4.06

In Table1vocab, we only count words that occur more than 5 times in the dataset.

https://github.com/cshizhe/VLN-DUET

Acknowledgements. This work was granted access to the HPC resources of IDRIS under the allocation 101002 made by GENCI. This work is funded in part by the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR19-P3IA-0001 (PRAIRIE 3IA Institute) and by Louis Vuitton ENS Chair on Artificial Intelligence.