Florian Strub

Mathieu Seurin

Ethan Perez

Harm De Vries

Jérémie Mary

Philippe Preux

Aaron Courville

Olivier Pietquin

Visual Reasoning with Multi-hop Feature Modulation

Keywords: Deep Learning, Computer Vision, Natural Language Understanding, Multi-modal Learning

Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue. For such tasks, one successful approach is to condition image-based convolutional network computation on language via Feature-wise Linear Modulation (FiLM) layers, i.e., per-channel scaling and shifting. We propose to generate the parameters of FiLM layers going up the hierarchy of a convolutional network in a multi-hop fashion rather than all at once, as in prior work. By alternating between attending to the language input and generating FiLM layer parameters, this approach is better able to scale to settings with longer input sequences such as dialogue. We demonstrate that multi-hop FiLM generation achieves state-of-the-art for the short input sequence task ReferIt-on-par with single-hop FiLM generationwhile also significantly outperforming prior state-of-the-art and singlehop FiLM generation on the GuessWhat?! visual dialogue task.

Introduction

Computer vision has witnessed many impressive breakthroughs over the past decades in image classification [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF][START_REF] He | Deep residual learning for image recognition[END_REF], image segmentation [START_REF] Long | Fully convolutional networks for semantic segmentation[END_REF], and object detection [START_REF] Girshick | Rich feature hierarchies for accurate object detection and semantic segmentation[END_REF] by applying convolutional neural networks to large-scale, labeled datasets, often exceeding human performance. These systems give outputs such as class labels, segmentation masks, or bounding boxes, but it would be more natural for humans to interact with these systems through natural language. To this end, the research community has introduced various multi-modal tasks, such as image captioning [START_REF] Xu | Show, attend and tell: Neural image caption generation with visual attention[END_REF], referring expressions [START_REF] Kazemzadeh | Referitgame: Referring to objects in photographs of natural scenes[END_REF], visual question-answering [START_REF] Antol | Vqa: Visual question answering[END_REF][START_REF] Malinowski | Ask your neurons: A neural-based approach to answering questions about images[END_REF], visual reasoning [START_REF] Johnson | Clevr: A diagnostic dataset for compositional language and elementary visual reasoning[END_REF], and visual dialogue [START_REF] De Vries | Guesswhat?! visual object discovery through multi-modal dialogue[END_REF][START_REF] Das | Visual dialog[END_REF].

These tasks require models to effectively integrate information from both vision and language. One common approach is to process both modalities independently with large unimodal networks before combining them through concatenation [START_REF] Malinowski | Ask your neurons: A neural-based approach to answering questions about images[END_REF], element-wise product [START_REF] Kim | Multimodal residual learning for visual qa[END_REF][START_REF] Lu | Hierarchical question-image co-attention for visual question answering[END_REF], or bilinear pooling [START_REF] Fukui | Multimodal compact bilinear pooling for visual question answering and visual grounding[END_REF]. Inspired by the success of attention in machine translation [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF], several works have proposed

ReferIt

GuessWhat?! -The girl with a sweater Is it a person? Yes -The fourth person Is it a girl? Yes -The girl holding a white Does she have a blue No frisbee frisbee?

Fig. 1: The ReferIt task identifies a selected object (in the bounding box) using a single expression, while in GuessWhat?!, a speaker localizes the object with a series of yes or no questions.

to incorporate various forms of spatial attention to bias models towards focusing on question-specific image regions [START_REF] Xu | Show, attend and tell: Neural image caption generation with visual attention[END_REF][START_REF] Xu | Ask, attend and answer: Exploring question-guided spatial attention for visual question answering[END_REF]. However, spatial attention sometimes only gives modest improvements over simple baselines for visual question answering [START_REF] Jabri | Revisiting visual question answering baselines[END_REF] and can struggle on questions involving multi-step reasoning [START_REF] Johnson | Clevr: A diagnostic dataset for compositional language and elementary visual reasoning[END_REF].

More recently, [START_REF] De Vries | Modulating early visual processing by language[END_REF][START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF] introduced Feature-wise Linear Modulation (FiLM) layers as a promising approach for vision-and-language tasks. These layers apply a per-channel scaling and shifting to a convolutional network's visual features, conditioned on an external input such as language, e.g., captions, questions, or full dialogues. Such feature-wise affine transformations allow models to dynamically highlight the key visual features for the task at hand. The parameters of FiLM layers which scale and shift features or feature maps are determined by a separate network, the so-called FiLM generator, which predicts these parameters using the external conditioning input. Within various architectures, FiLM has outperformed prior state-of-art for visual question-answering [START_REF] De Vries | Modulating early visual processing by language[END_REF][START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF], multi-modal translation [START_REF] Delbrouck | Modulating and attending the source image during encoding improves multimodal translation[END_REF], and language-guided image segmentation [START_REF] Rupprecht | Guide me: Interacting with deep networks[END_REF].

However, the best way to design the FiLM generator is still an open question. For visual question-answering and visual reasoning, prior work uses single-hop FiLM generators that predict all FiLM parameters at once [START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF][START_REF] De Vries | Modulating early visual processing by language[END_REF]. That is, a Recurrent Neural Network (RNN) sequentially processes input language tokens and then outputs all FiLM parameters via a Multi-Layer Perceptron (MLP). In this paper, we argue that using a Multi-hop FiLM Generator is better suited for tasks involving longer input sequences and multi-step reasoning such as dialogue. Even for shorter input sequence tasks, single-hop FiLM generators can require a large RNN to achieve strong performance; on the CLEVR visual reasoning task [START_REF] Johnson | Clevr: A diagnostic dataset for compositional language and elementary visual reasoning[END_REF] which only involves a small vocabulary and templated questions, the FiLM generator in [START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF] uses an RNN with 4096 hidden units that comprises almost 90% of the model's parameters. Models with Multi-hop FiLM Generators may thus be easier to scale to more difficult tasks involving human-generated language involving larger vocabularies and more ambiguity.

As an intuitive example, consider the dialogue in Fig. 1 through which one speaker localizes the second girl in the image, the one who does not "have a blue frisbee." For this task, a single-hop model must determine upfront what steps of reasoning to carry out over the image and in what order; thus, it might decide in a single shot to highlight feature maps throughout the visual network detecting either non-blue colors or girls. In contrast, a multi-hop model may first determine the most immediate step of reasoning necessary (i.e., locate the girls), highlight the relevant visual features, and then determine the next immediate step of reasoning necessary (i.e., locate the blue frisbee), and so on. While it may be appropriate to reason in either way, the latter approach may scale better to longer language inputs and/or or to ambiguous images where the full sequence of reasoning steps is hard to determine upfront, which can even be further enhanced by having intermediate feedback while processing the image.

In this paper, we therefore explore several approaches to generating FiLM parameters in multiple hops. These approaches introduce an intermediate context embedding that controls the language and visual processing, and they alternate between updating the context embedding via an attention mechanism over the language sequence (and optionally by incorporating image activations) and predicting the FiLM parameters. We evaluate Multi-hop FiLM generation on ReferIt [START_REF] Kazemzadeh | Referitgame: Referring to objects in photographs of natural scenes[END_REF] and GuessWhat?! [START_REF] De Vries | Guesswhat?! visual object discovery through multi-modal dialogue[END_REF], two vision-and-language tasks illustrated in Fig. 1. We show that Multi-hop FiLM models significantly outperform their single-hop counterparts and prior state-of-the-art for the longer input sequence, dialogue-based GuessWhat?! task while matching the state-of-the-art performance of other models on ReferIt. Our best GuessWhat?! model only updates the context embedding using the language input, while for ReferIt, incorporating visual feedback to update the context embedding improves performance. In summary, this paper makes the following contributions:

-We introduce the Multi-hop FiLM architecture and demonstrate that our approach matches or significantly improves state-of-the-art on the Guess-What?! Oracle task, GuessWhat?! Guesser task, and ReferIt Guesser task. -We show Multi-hop FiLM models outperforms their single-hop counterparts on vision-and-language tasks involving complex visual reasoning. -We find that updating the context embedding of Multi-hop FiLM Generator based on visual feedback may be helpful in some cases, such as for tasks which do not include object category labels like ReferIt.

Background

In this section, we explain the prerequisites to understanding our model: RNNs, attention mechanisms, and FiLM. We subsequently use these building blocks to propose a Multi-hop FiLM model.

Recurrent Neural Networks

One common approach in natural language processing is to use a Recurrent Neural Network (RNN) to encode some linguistic input sequence l into a fixed-size embedding. The input (such as a question or dialogue) consists of a sequence of words ω 1:T of length T , where each word ω t is contained within a predefined vocabulary V. We embed each input token via a learned look-up table e and obtain a dense word-embedding e ωt = e(ω t ). The sequence of embeddings {e ωt } T t=1 is then fed to a RNN, which produces a sequence of hidden states {s t } T t=1 by repeatedly applying a transition function f : s t+1 = f (s t , e ωt ) To better handle long-term dependencies in the input sequence, we use a Gated Recurrent Unit (GRU) [START_REF] Chung | Empirical evaluation of gated recurrent neural networks on sequence modeling[END_REF] with layer normalization [START_REF] Ba | Layer normalization[END_REF] as transition function. In this work, we use a bidirectional GRU, which consists of one forward GRU, producing hidden states -→ s t by running from ω 1 to ω T , and a second backward GRU, producing states ←s t by running from ω T to ω 1 . We concatenate both unidirectional GRU states s t = [ -→ s t ; ←s t ] at each step t to get a final GRU state, which we then use as the compressed embedding e l of the linguistic sequence l.

Attention

The form of attention we consider was first introduced in the context of machine translation [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF][START_REF] Luong | Effective approaches to attention-based neural machine translation[END_REF]. This mechanism takes a weighted average of the hidden states of an encoding RNN based on their relevance to a decoding RNN at various decoding time steps. Subsequent spatial attention mechanisms have extended the original mechanism to image captioning [START_REF] Xu | Show, attend and tell: Neural image caption generation with visual attention[END_REF] and other vision-and-language tasks [START_REF] Xu | Ask, attend and answer: Exploring question-guided spatial attention for visual question answering[END_REF][START_REF] Kim | Hadamard Product for Low-rank Bilinear Pooling[END_REF]. More formally, given an arbitrary linguistic embedding e l and image activations F w,h,c where w, h, c are the width, height, and channel indices, respectively, of the image features F at one layer, we obtain a final visual embedding e v as follows:

ξ w,h = M LP (g(F w,h,• , e l )) ; α w,h = exp(ξ w,h ) w ,h exp(ξ w ,h ) ; ev = w,h α w,h F w,h,• , (1)
where M LP is a multi-layer perceptron and g(., .) is an arbitrary fusion mechanism (concatenation, element-wise product, etc.). We will use Multi-modal Lowrank Bilinear (MLB) attention [START_REF] Kim | Hadamard Product for Low-rank Bilinear Pooling[END_REF] which defines g(., .) as:

g(F w,h,• , e l ) = tanh(U T F w,h,• ) • tanh(V T e l ), (2)
where • denotes an element-wise product and where U and V are trainable weight matrices. We choose MLB attention because it is parameter efficient and has shown strong empirical performance [START_REF] Kim | Hadamard Product for Low-rank Bilinear Pooling[END_REF][START_REF] Kafle | Visual question answering: Datasets, algorithms, and future challenges[END_REF].

Feature-wise Linear Modulation

Feature-wise Linear Modulation was introduced in the context of image stylization [START_REF] Dumoulin | A Learned Representation For Artistic Style[END_REF] and extended and shown to be highly effective for multi-modal tasks such as visual question-answering [START_REF] De Vries | Modulating early visual processing by language[END_REF][START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF][START_REF] Delbrouck | Modulating and attending the source image during encoding improves multimodal translation[END_REF].

A Feature-wise Linear Modulation (FiLM) layer applies a per-channel scaling and shifting to the convolutional feature maps. Such layers are parameter efficient (only two scalars per feature map) while still retaining high capacity, as they are able to scale up or down, zero-out, or negate whole feature maps. In vision-andlanguage tasks, another network, the so-called FiLM generator h, predicts these Fig. 2: The Multi-hop FiLM architecture, illustrating inputs (green), layers (blue), and activations (purple). In contrast, Single-hop FiLM models predict FiLM parameters directly from e l,T . modulating parameters from the linguistic input e l . More formally, a FiLM layer computes a modulated feature map Fw,h,c as follows:

[ γ ; β ] = h(e l ) ; F.,.,c = γ c F .,.,c + β c , (3)
where γ and β are the scaling and shifting parameters which modulate the activations of the original feature map F .,.,c . We will use the superscript k ∈ [1; K] to refer to the k th FiLM layer in the network. FiLM layers may be inserted throughout the hierarchy of a convolutional network, either pre-trained and fixed [START_REF] De Vries | Guesswhat?! visual object discovery through multi-modal dialogue[END_REF] or trained from scratch [START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF]. Prior FiLMbased models [START_REF] De Vries | Modulating early visual processing by language[END_REF][START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF][START_REF] Delbrouck | Modulating and attending the source image during encoding improves multimodal translation[END_REF] have used a single-hop FiLM generator to predict the FiLM parameters in all layers, e.g., an MLP which takes the language embedding e l as input [START_REF] De Vries | Modulating early visual processing by language[END_REF][START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF][START_REF] Delbrouck | Modulating and attending the source image during encoding improves multimodal translation[END_REF].

Multi-hop FiLM

In this section, we introduce the Multi-hop FiLM architecture (shown in Fig. 2) to predict the parameters of FiLM layers in an iterative fashion, to better scale to longer input sequences such as in dialogue. Another motivation was to better disantangle the linguistic reasoning from the visual one by iteratively attending to both pipelines.

We introduce a context vector c k that acts as a controller for the linguistic and visual pipelines. We initialize the context vector with the final state of a bidirectional RNN s T and repeat the following procedure for each of the FiLM layers in sequence (from lowest to highest convolutional layer): first, the context vector is updated by performing attention over RNN states (extracting relevant language information), and second, the context is used to predict a layer's FiLM parameters (dynamically modulating the visual information). Thus, the context vector enables the model to perform multi-hop reasoning over the linguistic pipeline while iteratively modulating the image features. More formally, the context vector is computed as follows:

c 0 = s T c k = t κ k t (c k-1 , s t )s t , (4)
where:

κ k t (c k-1 , s t ) = exp(χ k t ) t exp(χ k t ) ; χ k t (c k-1 , s t ) = M LP Attn (g (c k , s t )), (5)
where the dependence of χ k t on (c k-1 , s t ) may be omitted to simplify notation. M LP Attn is a network (shared across layers) which aids in producing attention weights. g can be any fusion mechanism that facilitates selecting the relevant context to attend to; here we use a simple dot-product following [START_REF] Luong | Effective approaches to attention-based neural machine translation[END_REF], so

g (c k , s t ) = c k • s t . Finally, FiLM is carried out using a layer-dependent neural network M LP k F iLM : [ γ k ; β k ] = M LP k F iLM (c k ) ; F k w,h,c = γ k c F k .,.,c + β k c . (6)
As a regularization, we append a normalization-layer [START_REF] Ba | Layer normalization[END_REF] on top of the context vector after each attention step. External information. Some tasks provide additional information which may be used to further improve the visual modulation. For instance, GuessWhat?! provides spatial features of the ground truth object to models which must answer questions about that object. Our model incorporates such features by concatenating them to the context vector before generating FiLM parameters.

Visual feedback. Inspired by the co-attention mechanism [START_REF] Lu | Hierarchical question-image co-attention for visual question answering[END_REF][START_REF] Zhuang | Parallel attention: A unified framework for visual object discovery through dialogs and queries[END_REF], we also explore incorporating visual feedback into the Multi-hop FiLM architecture. To do so, we first extract the image or crop features F k (immediately before modulation) and apply a global mean-pooling over spatial dimensions. We then concatenate this visual state into the context vector c k before generating the next set of FiLM parameters.

Experiments

In this section, we first introduce the ReferIt and GuessWhat?! datasets and respective tasks and then describe our overall Multi-hop FiLM architecture 1 . Fig. 3: Overall model, consisting of a visual pipeline (red and yellow) and linguistic pipeline (blue) and incorporating additional contextual information (green).

Dataset

ReferIt [START_REF] Kazemzadeh | Referitgame: Referring to objects in photographs of natural scenes[END_REF][START_REF] Yu | Modeling context in referring expressions[END_REF] is a cooperative two-player game. The first player (the Oracle) selects an object in a rich visual scene, for which they must generate an expression that refers to it (e.g., "the person eating ice cream"). Based on this expression, the second player (the Guesser) must then select an object within the image. There are four ReferIt datasets exist: RefClef, RefCOCO, RefCOCO+ and Re-fCOCOg. The first dataset contains 130K references over 20K images from the ImageClef dataset [START_REF] Mller | ImageCLEF: Experimental Evaluation in Visual Information Retrieval[END_REF], while the three other datasets respectively contain 142K, 142K and 86K references over 20K, 20k and 27K images from the MSCOCO dataset [START_REF] Lin | Microsoft coco: Common objects in context[END_REF]. Each dataset has small differences. RefCOCO and RefClef were constructed using different image sets. RefCOCO+ forbids certain words to prevent object references from being too simplistic, and RefCOCOg only relies on images containing 2-4 objects from the same category. RefCOCOg also contains longer and more complex sentences than RefCOCO (8.4 vs. 3.5 average words). Here, we will show results on both the Guesser and Oracle tasks.

GuessWhat?! [6] is a cooperative three-agent game in which players see the picture of a rich visual scene with several objects. One player (the Oracle) is randomly assigned an object in the scene. The second player (Questioner) aims to ask a series of yes-no questions to the Oracle to collect enough evidence to allow the third player (Guesser) to correctly locate the object in the image. The GuessWhat?! dataset is composed of 131K successful natural language dialogues containing 650k question-answer pairs on over 63K images from MSCOCO [START_REF] Lin | Microsoft coco: Common objects in context[END_REF]. Dialogues contain 5.2 question-answer pairs and 34.4 words on average. Here, we will focus on the Guesser and Oracle tasks.

Task Descriptions

Game Features. Both games consist of triplets (I, l, o), where I ∈ R 3×M ×N is an RGB image and l is some language input (i.e., a series of words) describing an object o in I. The object o is defined by an object category, a pixel-wise segmentation, an RGB crop of I based on bounding box information, and handcrafted spatial information x spatial , where

x spatial = [x min , y min , x max , y max , x center , y center , w box , h box ] (7)
We replace words with two or fewer occurrences with an <unk> token. The Oracle task. Given an image I, an object o, a question q, and a sequence δ of previous question-answer pairs (q, a) 1:δ where a ∈ {Yes, No, N/A}, the oracle's task is to produce an answer a that correctly answers the question q.

The Guesser task. Given an image I, a list of objects O = o 1:Φ , a target object o * ∈ O and the dialogue D, the guesser needs to output a probability σ φ that each object o φ is the target object o * . Following [START_REF] Hu | Natural language object retrieval[END_REF], the Guesser is evaluated by selecting the object with the highest probability of being correct. Note that even if the individual probabilities σ φ are between 0 and 1, their sum can be greater than 1. More formally, the Guesser loss and error are computed as follows:

L Guesser = -1 N games Ngames n 1 Φ n Φ φ log(p(o * |I n , o n φ , D n )) (8)
E Guesser = -1 N games Ngames n 1(o * = o argmax φ σ n φ ) ( 9
)
where 1 is the indicator function and Φ n the number of objects in the n th game.

Model

We use similar models for both ReferIt and GuessWhat?! and provide its architectural details in this subsection.

Object embedding The object category is fed into a dense look-up table e cat , and the spatial information is scaled to [-1;1] before being up-sampled via nonlinear projection to e spat . We do not use the object category in ReferIt models.

Visual Pipeline We first resized the image and object crop to 448×448 before extracting 14 × 14 × 1024 dimensional features from a ResNet-152 [START_REF] He | Deep residual learning for image recognition[END_REF] (block3) pre-trained on ImageNet [START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF]. Following [START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF], we feed these features to a 3 × 3 convolution layer with Batch Normalization [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF] and Rectified Linear Unit [START_REF] Nair | Rectified linear units improve restricted boltzmann machines[END_REF] (ReLU). We then stack four modulated residual blocks (shown in Fig 2), each producing a set of feature maps F k via (in order) a 1 × 1 convolutional layer (128 units), ReLU activations, a 3 × 3 convolutional layer (128 units), and an untrainable Batch Normalization layer. The residual block then modulates F k with a FiLM layer to get F k , before again applying ReLU activations. Lastly, a residual connection sums the activations of both ReLU outputs. After the last residual block, we use a 1 × 1 convolution layer (512 units) with Batch Normalization and ReLU followed by MLB attention [START_REF] Kim | Hadamard Product for Low-rank Bilinear Pooling[END_REF] (256 units and 1 glimpse) to obtain the final embedding e v . Note our model uses two independent visual pipeline modules: one to extract modulated image features e img v , one to extract modulated crop features e crop v .

To incorporate spatial information, we concatenate two coordinate feature maps indicating relative x and y spatial position (scaled to [-1, 1]) with the image features before each convolution layer (except for convolutional layers followed by FiLM layers). In addition, the pixel-wise segmentations S ∈ {0, 1} M ×N are rescaled to 14 × 14 floating point masks before being concatenated to the feature maps.

Linguistic Pipeline We compute the language embedding by using a wordembedding look-up (200 dimensions) with dropout followed by a Bi-GRU (512 × 2units) with Layer Normalization [START_REF] Ba | Layer normalization[END_REF]. As described in Section 3, we initialize the context vector with the last RNN state c 0 = s T . We then attend to the other Bi-GRU states via an attention mechanism with a linear projection and ReLU activations and regularize the new context vector with Layer Normalization.

FiLM parameter generation We concatenate spatial information e spat and object category information e cat to the context vector. In some experiments, we also concatenate a fourth embedding consisting of intermediate visual features F k after mean-pooling. Finally, we use a linear projection to map the embedding to FiLM parameters.

Final Layers We first generate our final embedding by concatenating the output of the visual pipelines e f inal = [e img v ; e crop v ] before applying a linear projection (512 units) with ReLU and a softmax layer.

Training Process We train our model end-to-end with Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] (learning rate 3e -4 ), dropout (0.5), weight decay (5e -6 ) for convolutional network layers, and a batch size of 64. We report results after early stopping on the validation set with a maximum of 15 epochs.

Baselines

In our experiments, we re-implement several baseline models to benchmark the performance of our models. The standard Baseline NN simply concatenates the image and object crop features after mean pooling, the linguistic embedding, and the spatial embedding and the category embedding (GuessWhat?! only), passing those features to the same final layers described in our proposed model. We refer to a model which uses the MLB attention mechanism to pool the visual features as Baseline NN+MLB. We also implement a Single-hop FiLM mechanism which is equivalent to setting all context vectors equal to the last state of the Bi-GRU e l,T . Finally, we experiment with injecting intermediate visual features into the FiLM Generator input, and we refer to the model as Multi-hop FiLM (+img).

Results

ReferIt Guesser We report the best test error of the outlined methods on the ReferIt Guesser task in Tab. 1. Note that RefCOCO and RefCOCO+ split test sets into TestA and TestB, only including expression referring towards people and objects, respectively. We do not report [START_REF] Yu | Mattnet: Modular attention network for referring expression comprehension[END_REF] and [START_REF] Yu | A joint speakerlistener-reinforcer model for referring expressions[END_REF] scores on RefCOCOg as the authors use a different split (umd). Our initial baseline achieves 77.6%, 60.8%, 63.1%, 73.4% on the RefCOCO, RefCOCO+, RefCOCOg, RefClef datasets, respectively, performing comparably to state-of-the-art models. We observe a significant improvements using a FiLM-based architecture, jumping to 84.9%, 87.4%, 73.8%, 71.5%, respectively, and outperforming most prior methods and achieving comparably performance with the concurrent MAttN [START_REF] Yu | Mattnet: Modular attention network for referring expression comprehension[END_REF] model. Interestingly, MAttN and Multi-hop FiLM are built in two different manners; while the former has three specialized reasoning blocks, our model uses a generic feature modulation approach. These architectural differences surface when examining test splits: MAttN achieves excellent results on referring expression towards objects while Multi-hop FiLM performs better on referring expressions towards people.

GuessWhat?! Oracle We report the best test error of several variants of GuessWhat?! Oracle models in Tab. 2. First, we baseline any visual or language biases by predicting the Oracle's target answer using only the image (46.7% error) or the question (41.1% error). As first reported in [START_REF] De Vries | Guesswhat?! visual object discovery through multi-modal dialogue[END_REF], we observe that the baseline methods perform worse when integrating the image and crop inputs (21.1%) rather than solely using the object category and spatial location (20.6%). On the other hand, concatenating previous question-answer pairs to answer the current question is beneficial in our experiments. Finally, using Single-hop FiLM reduces the error to 17.6% and Multi-hop FiLM further to 16.9%, outperforming the previous best model by 2.4%.

GuessWhat?! Guesser We provide the best test error of the outlined methods on the GuessWhat?! Guesser task in Tab. 3. As a baseline, we find that random object selection achieves an error rate of 82.9%. Our initial model baseline performs significantly worse (38.3%) than concurrent models (36.6%), highlighting that successfully jointly integrating crop and image features is far from trivial. However, Single-hop FiLM manages to lower the error to 35.6%. Finally, Multihop FiLM architecture outperforms other models with a final error of 30.5%.

Discussion

Single-hop FiLM vs. Multi-hop FiLM In the GuessWhat?! task, Multi-hop FiLM outperforms Single-hop FiLM by 6.1% on the Guesser task but only 0.7% on the Oracle task. We think that the small performance gain for the Oracle task is due to the nature of the task; to answer the current question, it is often not necessary to look at previous question-answer pairs, and in most cases this task does not require a long chain of reasoning. On the other hand, the Guesser task needs to gather information across the whole dialogue in order to correctly retrieve the object, and it is therefore more likely to benefit from multi-hop reasoning. The same trend can be observed for ReferIt. Single-hop FiLM and Multi-hop FiLM perform similarly on RefClef and RefCOCO, while we observe 1.3% and 2% gains on RefCOCO+ and RefCOCOg, respectively. This pattern of performance is intuitive, as the former datasets consist of shorter referring expressions (3.5 average words) than the latter (8.4 average words in RefCOCOg), and the latter datasets also consist of richer, more complex referring expressions due e.g. to taboo words (RefCOCO+). In short, our experiments demonstrate that Multihop FiLM is better able reason over complex linguistic sequences.

Reasoning mechanism We conduct several experiments to better understand our method. First, we assess whether Multi-hop FiLM performs better because of increased network capacity. We remove the attention mechanism over the linguistic sequence and update the context vector via a shared MLP. We observe that this change significantly hurts performance across all tasks, e.g., increasing the Multi-hop FiLM error of the Guesser from 30.5 to 37.3%. Second, we in- vestigate how the model attends to GuessWhat?! dialogues for the Oracle and Guesser tasks, providing more insight into how to the model reasons over the language input. We first look at the top activation in the (crop) attention layers to observe where the most prominent information is. Note that similar trends are observed for the image pipeline. As one would expect, the Oracle is focused on a specific word in the last question 99.5% of the time, one which is crucial to answer the question at hand. However, this ratio drops to 65% in the Guesser task, suggesting the model is reasoning in a different way. If we then extract the top 3 activations per layer, the attention points to <yes> or <no> tokens (respectively) at least once, 50% of the time for the Oracle and Guesser, showing that the attention is able to correctly split the dialogue into question-answer pairs. Finally, we plot the attention masks for each FiLM layer to have a better intuition of this reasoning process in Fig. 4. Crop vs. Image. We also evaluate the impact of using the image and/or crop on the final error for the Guesser task 3. Using the image alone (while still including object category and spatial information) performs worse than using the crop. However, using image and crop together inarguably gives the lowest errors, though prior work has not always used the crop due to architecture-specific GPU limitations [START_REF] De Vries | Modulating early visual processing by language[END_REF].

Visual feedback We explore whether adding visual feedback to the context embedding improves performance. While it has little effect on the GuessWhat?! Oracle and Guesser tasks, it improves the accuracy on ReferIt by 1-2%. Note that ReferIt does not include class labels of the selected object, so the visual feedback might act as a surrogate for this information. To further investigate this hypothesis, we remove the object category from the GuessWhat?! task and report results in Tab. 5 in the supplementary material. In this setup, we indeed observe a relative improvement 0.4% on the Oracle task, further confirming this hypothesis.

Pointing Task In GuessWhat?!, the Guesser must select an object from among a list of items. A more natural task would be to have the Guesser directly point out the object as a human might. Thus, in the supplementary material, we introduce this task and provide initial baselines (Tab. 7) which include FiLM models. This task shows ample room for improvement with a best test error of 84.0%.

Related Work

The ReferIt game [START_REF] Kazemzadeh | Referitgame: Referring to objects in photographs of natural scenes[END_REF] has been a testbed for various vision-and-language tasks over the past years, including object retrieval [START_REF] Nagaraja | Modeling context between objects for referring expression understanding[END_REF][START_REF] Yu | Modeling context in referring expressions[END_REF][START_REF] Yu | A joint speakerlistener-reinforcer model for referring expressions[END_REF][START_REF] Zhuang | Parallel attention: A unified framework for visual object discovery through dialogs and queries[END_REF][START_REF] Luo | Comprehension-guided referring expressions[END_REF][START_REF] Yu | Mattnet: Modular attention network for referring expression comprehension[END_REF], semantic image segmentation [START_REF] Hu | Segmentation from natural language expressions[END_REF][START_REF] Rohrbach | Grounding of textual phrases in images by reconstruction[END_REF], and generating referring descriptions [START_REF] Yu | Modeling context in referring expressions[END_REF][START_REF] Luo | Comprehension-guided referring expressions[END_REF][START_REF] Yu | A joint speakerlistener-reinforcer model for referring expressions[END_REF]. To tackle object retrieval, [START_REF] Nagaraja | Modeling context between objects for referring expression understanding[END_REF][START_REF] Yu | Modeling context in referring expressions[END_REF][START_REF] Yu | Mattnet: Modular attention network for referring expression comprehension[END_REF] extract additional visual features such as relative object locations and [START_REF] Yu | A joint speakerlistener-reinforcer model for referring expressions[END_REF][START_REF] Luo | Comprehension-guided referring expressions[END_REF] use reinforcement learning to iteratively train the object retrieval and description generation models. Closer to our work, [START_REF] Hu | Natural language object retrieval[END_REF][START_REF] Zhuang | Parallel attention: A unified framework for visual object discovery through dialogs and queries[END_REF] use the full image and the object crop to locate the correct object. While some previous work relies on task-specific modules [START_REF] Yu | Modeling context in referring expressions[END_REF][START_REF] Yu | Mattnet: Modular attention network for referring expression comprehension[END_REF], our approach is general and can be easily extended to other vision-and-language tasks.

The GuessWhat?! game [START_REF] De Vries | Guesswhat?! visual object discovery through multi-modal dialogue[END_REF] can be seen as a dialogue version of the ReferIt game, one which additionally draws on visual question answering ability. [START_REF] Strub | End-to-end optimization of goal-driven and visually grounded dialogue systems harm de vries[END_REF][START_REF] Lee | Answerer in questioner's mind for goal-oriented visual dialogue[END_REF][START_REF] Zhu | Reasoning about fine-grained attribute phrases using reference games[END_REF] make headway on the dialogue generation task via reinforcement learning. However, these approaches are bottlenecked by the accuracy of Oracle and Guesser models, despite existing modeling advances [START_REF] Zhuang | Parallel attention: A unified framework for visual object discovery through dialogs and queries[END_REF][START_REF] De Vries | Modulating early visual processing by language[END_REF]; accurate Oracle and Guesser models are crucial for providing a meaningful learning signal for dialogue generation models, so we believe the Multi-hop FiLM architecture will facilitate high quality dialogue generation as well.

A special case of Feature-wise Linear Modulation was first successfully applied to image style transfer [START_REF] Dumoulin | A Learned Representation For Artistic Style[END_REF], whose approach modulates image features according to some image style (i.e., cubism or impressionism). [START_REF] De Vries | Modulating early visual processing by language[END_REF] extended this approach to vision-and-language tasks, injecting FiLM-like layers along the entire visual pipeline of a pre-trained ResNet. [START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF] demonstrates that a convolutional network with FiLM layers achieves strong performance on CLEVR [START_REF] Johnson | Clevr: A diagnostic dataset for compositional language and elementary visual reasoning[END_REF], a task that focuses on answering reasoning-oriented, multi-step questions about synthetic images. Subsequent work has demonstrated that FiLM and variants thereof are effective for video object segmentation where the conditioning input is the first image's segmentation (instead of language) [START_REF] Yang | Efficient video object segmentation via network modulation[END_REF] and language-guided image segmentation [START_REF] Rupprecht | Guide me: Interacting with deep networks[END_REF]. Even more broadly, [START_REF] Dumoulin | Feature-wise transformations[END_REF] overviews the strength of FiLMrelated methods across machine learning domains, ranging from reinforcement learning to generative modeling to domain adaptation.

There are other notable models that decompose reasoning into different modules. For instance, Neural Turing Machines [START_REF] Graves | Neural turing machines[END_REF][START_REF] Graves | Hybrid computing using a neural network with dynamic external memory[END_REF] divide a model into a controller with read and write units. Memory networks use an attention mechanism to answer a query by reasoning over a linguistic knowledge base [START_REF] Weston | Memory networks[END_REF][START_REF] Sukhbaatar | End-to-end memory networks[END_REF] or image features [START_REF] Xiong | Dynamic memory networks for visual and textual question answering[END_REF]. A memory network updates a query vector by performing several attention hops over the memory before outputting a final answer from this query vector. Although Multi-hop FiLM computes a similar context vector, this intermediate embedding is used to predict FiLM parameters rather than the final answer. Thus, Multi-hop FiLM includes a second reasoning step over the image.

Closer to our work, [START_REF] Hudson | Compositional attention networks for machine reasoning[END_REF] designed networks composed of Memory, Attention, and Control (MAC) cells to perform visual reasoning. Similar to Neural Turing Machines, each MAC cell is composed of a control unit that attends over the language input, a read unit that attends over the image and a write unit that fuses both pipelines. Though conceptually similar to Multi-hop FiLM models, Compositional Attention Networks differ structurally, for instance using a dynamic neural architecture and relying on spatial attention rather than FiLM.

Conclusion

In this paper, we introduce a new way to exploit Feature-wise Linear Modulation (FiLM) layers for vision-and-language tasks. Our approach generates the parameters of FiLM layers going up the visual pipeline by attending to the language input in multiple hops rather than all at once. We show Multi-hop FiLM Generator architectures are better able to handle longer sequences than their single-hop counterparts. We outperform state-of-the-art vision-and-language models significantly on the long input sequence GuessWhat?! tasks, while maintaining stateof-the-art performance for the shorter input sequence ReferIt task. Finally, this Multi-hop FiLM Generator approach uses few problem-specific priors, and thus we believe it can extended to a variety of vision-and-language tasks, particularly those requiring complex visual reasoning. For existing tasks on the GuessWhat?! dataset, the Guesser selects its predicted target object from among a provided list of possible answers. A more natural task would be for the Guesser to directly point out the object, much as a human might. Thus, we introduce a pointing task as a new benchmark for GuessWhat?!. The specific task is to locate the intended object based on a series of questions and answers; however, instead of selecting the object from a list, the Guesser must output a bounding box around the object of its guess, making the task more challenging. This task also does not include important side information, namely object category and (x,y)-position [START_REF] De Vries | Guesswhat?! visual object discovery through multi-modal dialogue[END_REF], making the object retrieval more difficult than the originally introduced Guesser task as well. The bounding box is defined more specifically as the 4-tuple (x, y, width, height), where (x, y) is the coordinate of the top left corner of the box within the original image I, given an input dialogue.

Guesser (Pointing Task)

We assess bounding box accuracy using the Intersection Over Union (IoU) metric: the area of the intersection of predicted and ground truth bounding boxes, divided by the area of their union. Prior work [START_REF] Everingham | The pascal visual object classes (voc) challenge[END_REF][START_REF] Girshick | Rich feature hierarchies for accurate object detection and semantic segmentation[END_REF], generally considers an object found if IoU exceeds 0.5.

IoU = |bboxA ∩ bboxB| |bboxA ∪ bboxB| = |bboxA ∩ bboxB| |bboxA| + |bboxB| -|bboxA ∩ bboxB| (10)
We report model error in Table 7. Interestingly, the baseline obtains 92.0% error while Multi-hop FiLM obtains 84.0% error. As previously mentioned, reinjecting visual features into the Multi-hop FiLM Generator's context cell is beneficial. The error rates are relatively high but still in line with those of similar pointing tasks such as SCRC [START_REF] Hu | Segmentation from natural language expressions[END_REF][START_REF] Hu | Natural language object retrieval[END_REF] (around 90%) on ReferIt.

Attention Visualizations

Fig. 4 :

 4 Fig. 4: Guesser (left) and Oracle (right) attention visualizations for the visual pipeline which processes the object crop.

Fig. 5 :

 5 Fig. 5: The crop pipeline Oracle's attention over the last question when the model succeeds.

Fig. 6 :

 6 Fig. 6: The crop pipeline Oracle's attention over the last question, showing more advanced reasoning.

Fig. 7 :

 7 Fig. 7: The crop pipeline Oracle's attention over the last question when the model fails.

Fig. 8 :

 8 Fig. 8: The crop pipeline Guesser's attention when the model succeeds.

Fig. 9 :

 9 Fig. 9: The crop pipeline Guesser's attention when the model fails.

Table 1 :

 1 ReferIt Guesser Error.

	Referit		RefCOCO			RefCOCO+		RefCOCOg
	Split by		(unc)			(unc)		(google)
	Report on	Valid	TestA	TestB	Valid	TestA	TestB	Val
	MMI [36]	-	71.7%	71.1%	-	58.4%	51.2%	59.3%
	visDif + MMI [51]	-	74.6%	76.6%	-	59.2%	55.6%	64.0%
	NEG Bag [36]	-	75.6%	78.0%	-	-	-	68.4%
	Joint-SLR [52]	78.9%	78.0%	80.7%	61.9%	64.0%	59.2%	-
	PLAN [54]	81.7%	80.8%	81.3%	64.2%	66.3%	61.5%	69.5%
	MAttN [50]	85.7%	85.3%	84.6%	71.0%	75.1%	66.2%	-
	Baseline NN+MLB	77.6%	79.6%	77.2%	60.8%	59.7%	66.2%	63.1%
	Single-hop FiLM	83.4%	85.8%	80.9%	72.1%	77.3%	63.9%	67.8%
	Multi-hop FiLM	83.5%	86.5%	81.3%	73.4%	77.7%	64.5%	69.8%
	Multi-hop FiLM (+img) 84.9%	87.4%	83.1%	73.8%	78.7%	65.8%	71.5%

Table 2 :

 2 GuessWhat?! Oracle Error by Model and Input Type.

	Oracle Models	Quest.	Dial.	Object	Image	Crop	Test Error
	Dominant class ("no")						50.9%
	Question only [6]						41.2%
	Image only [6]						46.7%
	Crop only [6]						43.0%
	No-Vision (Quest.) [6]						21.5%
	No-Vision (Dial.)						20.6%
	Baseline NN (Quest.)						23.3%
	Baseline NN (Dial.)						22.4%
	Baseline NN + MLB (Quest.)						21.8%
	Baseline NN + MLB (Dial.)						21.1%
	MODERN [44]						19.5%
	Single-hop FiLM (Quest.)						17.8%
	Single-hop FiLM (Dial.)						17.6%
	Multi-hop FiLM						16.9%
	Multi-hop FiLM (+img)						17.1%

Table 3 :

 3 GuessWhat?! Guesser Error.

	Guesser Error	Test Error
	Random	82.9%	Guesser Error	Crop	Image	Crop+Img
	LSTM [6] LSTM + Img [6] PLAN [54]	38.7% 39.5% 36.6%	Baseline NN Single-hop FiLM Multi-hop FiLM	38.3% 35.3% 32.3%	40.0% 35.7% 35.0%	45.1% 35.6% 30.5%
	Base NN + MLB (crop)	38.3%	Multi-hop FiLM (no categ.) 33.1%	40%	33.4%
	Single-hop FiLM	35.6%
	Multi-hop FiLM	30.5%

Table 7 :

 7 Guesser pointing error for different IoU thresholds.

	Guesser Model	IoU > 0.3 IoU >0.5 IoU > 0.7
	Baseline NN	81.4%	92.0%	98.2%
	FiLM	74.0%	85.9%	94.7%
	Multi-hop FiLM	73.4%	84.6%	93.7%
	Multi-hop FiLM (+img) 71.9%	84.0%	93.6%

The code and hyperparameters are available at https://github.com/ GuessWhatGame

Acknowledgements The authors would like to acknowledge the stimulating research environment of the SequeL Team. We also thank Vincent Dumoulin for helpful discussions. We acknowledge the following agencies for research funding and computing support: Project BabyRobot (H2020-ICT-24-2015, grant agreement no.687831), CHISTERA IGLU and CPER Nord-Pas de Calais/FEDER DATA Advanced data science and technologies 2015-2020, NSERC, Calcul Québec, Compute Canada, the Canada Research Chairs, and CIFAR.

Additional Results

ReferIt ImageClef


Elise Buisson
email: elise.buisson@univ-avignon.fr

Renaud Jaunatre

Christine Römermann

Adeline Bulot

Thierry Dutoit

Species transfer via topsoil translocation: lessons from two large Mediterranean restoration projects Running head: Mediterranean grassland topsoil translocation

Keywords: bulk topsoil transfer, sod dumping, soil inoculation, seed bank, steppe Implications

Restoration success of species-rich grasslands is often limited by low seed dispersal. To reintroduce target species of local provenances, bulk topsoil transfer is performed by excavating soil and vegetation from donor sites and spreading these on receptor sites. Our first objective was to determine the most advantageous season for transferring topsoil in Mediterranean grasslands. The second objective was to assess which treatment -combination of season (spring or autumn) and transfer ratio (1:1 or 1:3) -performed best in restoring a Mediterranean grassland through bulk topsoil transfer after two large restoration projects.

Just-in-time soil transfer (i.e. with no stockpiling) was implemented 1) on a former 3-ha orchard where topsoil was spread after removing trees and levelling soil, and 2) after a 5-ha oil pipeline leak where polluted soil was removed prior to treatment (soil horizons were reconstituted). A seed bank study showed that the summer seed bank contained higher seed densities, species richness and similarity to the reference site than the spring seed bank.

Spring transfers gave better results than autumn transfers in terms of species richness and composition similarity with the reference site, while transfer ratios gave similar results. Longterm success was not driven by season or transfer ratio but by the underlying seed bank at receptor sites: the former orchard's weed-containing seed bank hampered topsoil transfer success. This study also suggests that restoration success cannot be deduced from seed bank studies alone, as species establishment is highly dependent on differences in growing conditions (including competition at receptor sites).

Introduction

The restoration of species-rich grasslands is often limited by low seed dispersal [START_REF] Buisson | The implications of seed rain and seed bank patterns for plant succession at the edges of abandoned fields in Mediterranean landscapes[END_REF][START_REF] Grman | Dispersal and establishment filters influence the assembly of restored prairie plant communities[END_REF][START_REF] Halassy | The application of a filter-based assembly model to develop best practices for Pannonian sand grassland restoration[END_REF]), both in space and time (e.g. [START_REF] Römermann | Influence of former cultivation on the unique Mediterranean steppe of France and consequences for conservation management[END_REF]. Successful methods of reintroducing target species of local provenances commonly include sowing seed mixtures and transferring seed-containing hay or soil [START_REF] Kiehl | Species introduction in restoration projects -Evaluation of different techniques for the establishment of seminatural grasslands in Central and Northwestern Europe[END_REF].

Soil transfer can be accomplished through i) turf transfer, also called community translocation, or ii) bulk topsoil transfer, also called sod dumping [START_REF] Bullock | Community translocation in Britain: Setting objectives and measuring consequences[END_REF][START_REF] Kiehl | Species introduction in restoration projects -Evaluation of different techniques for the establishment of seminatural grasslands in Central and Northwestern Europe[END_REF]Jaunatre 2014a;[START_REF] Bulot | comparison of different soil transfer strategies for restoring a Mediterranean steppe after a pipeline leak (La Crau plain, South-Eastern France)[END_REF][START_REF] Bulot | Ecological restoration by soil transfer: impacts on restored soil profiles and topsoil functions[END_REF]. Soil transfer means that not only seeds are transferred, but also other propagules such as rhizomes, small shallow-rooted plants, and micro-organisms that may play an important role in the restoration of the plant community [START_REF] Török | Grassland restoration on former croplands in Europe: an assessment of applicability of techniques and costs[END_REF][START_REF] Wubs | Soil inoculation steers restoration of terrestrial ecosystems[END_REF]. Of the two techniques, bulk topsoil transfer is the cheapest [START_REF] Good | Translocation of herb-rich grassland from a site in Wales prior to opencast coal extraction[END_REF], especially if soil is spread at a ratio lower than 1:1, i.e. spread on an area larger than where it was gathered (e.g. ratio of 1:2, [START_REF] Good | Translocation of herb-rich grassland from a site in Wales prior to opencast coal extraction[END_REF]; ratio of 1:3, Jaunatre et al. 2014a).

While both soil transfer techniques are based on extracting soil from donor sites scheduled to be destroyed and thus are not sustainable, they are fundamentally different. Turf transfer consists in cutting and transferring turfs and mainly relies on the survival and colonization potential of the plants contained in the turfs [START_REF] Bullock | Community translocation in Britain: Setting objectives and measuring consequences[END_REF][START_REF] Kardol | Soil organism and plant introductions in restoration of species-rich grassland communities[END_REF][START_REF] Vécrin | Top-soil translocation as a technique in the re-creation of species-rich meadows[END_REF]. Bulk topsoil transfer, the type explored in this study, consists in excavating soil and vegetation from donor sites and spreading these on receptor sites, often at a ratio of 1:1 (see [START_REF] Kiehl | Species introduction in restoration projects -Evaluation of different techniques for the establishment of seminatural grasslands in Central and Northwestern Europe[END_REF] for review); it relies mainly on the availability of viable soil-stored seeds and other propagules.

Prior to large-scale bulk topsoil transfer, seed bank studies are often carried out at the donor sites. First, restorationists need to evaluate the potential of implementing such a technique: topsoil transfer is worth implementing only if the topsoil contains seeds of target species, particularly in regions where seeds are not available commercially [START_REF] Fowler | Evaluating restoration potential of transferred topsoil[END_REF]. Using soil seed banks as a source of material for restoration of various plant communities (forests, marshes, grasslands, heathlands) has been widely explored via seed bank studies (e.g. the review by [START_REF] Bossuyt | Can the seed bank be used for ecological restoration? An overview of seed bank characteristics in European communities[END_REF] of 102 studies and the case study by [START_REF] Kalamees | Restoration potential of the persistent soil seed bank in successional calcareous (alvar) grasslands in Estonia[END_REF]. With the exception of some grasslands, heathlands and early successional communities, these 103 seed bank studies concluded that restoration cannot rely solely on seed banks. However, bulk topsoil transfer is still used in many countries and for the restoration of various types of ecosystems, such as roadsides, meadows, grasslands, forests, etc., with relative success [START_REF] Kiehl | Species introduction in restoration projects -Evaluation of different techniques for the establishment of seminatural grasslands in Central and Northwestern Europe[END_REF][START_REF] Fowler | Evaluating restoration potential of transferred topsoil[END_REF]. While this discrepancy between the findings of seed bank studies and the continuing practice of bulk topsoil transfer [START_REF] Fowler | Evaluating restoration potential of transferred topsoil[END_REF] remains insufficiently studied, both bulk topsoil transfer and seed bank studies continue to be performed.

The second reason for performing seed bank studies prior to large-scale bulk topsoil transfer is to determine the most advantageous time for transfer. Ideally, transfer should be carried out at a season when a maximum proportion of the target species are found as seeds in the soil (De [START_REF] Villiers | The restoration of Strandveld and Succulent Karoo degraded by mining: an enumeration of topsoil seed banks[END_REF][START_REF] Hosogi | Timing for the collection of topsoil from a deciduous forest for use as planting material in suburban Tokyo, Japan[END_REF]), typically at the end of the vegetation period and, in the Mediterranean region, before the onset of the autumn rainy season [START_REF] Grillas | Les mares temporaires méditerranéennes: enjeux de conservation[END_REF][START_REF] Villiers | The restoration of Strandveld and Succulent Karoo degraded by mining: an enumeration of topsoil seed banks[END_REF]). This is particularly important when aiming to restore plant communities containing many species with a transient seed bank [START_REF] Buisson | Creation of the natural reserve of La Crau: Implications for the creation and management of protected areas[END_REF]. The third objective of prior seed bank studies is to permit subsequent evaluation of restoration success. After topsoil transfer, monitoring is carried out to document vegetation establishment on the receptor sites; the composition of the vegetation is often compared with the seed bank and vegetation of the donor sites to evaluate restoration success [START_REF] Vécrin | Top-soil translocation as a technique in the re-creation of species-rich meadows[END_REF].

The objectives of this study are 1) using a seed bank study, to assess whether spring or summer is more advantageous for transferring topsoil to restore Mediterranean grasslands and 2) using a topsoil transfer experiment, to assess the best combination of season (spring or autumn) and bulk topsoil transfer ratio (1:1 or 1:3 hereafter termed topsoil transfer ratio) to restore a Mediterranean grassland plant community. The best transfer time should be summer, when topsoil contains both species that form a more persistent seed bank and species that form only a transient seed bank; however, this season could not be tested mainly due to constraints inherent to worksites. Soil transfers were implemented in autumn and spring. Our hypotheses are that i) the best transfer time between spring and autumn is spring, when topsoil contains the few species fruiting in late autumn, ii) the best topsoil transfer treatment is at a ratio of 1:1 (delivering a higher number of seeds per area unit).

Methods

Study sites

The study sites are situated in south-eastern France, in a pseudo-steppe located in the plain of La Crau which has recently undergone several large restoration projects, three of which included soil transfer. La Crau is the ancient delta of the Durance River, whose dry grassland vegetation once covered 60 000 ha. Dominated by perennial species and also including many annual species, this vegetation has been shaped by a dry and windy Mediterranean climate, particular soil conditions (i.e. shallow, relatively poor Haplic Cambisol (Calcaric) soil with silicaceous stones, covering about 50% of the ground) and many centuries of itinerant sheep grazing [START_REF] Buisson | The implications of seed rain and seed bank patterns for plant succession at the edges of abandoned fields in Mediterranean landscapes[END_REF]; Figure S1). This now fragmented grassland vegetation, of which only 11 500 ha remain, serves as a reference ecosystem and donor sites for soil transfers [START_REF] Jaunatre | Restoring speciesrich Mediterranean dry grassland in France using different species-transfer methods[END_REF]). The first restoration site is an area of 357 ha managed from 1987 to 2006 as an orchard, and rehabilitated in 2008 as part of the first mitigation banking experiment launched by the French government [START_REF] Dutoit | Mediterranean steppe restoration in France[END_REF]. Concurrently with this rehabilitation project, restoration trials were carried out on this orchard: direct topsoil transfer from La Crau dry Mediterranean grassland (with no stockpiling) was implemented on a few hectares in spring and autumn 2009 at ratios of 1:1 or 1:3 (partly described in Jaunatre et al. 2014a; Figure S1). Less than one kilometer away from this former orchard (Figure 1), the grassland soil and vegetation of a 5.5 ha grassland area were completely destroyed by an oil spill due to an underground pipeline leak in 2009 (second site hereafter referred to as pipeline leak site). Restoration consisted in excavating the polluted soil down to the bedrock (40 cm) followed by direct topsoil transfer from La Crau dry Mediterranean grassland, reconstituting soil horizons in spring 2011 [START_REF] Bulot | Ecological restoration by soil transfer: impacts on restored soil profiles and topsoil functions[END_REF]; Figure S1).

Seed bank study

To identify the season when the topsoil seed bank of the reference site has the highest seed content, ten replicate 2.5 L soil samples were taken at the site of Peau de Meau, in the center of the plain of La Crau, in April and August 2001 (Table 1; [START_REF] Römermann | Influence of former cultivation on the unique Mediterranean steppe of France and consequences for conservation management[END_REF]. These 20 samples were left to germinate following the protocol described hereafter. Each soil sample taken in the field consisted of 10 pooled subsamples, as recommended by Bakker et al. (1996) taken between 0 and 20 cm deep using a 4-cm diameter soil corer. The seedling emergence method [START_REF] Heerdt | An improved method for seedbank analysis: seedling emergence after removing the soil by sieving[END_REF] was selected to qualify and quantify germinable seeds in the soil seed banks. Soil samples were concentrated by washing them with water on two different sieves (2 mm and 200 µm) to reduce bulk and clay. The concentrated soil was spread in a thin layer (0.5 cm) on plastic trays filled with 2 cm vermiculite topped with medical compresses (mesh size about 100 µm). The trays were covered with fine gauze to prevent contamination. All trays were watered frequently from below. Emerged seedlings were identified, counted and removed weekly. Unknown seedlings were grown for identification. After three months of cultivation (greenhouses), the samples were allowed to dry out. Following six weeks of cold stratification (5°C), the samples were cultivated for a further three months. Vegetation monitoring was carried out in ten replicate 4-m² quadrats in May 2001 (Table 1): we recorded the presence of each species.

Topsoil transfers and vegetation sampling

Direct topsoil transfer of the 20-cm upper soil layer was implemented on the former orchard in mid-March 2009 at a ratio of 1:3 on 1 ha (Table 1 and2), and in late October 2009 both at a ratio of 1:3 on 3 ha (Jaunatre et al. 2014a) and at a ratio of 1:1 on 0.3 ha (Jaunatre et al. 2014a). Direct topsoil transfer was also implemented on the pipeline leak site in spring 2011 at a ratio of 1:1 on 5.5 ha (Table 1 and2; [START_REF] Bulot | Ecological restoration by soil transfer: impacts on restored soil profiles and topsoil functions[END_REF]. At the former orchard, topsoil was spread over the orchard soil (Jaunatre et al. 2014a). At the pipeline leak site, all soil horizons were reconstituted: i) altered bedrock (35-40 cm deep), ii) subsoil (20-35 cm deep) and iii) topsoil (0-20 cm deep) [START_REF] Bulot | Ecological restoration by soil transfer: impacts on restored soil profiles and topsoil functions[END_REF][START_REF] Bulot | comparison of different soil transfer strategies for restoring a Mediterranean steppe after a pipeline leak (La Crau plain, South-Eastern France)[END_REF]. Vegetation monitoring was carried out in May 2010, May 2011 and May 2012 on the former orchard and in May 2012, May quadrats were sampled (Table 1): we recorded the cover of each species and the total percent cover of vegetation using a modified Braun-Blanquet scale [START_REF] Braun-Blanquet | Les groupements végétaux de la France méditerranéenne[END_REF]): 0.5 for species covering less than 1% of the quadrat, 1 for those covering between 1% and 5%, 2 for those covering between 5% and 25%, 3 for those covering between 25% and 50%, 4 for those covering between 50% and 75% and 5 for species covering more than 75% of the quadrat.

Data analysis

To identify the best time to transfer topsoil to the former orchard and pipeline leak sites, we compared spring and summer seed bank characteristics. Differences in composition between seed banks were assessed using i) species richness and seed density per sample and ii) a redundancy analysis (RDA; matrix: 39 samples × 50 species) based on abundance data from the seed bank, after a Hellinger transformation [START_REF] Legendre | Ecologically meaningful transformations for ordination of species data[END_REF]. Differences in composition between seed banks with above-ground vegetation were assessed using normalized Community Structure Integrity Indices (CSIInorm) calculated on plant species presence / absence data [START_REF] Jaunatre | New synthetic indicators to assess community resilience and restoration success[END_REF]. To compare means between seasons, GLMs using a Poisson distribution were performed on species richness and seed abundance data as they are count data, and a t-test was performed on CSIInorm as they met parametric conditions.

To compare topsoil transfers implemented to restore Mediterranean dry grassland at the former orchard and pipeline leak sites, and to test the effects of season and transfer ratio (treatments = ratio-season combination) on success, we compared plant species richness and composition between the restored sites. Here again, differences in composition were assessed using i) a normalized Community Structure Integrity Index (CSIInorm) calculated on plant species presence / absence data and ii) a non-metric multidimensional scaling (NMDS; matrix: 33 quadrats × 99 species) based on Bray-Curtis distance calculated from presence / absence data. As species richness data is count data, they were analyzed with a Generalized Linear Model (GLM) procedure using a Poisson distribution to compare means between restored sites. CSIInorm met parametric conditions; we therefore performed an ANOVA.

Three pairwise comparisons, with Benjamini-Hochberg adjustment, were carried out: 1) to compare dates of transfer at the 1:3 ratio on the former orchard, 2) to compare autumn transfer ratios on the former orchard and 3) to compare the spring 1:3 transfer on the former orchard and the spring 1:1 transfer on the pipeline leak site. To determine the combined effect of year (1, 2 and 3 years after transfer), and treatments (ratio-season combination) on CSIInorm we performed two-way ANOVA on repeated measures, followed by Tukey honestly significant difference post-hoc tests.

All analyses were performed with R software 3.3.2 (R Core Team 2016) and its 'vegan' package [START_REF] Oksanen | ) R: A Language and Environment for Statistical Computing[END_REF].

Results

Seed bank

The soil seed bank collected in summer contained more species (15.2±0.6 species / sample) than that collected in early spring (5.4±1.0 species / sample) (z = 45.08, p<0.001; Figure 2a).

The seed bank collected in summer also contained more seeds (101.1±8.3 seeds / sample) than that collected in early spring (8.7±1.8 seeds / sample) (z = -852.86, p<0.001; Figure 2b).

Although they shared many species, the two seed bank communities were separated along the RDA first axis (Figure 3). Spring samples were differentiated from summer samples by the presence of Sagina apetala, absent from the summer seed bank. The summer seed bank samples were characterized by species like Gastridium ventricosum, Vulpia sp., Aira cupaniana, Catapodium rigidum, Galium parisiense, Bromus hordeaceus and Melica ciliata which were not, or only rarely, found in the spring seed bank. The samples collected in spring were more dispersed on the RDA ordination than those collected in summer. Most of the species found in the spring and summer seed bank were species found in the reference grassland (thus target species), although similarity was below 0.4 for both seasons (Figure 3; t=-6.58; p<0.001). Out of the 1089 germinations, we identified 50 species (14 in spring and 38 in summer), only 5 of which were non-target species. Senecio vulgaris (49 germinations), which mainly germinated outside the summer seed bank, is naturally and sporadically found in La Crau grasslands. The other four species, Verbena officinalis (65 germinations, 13 in spring and 52 in summer), Kickxia elatine (3 germinations, 1 in spring and 2 in summer) Chenopodium album (2 germinations, 1 in spring and 1 in summer) and Portulaca oleracea (2 germinations in spring) were not found in the above-ground grassland vegetation; they are arable weeds dispersed from formerly irrigated fields found around the study site [START_REF] Römermann | Influence of former cultivation on the unique Mediterranean steppe of France and consequences for conservation management[END_REF][START_REF] Buisson | Creation of the natural reserve of La Crau: Implications for the creation and management of protected areas[END_REF].

Topsoil transfers

The year following topsoil transfer, species richness (z = 106.59; p<0.001) and similarity to the reference site (CSIInorm; F = 14.35; p<0.001) significantly differed between treatments (ratio-season combination): both these variables were higher for spring soil transfer than for autumn soil transfer on the former orchard (see results of post-hoc tests, Figure 4). Moreover, for the spring transfers, similarity to the reference site was slightly significantly higher with the 1:1 transfer ratio on the pipeline leak site than with the 1:3 transfer ratio on the former orchard (Figure 4). Transfer ratios did not affect species richness and similarity to the reference site for the former orchard. In spring, similarity was higher for the pipeline leak site transfer at a ratio of 1:1 than for the former orchard transfer at a ratio of 1:3 (Figure 4).

Overall, the plant compositions of the different topsoil transfers were rather similar, with many species characteristic of the reference grassland such as Carlina corymbosa, Thymus vulgaris or Trifolium stellatum (Figure 5). However the topsoil transferred onto the pipeline leak site had a slightly different plant composition characterized by ruderal species like Cardamine hirsuta or Lolium perenne. The topsoil transferred onto the former orchard was characterized by ruderal species like Bromus diandrus, Chenopodium album, Hirschfeldia incana and Lactuca serriola (Figure 5).

From the second year after transfers, only the spring pipeline leak site transfer at a ratio of 1:1 retained high similarity to the reference site, above 80%. For all transfers implemented at the former orchard site, regardless of transfer season or ratio, similarity dropped to below 60% (Figure 6).

Discussion

Topsoil transfer timing

The La Crau Mediterranean dry grassland has a mainly transient seed bank [START_REF] Buisson | Creation of the natural reserve of La Crau: Implications for the creation and management of protected areas[END_REF]. Since most of the plants are annuals and spend the harsh season (summer) as seeds in the seed bank, the most advantageous season for topsoil transfer should clearly be summer.

Moreover, seeds can be expected to germinate in autumn: in a Mediterranean climate, not only is germination triggered by temperature and light, but it also occurs only after a threshold amount of precipitation [START_REF] Ne'eman | Phenology of east-Mediterranean vegetation[END_REF]. Differences cannot simply be explained by dormancy cycles, as germination of annual Mediterranean species depends on the timing and amount of autumn precipitation [START_REF] Espigares | Mediterranean pasture dynamics: the role of germination[END_REF][START_REF] Espigares | Mediterranean annual pasture dynamics: impact of autumn drought[END_REF]. The results of our seed bank study support our first hypothesis, showing that species richness and seed density and similarity of species composition between seed bank and above-ground vegetation of the reference site are much higher in the summer than in the spring seed bank.

The latter is more dispersed on the community ordination because only a few seeds germinated in each spring sample [START_REF] Römermann | Influence of former cultivation on the unique Mediterranean steppe of France and consequences for conservation management[END_REF], giving each species a higher weight than the species composing the more abundant summer seed bank community.

While summer appears to be the most appropriate season for transferring the seed bank, restoration work cannot always be implemented when it would be ideal in terms of resilience of the restored plant community. This is mainly due to constraints inherent to worksites and because restoration planning usually takes into account not only the phenology of plant communities but also that of endangered plant species and of animals, such as nesting birds [START_REF] Buisson | Plant phenological research enhances ecological restoration[END_REF]). Thus, topsoil transfer was never implemented in summer to restore our Mediterranean dry grassland.

Season for Mediterranean dry grassland topsoil transfer

Topsoil transfer was not carried out in summer on the Mediterranean grassland due to site preparation delays (soil levelling on the former orchard and polluted soil excavation on the pipeline leak site; see Jaunatre et al. 2014a;[START_REF] Bulot | comparison of different soil transfer strategies for restoring a Mediterranean steppe after a pipeline leak (La Crau plain, South-Eastern France)[END_REF]; 2017 for details). As many annual species germinate with the first autumn rains, it was difficult to hypothesize that either autumn or spring would be the best transfer season. In the first year, spring topsoil transfer led to high species richness on the restoration site and high similarity with the reference site (ca. 80%) compared to autumn transfer (slightly over 40%). This may be partly due to the fact that a few species, such as Bellis sylvestris, Lobularia maritima, flower and produce seeds mainly in autumn and are thus added to the pool of potentially germinable species in spring. In this study, these two species germinated with both transfers but their frequency of occurrence was higher on the restoration sites where spring topsoil transfers were performed.

We observed annual Fabaceae common in the reference grassland, such as Medicago minima, Medicago monspeliaca, Medicago truncatula, Trifolium cherleri, Trifolium scabrum and Trifolium stellatum, more frequently with the spring than with the autumn transfers. Van [START_REF] Van Assche | Seasonal cycles in the germination capacity of buried seeds of some Leguminosae (Fabaceae)[END_REF] showed that some species congeneric to those cited here show increased germination efficiency with cold stratification. They would therefore germinate better after winter, explaining the results from spring transfers in our study. Surprisingly, numerous annual species known through seedling emergence studies to have a transient seed bank and accordingly not found in our spring seed bank study, were found under spring soil transfer. This may partly be due to the fact that species with low seed production and species whose germination requirements are narrow are less likely to be detected by seed bank studies [START_REF] Saatkamp | The seed bank longevity index revisited: limited reliability evident from a burial experiment and database analyses[END_REF]. Moreover, species whose seeds massively germinate with autumn rain are hard to detect in spring. It can thus be expected that a wider range of species will establish from spring topsoil transfer than predicted from seed bank studies [START_REF] Kalamees | The role of the seed bank in gap regeneration in a calcareous grassland community[END_REF]. Here, the restoration following polluted soil excavation on the pipeline leak site, which involved a spring transfer of 41 000 tons of soil, gave better results than predicted via seed bank studies on a few kilos of soil [START_REF] Bulot | comparison of different soil transfer strategies for restoring a Mediterranean steppe after a pipeline leak (La Crau plain, South-Eastern France)[END_REF].

Perennial species of this Mediterranean dry grassland do produce seeds that can germinate in autumn in the field and form a seed bank (e.g. Dactylis glomerata; 64 germinations in the summer seed bank vs. 14 in the spring seed bank / 1089 germinations).

Perennial plants also make it through summer drought thanks to the survival of the organs (meristematic tissues at the bases of leaves, roots, underground storage organs) from which they regrow once water is available [START_REF] Volaire | Summer drought survival strategies and sustainability of perennial temperate forage grasses in Mediterranean areas[END_REF][START_REF] Moreira | To resprout or not to resprout: factors driving intraspecific variability in resprouting[END_REF]. They store carbohydrates during the growing season, before the onset of drought, which allows them to maintain respiration demands during summer and to initiate resprouting [START_REF] Volaire | Summer drought survival strategies and sustainability of perennial temperate forage grasses in Mediterranean areas[END_REF][START_REF] Moreira | To resprout or not to resprout: factors driving intraspecific variability in resprouting[END_REF]. Thus, topsoil transfer in autumn may not have given them enough time to replenish their reserves to be resilient to a second stressful situation, such as translocation:

Brachypodium retusum established better in spring than in autumn, as did Cynodon dactylon and Bothriochloa ischaemum, though to a lesser extent. An attempt to translocate perennial species in summer would show whether experiencing two simultaneous disturbances (drought and translocation), rather than one occurring after the other, provides a better chance of survival.

Differences between restored grassland sites

Topsoil transfer on the pipeline leak site yielded few ruderal species, and even those were ruderal species also found on the reference grassland, such as Cardamine hirsuta or Lolium perenne. Contrastingly, topsoil transfer on the former orchard led to communities dominated by ruderal species like Bromus diandrus, Chenopodium album and Hirschfeldia incana, absent from the reference grassland but common on the rehabilitated areas of the former orchard where topsoil transfer was not performed (Jaunatre et al 2014a). This major difference between the two restored sites is due to a difference in restoration process: on the pipeline leak site, the soil horizons were entirely reconstituted, while on the former orchard, topsoil was spread on top of orchard soil containing its own seed bank of weeds (Jaunatre et al 2014a). It is likely that the different restoration processes also largely explain the differences between transfer ratios in spring: the 1:1 ratio soil transfer implemented on the pipeline leak site produced results more similar to the reference one year after transfer than the 1:3 ratio soil transfer implemented on the former orchard.

Autumn soil transfer ratios on the former orchard did not affect species richness and similarity to the reference one year after transfer. The representation on the ordination of the autumn 1:3 transfer ratio on the former orchard is more dispersed, most probably due to the greater sampling effort.

In subsequent years, 80% similarity to the reference site was maintained with the spring 1:1 ratio topsoil transfer on the pipeline leak site, whereas the spring 1:3 ratio topsoil transfer on the former orchard led to similarity falling to below 50%. There are two possible explanations. First, the 1:3 ratio does not transfer enough seeds for the community to be maintained long-term. However, this is contradicted by the fact that similarity with the reference is identical with both ratios for the autumn transfers. The second likely explanation is the differences between sites and restoration processes. The seed bank and soil nutrient content are affected by cultivation legacy, and there can be long-term effects on plant communities [START_REF] Cramer | What's new about old fields? Land abandonment and ecosystem assembly[END_REF][START_REF] Smits | Long-term after-effects of fertilisation on the restoration of calcareous grasslands[END_REF]). On the former orchard, weed seed load in the seed bank and nutrients in the soil are therefore greater, resulting in high average vegetation height and cover (Jaunatre et al 2014a). This may have had a negative effect on the germination and growth of less competitive light-demanding grassland target species [START_REF] Hautier | Competition for light causes plant biodiversity loss after eutrophication[END_REF].

Seed bank studies help acquire a better understanding of ecosystem functioning and thus help restorationists plan better transfer timing. However, they do not provide a clear prediction of restoration success, since they largely underestimate the true seed density in the seed bank due to detection probabilities, dormancy and germination niches (De [START_REF] Villiers | The restoration of Strandveld and Succulent Karoo degraded by mining: an enumeration of topsoil seed banks[END_REF]Fowler et al. 2005). The outcome for bulk soil transfers in Mediterranean ecosystem restoration depends on several factors: transferred soil depth, whether the soil is stockpiled and whether soil horizons are reconstituted [START_REF] Koch | Alcoa's mining and restoration process in South Western Australia[END_REF][START_REF] Coiffait-Gombault | Hay transfer promotes establishment of Mediterranean steppe vegetation on soil disturbed by pipeline construction[END_REF][START_REF] Bulot | comparison of different soil transfer strategies for restoring a Mediterranean steppe after a pipeline leak (La Crau plain, South-Eastern France)[END_REF], the proportion of annual species in the donor site seed bank, and the fertility and seed bank of the receptor site (Jaunatre et al 2014a;[START_REF] Fowler | Evaluating restoration potential of transferred topsoil[END_REF]. Restoration success is also highly dependent on appropriate scheduling of the various tasks, although in practice timing is also governed by worksite imperatives. More importantly, restoration implies taking into account a set of environmental parameters not at play in seed bank studies. The patterns of some of these parameters can partly be foreseen on worksites (e.g. sites scheduled for restoration may be colonized by arable weeds germinating from the seed bank dating back to former agricultural activities; [START_REF] Buisson | Creation of the natural reserve of La Crau: Implications for the creation and management of protected areas[END_REF]Jaunatre et al. 2014a). However, this does not necessarily mean that they can be dealt with in advance. Ecological restoration will therefore partly remain a trial and error process and restored sites will require subsequent adaptive management.

Figure 2

 2 Figure 2. a. Species richness, b. seed density density and c. normalized Community structure Integrity Index of the soil seed bank of Peau de Meau, Mediterranean dry grassland (reference grassland), at two seasons (spring: early spring and summer: end of summer).

Figure 3 .

 3 Figure 3. Soil seed bank ordination of samples from Peau de Meau, Mediterranean dry grassland (reference grassland) gathered at two seasons (spring: early spring and summer: end of summer) based on RDA of plant community composition. For clarity, only the 16 most correlated species are shown (out of 38).

Figure 4

 4 Figure 4. a. Species richness of the areas restored using topsoil transfers from Mediterranean dry grassland and; b. CSIInorm, normalized Community Structure Integrity Index with reference grassland vegetation monitoring as reference compared with restored communities. Soil gathering season (spring in pink; Aut.: autumn in green), transfer ratio (1:3 light colors; 1:1 dark colors) and receptor sites (Former Orchard; Pipeline Leak site). Error bars represent ±SE, horizontal bars represent pairwise comparisons (NS: p>0.05; *: p<0.05; ***: p<0.001).

Figure 5 .

 5 Figure 5. Plant community ordination one year after different topsoil transfers from Mediterranean dry grassland for different soil gathering seasons (spring in pink; autumn in green), transfer ratios (1:3 light colors; 1:1 dark colors) and receptor sites (Former Orchard; Pipeline Leak site) based on NMDS of plant community composition, final stress=0.16. For clarity, only the 19 most correlated species are shown (out of 99).

Figure 6 .

 6 Figure 6. CSIInorm, normalized Community Structure Integrity Index with reference grassland vegetation monitoring as reference compared with communities restored using topsoil transfers from Mediterranean dry grassland and interactions with years (1 to 3 years after transfers, from dark to light shades). Soil gathering seasons

  Figure 1.

Figure 1 .

 1 Figure 1. Map of the donor and restored (receptor) sites and as well as of the seed bank study site. FO = former orchard and PL = pipeline leak site where dry grassland topsoil transfers were implemented. SeeTable 1 for details.

  Figure 2.

Figure 2

 2 Figure 2. a. Species richness, b. seed density and c. normalized Community structure Integrity Index of the soil seed bank of Peau de Meau, Mediterranean dry grassland (reference grassland), at two seasons (spring: early spring in pink and summer: end of summer in yellow). Error bars represent ±SE.

  Figure 3.

Figure 3 .

 3 Figure 3. Soil seed bank ordination of samples from Peau de Meau, Mediterranean dry grassland (reference grassland) gathered at two seasons (spring: early spring in pink and summer: end of summer in yellow) based on RDA of plant community composition. For clarity, only the 16 most correlated species are shown (out of 38).

  Figure 4.

Figure 4

 4 Figure 4. a. Species richness of the areas restored using topsoil transfers from Mediterranean dry grassland and; b. CSIInorm, normalized Community Structure Integrity Index with reference grassland vegetation monitoring as reference compared with restored communities. Soil gathering seasons (spring in pink; Aut.: autumn in green), transfer ratios (1:3 light colors; 1:1 dark colors) and receptor sites (Former Orchard; Pipeline Leak site). Error bars represent ±SE, horizontal bars represent pairwise comparisons (NS: p>0.05; *: p<0.05; ***: p<0.001).

  Figure 5.

Figure 5 .

 5 Figure 5. Plant community ordination one year after different topsoil transfers from Mediterranean dry grassland for different soil gathering seasons (spring in pink; autumn in green), transfer ratios (1:3 light colors; 1:1 dark colors) and receptor sites (Former Orchard; Pipeline Leak site) based on NMDS of plant community composition, final stress=0.16. For clarity, only the 19 most correlated species are shown (out of 99).

  Figure 6.

Figure 6 .

 6 Figure 6. CSIInorm, normalized Community Structure Integrity Index with reference grassland vegetation monitoring as reference compared with communities restored using topsoil transfers from Mediterranean dry grassland and interactions with years (1 to 3 years after transfers, from dark to light shades). Soil gathering seasons (spring in pink; autumn in green), transfer ratios (1:3 light colors; 1:1 dark colors) and receptor sites (Former Orchard; Pipeline Leak site). Error bars represent ±SE, bars having a common letter are not significantly different (Tukey honest significant differences post-hoc tests after ANOVA on repeated measures; p>0.05).

Table 1 .

 1 Dry grassland seed bank study and topsoil transfer details, showing protocols as well as subsequent vegetation monitoring. FO stands for former orchard and PL for pipeline leak site.

		reference vegetation	seed bank and season	topsoil transfer monitoring	objectives
	sites	3 sites of dry grassland	1 site of dry grassland (Peau de Meau; 43°34'01.85"N, 4°50'07.12"E)	2 sites (FO, PL), see Table 1 for details	to determine the best time to transfer topsoil in Mediterranean grasslands
				Aut. 1:1 -FO: transfer
				from Ménudelle 1:1 (3
				× 0.1 ha) in autumn:
			Seed bank: 10	three 4-m² plots
			2.5-L samples	Aut. 1:3 -FO: transfer
			(pooled	from Ménudelle 1:3 (3
			subsamples)	× 1 ha) in autumn:
	monitoring	6 × 4m² plots / site	taken between 0 and 20 cm deep, 2 seasons (April, August) Vegetation: 10 × 4m² plots	eighteen 4-m² plots Spring 1:3 -FO: transfer from Massilia 1:3 (1 × 1 ha) in spring: three 4-m² plots Spring 1:1 -PL (1 × 5.5 ha) in spring: nine	to determine the best topsoil transfer treatment (ratio-season combination)
				4-m² plots
			germination
		vegetation sampling, BB index (total = 18 plots)	identification (presence / and count, vegetation sampling	vegetation sampling, BB index
			absence)
	references	Jaunatre et al. 2014a	Römermann et al. 2005 and unpublished	Jaunatre et al. 2014a,b; Bulot et al. 2017 and unpublished

Table 2 .

 2 Topsoil transfer data used to determine the best topsoil transfer treatment (ratio-season combination). Coordinates of and distance between donor and restored (receptor) sites. Area covered (ha), ratio and season are indicated for each transfer. Map of the donor and restored (receptor) sites and as well as of the seed bank study site. FO = former orchard and PL = pipeline leak site where dry grassland topsoil transfers were implemented. See Table1for details.

					distance (km)	Surface area
	donor sites	coordinates	restored (receptor) sites	coordinates	between donor and restored	(ha) covered by topsoil	ratio season	month
					sites	transfer
	Massilia	43°29'03.61"N , 4°53'16.56"E				1 × 1 ha	1:3	spring	March 2009
			Former orchard	43°31'07.38"N, 4°51'35.05"E	3.5	3 × 1 ha	1:3	autum n	Oct. 2009
	Ménudell e	43°30'30.22"N , 4°54'03.14"E				3 × 0.1 ha	1:1	autum n	Oct. 2009
			Pipeline leak site	43°31′36.77′′N, 4°53′04.50′′E	2.5	1 × 5.5 ha	1:1	spring	March-April 2011

Acknowledgments

This study was supported by CDC Biodiversité, the Conseil Régional de Provence Alpes Côtes d'Azur and CNRS RTP Ingecotech funding. We are also grateful to CEN PACA Ecomusée de Crau, Réserve Naturelle des Coussouls de Crau, and Société des Carrières de La Ménudelle for their collaboration. The authors would like to thank M. Sweetko for reviewing the English, and N. Hölzel and an anonymous reviewer, as well as Coordinating Editor A. Helm, for improving previous versions.