Malik Boudiaf

Éts Montreal

Hoel Kervadec

Ziko Imtiaz Masud

Pablo Piantanida

Ismail Ben

Ayed Éts Montreal

Jose Dolz

Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need?

We show that the way inference is performed in few-shot segmentation tasks has a substantial effect on performances-an aspect often overlooked in the literature in favor of the meta-learning paradigm. We introduce a transductive inference for a given query image, leveraging the statistics of its unlabeled pixels, by optimizing a new loss containing three complementary terms: i) the crossentropy on the labeled support pixels; ii) the Shannon entropy of the posteriors on the unlabeled query-image pixels; and iii) a global KL-divergence regularizer based on the proportion of the predicted foreground. As our inference uses a simple linear classifier of the extracted features, its computational load is comparable to inductive inference and can be used on top of any base training. Foregoing episodic training and using only standard cross-entropy training on the base classes, our inference yields competitive performances on standard benchmarks in the 1-shot scenarios. As the number of available shots increases, the gap in performances widens: on PASCAL-5 i , our method brings about 5% and 6% improvements over the state-ofthe-art, in the 5-and 10-shot scenarios, respectively. Furthermore, we introduce a new setting that includes domain shifts, where the base and novel classes are drawn from different datasets. Our method achieves the best performances in this more realistic setting. Our code is freely available online: https://github.com/mboudiaf/ RePRI-for-Few-Shot-Segmentation.

Introduction

Few-shot learning, which aims at classifying instances from unseen classes given only a handful of training ex-⇤ Corresponding author: malik.boudiaf.1@etsmtl.net amples, has witnessed a rapid progress in the recent years. To quickly adapt to novel classes, there has been a substantial focus on the meta-learning (or learning-to-learn) paradigm [START_REF] Ren | Meta-learning for semi-supervised few-shot classification[END_REF][START_REF] Snell | Prototypical networks for few-shot learning[END_REF][START_REF] Vinyals | Matching networks for one shot learning[END_REF]. Meta-learning approaches popularized the need of structuring the training data into episodes, thereby simulating the tasks that will be presented at inference. Nevertheless, despite the achieved improvements, several recent image classification works [2, [START_REF] Chen | A closer look at few-shot classification[END_REF][START_REF] Singh Dhillon | A baseline for fewshot image classification[END_REF][START_REF] Guo | A broader study of cross-domain few-shot learning[END_REF][START_REF] Tian | Rethinking few-shot image classification: a good embedding is all you need?[END_REF][START_REF] Imtiaz | Laplacian regularized few-shot learning[END_REF] observed that meta-learning might have limited generalization capacity beyond the standard 1-or 5-shot classification benchmarks. For instance, in more realistic settings with domain shifts, simple classification baselines may outperform much more complex meta-learning methods [START_REF] Chen | A closer look at few-shot classification[END_REF][START_REF] Guo | A broader study of cross-domain few-shot learning[END_REF].

Deep-learning based semantic segmentation has been generally nurtured from the methodological advances in image classification. Few-shot segmentation, which has gained popularity recently [START_REF] Gairola | SimPropNet: Improved similarity propagation for few-shot image segmentation[END_REF][START_REF] Li | FSS-1000: A 1000-class dataset for few-shot segmentation[END_REF][START_REF] Liu | CRNet: Cross-reference networks for few-shot segmentation[END_REF][START_REF] Nguyen | Feature weighting and boosting for few-shot segmentation[END_REF][START_REF] Rakelly | Conditional networks for few-shot semantic segmentation[END_REF][START_REF] Tian | Prior guided feature enrichment network for few-shot segmentation[END_REF][START_REF] Wang | Few-shot semantic segmentation with democratic attention networks[END_REF][START_REF] Wang | PANet: Few-shot image semantic segmentation with prototype alignment[END_REF][START_REF] Yang | Prototype mixture models for few-shot semantic segmentation[END_REF][START_REF] Yang | A new local transformation module for few-shot segmentation[END_REF][START_REF] Zhang | CANet: Class-agnostic segmentation networks with iterative refinement and attentive fewshot learning[END_REF]42], is no exception. In this setting, a deep segmentation model is first pre-trained on base classes. Then, model generalization is assessed over few-shot tasks and novel classes unseen during base training. Each task includes an unlabeled test image, referred to as the query, along with a few labeled images (the support set). The recent literature in few-shot segmentation follows the learning-tolearn paradigm, and substantial research efforts focused on the design of specialized architectures and episodic-training schemes for base training. However, i) episodic training itself implicitly assumes that testing tasks have a structure (e.g., the number of support shots) similar to the tasks used at the meta-training stage; and ii) base and novel classes are often assumed to be sampled from the same dataset.

In practice, those assumptions may limit the applicability of the existing few-shot segmentation methods in realistic scenarios [START_REF] Cao | A theoretical analysis of the number of shots in few-shot learning[END_REF][START_REF] Chen | A closer look at few-shot classification[END_REF]. In fact, our experiments proved consistent with findings in few-shot classification when going beyond the standard settings and benchmarks. Particularly, we ob-served among state-of-the-art methods a saturation in performances [START_REF] Cao | A theoretical analysis of the number of shots in few-shot learning[END_REF] when increasing the number of labeled samples (See Table 3). Also, in line with very recent observations in image classification [START_REF] Chen | A closer look at few-shot classification[END_REF], existing meta-learning methods prove less competitive in cross-domains scenarios (See Table 4). This casts doubts as to the viability of the current few-shot segmentation benchmarks and datasets; and motivates re-considering the relevance of the meta-learning paradigm, which has become the de facto choice in the fewshot segmentation litterature.

Contributions

In this work, we forego meta-learning, and re-consider a simple cross-entropy supervision during training on the base classes for feature extraction. Additionally, we propose a transductive inference that better leverages the support-set supervision than the existing methods. Our contributions can be summarized as follows:

• We present a new transductive inference-RePRI (Region Proportion Regularized Inference)-for a given few-shot segmentation task. RePRI optimizes a loss integrating three complementary terms: i) a standard cross-entropy on the labeled pixels of the support images; ii) the entropy of the posteriors on the query pixels of the test image; and iii) a global KL divergence regularizer based on the proportion of the predicted foreground pixels within the test image. RePRI can be used on top of any trained feature extractor, and uses exactly the same information as standard inductive methods for a given few-shot segmentation task.

• Although we use a basic cross-entropy training on the base classes, without complex meta-learning schemes, RePRI yields highly competitive performances on the standard few-shot segmentation benchmarks, PASCAL-5 i and COCO-20 i , with gains around 5% and 6% over the state-of-the-art in the 5-and 10shot scenarios, respectively.

• We introduce a more realistic setting where, in addition to the usual shift on classes between training and testing data distributions, a shift on the images' feature distribution is also introduced. Our method achieves the best performances in this scenario.

• We demonstrate that a precise region-proportion information on the query object improves substantially the results, with an average gain of 13% on both datasets. While assuming the availability of such information is not realistic, we show that inexact estimates can still lead to drastic improvements, opening a very promising direction for future research.

Related Work

Few-Shot Learning for classification Meta-learning has become the de facto solution to learn novel tasks from a few labeled samples. Even though the idea is not new [28], it has been revived recently by several popular works in few-shot classification [START_REF] Finn | Model-agnostic meta-learning for fast adaptation of deep networks[END_REF][START_REF] Ravi | Optimization as a model for few-shot learning[END_REF][START_REF] Ren | Meta-learning for semi-supervised few-shot classification[END_REF][START_REF] Snell | Prototypical networks for few-shot learning[END_REF][START_REF] Vinyals | Matching networks for one shot learning[END_REF]. These works can be categorized into gradient-or metric-learning-based methods. Gradient approaches resort to stochastic gradient descent (SGD) to learn the commonalities among different tasks [START_REF] Ravi | Optimization as a model for few-shot learning[END_REF][START_REF] Finn | Model-agnostic meta-learning for fast adaptation of deep networks[END_REF]. Metric-learning approaches [START_REF] Vinyals | Matching networks for one shot learning[END_REF][START_REF] Snell | Prototypical networks for few-shot learning[END_REF] adopt deep networks as feature-embedding functions, and compare the distances between the embeddings. Furthermore, in a recent line of works, the transductive setting has been investigated for few-shot classification [START_REF] Singh Dhillon | A baseline for fewshot image classification[END_REF]2,[START_REF] Hou | Cross attention network for few-shot classification[END_REF][START_REF] Kim | Edge-labeling graph neural network for few-shot learning[END_REF][START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF][START_REF] Qiao | Transductive episodicwise adaptive metric for few-shot learning[END_REF][START_REF] Snell | Prototypical networks for few-shot learning[END_REF][START_REF] Imtiaz | Laplacian regularized few-shot learning[END_REF], and yielded performance improvements over inductive inference. These results are in line with established facts in classical transductive inference [START_REF] Vladimir | An overview of statistical learning theory[END_REF]15,[START_REF] Dengyong | Learning with local and global consistency[END_REF], well-known to outperform its inductive counterpart on small training sets.

To a large extent, these transductive classification works follow well-known concepts in semi-supervised learning, such as graph-based label propagation [START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF], entropy minimization [START_REF] Singh Dhillon | A baseline for fewshot image classification[END_REF] or Laplacian regularization [START_REF] Imtiaz | Laplacian regularized few-shot learning[END_REF]. While the entropy is a part of our transductive loss, we show that it is not sufficient for segmentation tasks, typically yielding trivial solutions.

Few-shot segmentation Segmentation can be viewed as a classification at the pixel level, and recent efforts mostly went into the design of specialized architectures. Typically, the existing methods use a two-branch comparison framework, inspired from the very popular prototypical networks for few-shot classification [START_REF] Snell | Prototypical networks for few-shot learning[END_REF]. Particularly, the support images are employed to generate class prototypes, which are later used to segment the query images via a prototypequery comparison module. Early frameworks followed a dual-branch architecture, with two independent branches [START_REF] Shaban | One-shot learning for semantic segmentation[END_REF][START_REF] Dong | Few-shot semantic segmentation with prototype learning[END_REF][START_REF] Rakelly | Conditional networks for few-shot semantic segmentation[END_REF], one generating the prototypes from the support images and the other segmenting the query images with the learned prototypes. More recently, the dual-branch setting has been unified into a single-branch, employing the same embedding function for both the support and query sets [42,[START_REF] Siam | AMP: Adaptive masked proxies for fewshot segmentation[END_REF][START_REF] Wang | PANet: Few-shot image semantic segmentation with prototype alignment[END_REF][START_REF] Yang | Prototype mixture models for few-shot semantic segmentation[END_REF][START_REF] Liu | Part-aware prototype network for fewshot semantic segmentation[END_REF]. These approaches mainly aim at exploiting better guidance for the segmentation of query images [42,[START_REF] Nguyen | Feature weighting and boosting for few-shot segmentation[END_REF][START_REF] Wang | Few-shot semantic segmentation with democratic attention networks[END_REF][START_REF] Zhang | Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation[END_REF], by learning better class-specific representations [START_REF] Wang | PANet: Few-shot image semantic segmentation with prototype alignment[END_REF][START_REF] Liu | CRNet: Cross-reference networks for few-shot segmentation[END_REF][START_REF] Liu | Part-aware prototype network for fewshot semantic segmentation[END_REF][START_REF] Yang | Prototype mixture models for few-shot semantic segmentation[END_REF][START_REF] Siam | AMP: Adaptive masked proxies for fewshot segmentation[END_REF] or iteratively refining these [START_REF] Zhang | CANet: Class-agnostic segmentation networks with iterative refinement and attentive fewshot learning[END_REF]. Graph CNNs have also been employed to establish more robust correspondences between the support and query images, enhancing the learned prototypes [START_REF] Wang | Few-shot semantic segmentation with democratic attention networks[END_REF]. Alternative solutions to learn better class representations include: imprinting the weights for novel classes [START_REF] Siam | AMP: Adaptive masked proxies for fewshot segmentation[END_REF], decomposing the holistic class representation into a set of part-aware prototypes [START_REF] Liu | Part-aware prototype network for fewshot semantic segmentation[END_REF] or mixing several prototypes, each corresponding to diverse image regions [START_REF] Yang | Prototype mixture models for few-shot semantic segmentation[END_REF]. 3. Formulation

Few-shot Setting

Formally, we define a base dataset D base with base semantic classes Y base , employed for training. Specifically, D base = {(x n , y n )} N n=1 , ⌦ ⇢ R 2 an image space, x n : ⌦ ! R 3 an input image, and y n : ⌦ ! {0, 1} |Ybase| its corresponding pixelwise one-hot annotation. At inference, we test our model through a series of K-shots tasks. Each K-shots task consists of a support set S = {(x k , y k )} K k=1 , i.e. K fully annotated images, and one unlabelled query image x Q , all from the same novel class. This class is randomly sampled from a set of novel classes Y novel such that Y base \ Y novel = ;. The goal is to leverage the supervision provided by the support set in order to properly segment the object of interest in the query image.

Base training Inductive bias in episodic training

There exist different ways of leveraging the base set D base . Meta-learning, or learning to learn, is the dominant paradigm in the few-shot literature. It emulates the test-time scenario during training by structuring D base into a series of training tasks. Then, the model is trained on these tasks to learn how to best leverage the supervision from the support set in order to enhance its query segmentation. Recently, Cao et al. [START_REF] Cao | A theoretical analysis of the number of shots in few-shot learning[END_REF] formally proved that the number of shots K train used in train-ing episodes in the case of prototypical networks represents a learning bias, and that the testing performance saturates quickly when K test differs from K train . Empirically, we observed the same trend for current few-shot segmentation methods, with minor improvements from 1-shot to 5-shot performances (Table 1).

Standard training

In practice, the format of the test tasks may be unknown beforehand. Therefore, we want to take as little assumptions as possible on this. This motivates us to employ a feature extractor f trained with standard crossentropy supervision on the whole D base set instead, without resorting to episodic training.

Inference

Objective In what follows, we use ⇧ as a placeholder to denote either a support subscript k 2 {1, ..., K} or the query subscript Q. At inference, we consider the 1-way segmentation problem: y ⇧ : ⌦ ! {0, 1} 2 is the function representing the dense background/foreground (B/F) mask in image x ⇧ . For both support and query images, we extract features z ⇧ := f (x ⇧ ) and z ⇧ :

! R C , where C is the channel dimension in the feature space , with lower pixel resolution | | < |⌦|.

Using features z ⇧ , our goal is to learn the parameters ✓ of a classifier that properly discriminates foreground from background pixels. Precisely, our classifier p ⇧ : ! [0, 1] 2 assigns a (B/F) probability vector to each pixel j 2 in the extracted feature space.

For each test task, we find the parameters ✓ of the classifier by optimizing the following transductive objective:

min ✓ CE + H H + KL D KL , (1)
where H , KL 2 R are non-negative hyper-parameters balancing the effects of the different terms.

We now describe in details each of the terms in Eq. ( 1):

CE = 1 K| | K X k=1 X j2 e y k (j) > log(p k (j))
is the cross-entropy between the downsampled labels e y k from support images and our classifier's soft predictions. Simply minimizing this term will often lead to degenerate solutions, especially in the 1-shot setting, as observed in Figure 1-the classifier ✓ typically overfits the support set S, translating into small activated regions on the query image.

H = 1 | | X j2 p Q (j) > log (p Q (j))
is the Shannon entropy of the predictions on the queryimage pixels. The role of this entropy term is to make the model's predictions more confident on the query image. The use of H originates from the semi-supervised literature [START_REF] Grandvalet | Semisupervised learning by entropy minimization[END_REF][START_REF] Miyato | Virtual adversarial training: a regularization method for supervised and semi-supervised learning[END_REF][START_REF] Berthelot | Mixmatch: A holistic approach to semi-supervised learning[END_REF]. Intuitively, it pushes the decision boundary drawn by the linear classifier towards low-density regions of the extracted query feature space. While this term plays a crucial role in conserving object regions that were initially predicted with only medium confidence, its sole addition to CE does not solve the problem of degenerate solutions, and may even worsen it in some cases.

D KL = b p > Q log ✓ b p Q ⇡ ◆ , with b p Q = 1 | | P j2 p Q (j)
, is a Kullback-Leibler (KL) Divergence term that encourages the B/F proportion predicted by the model to match a parameter ⇡ 2 [0, 1] 2 . Notice that the division inside the log applies element-wise. The joint estimation of parameter ⇡ in our context is further discussed in a following paragraph. Here, we argue that this term plays a key role in our loss. First, in the case where parameter ⇡ does not match the exact B/F proportion of the query image, this term still helps avoiding the degenerate solutions stemming from CE and H minimization. Second, should an accurate estimate of the B/F proportion in the query image be available, it could easily be embedded through this term, resulting in a substantial performance boost, as discussed in Section 4.

Choice of the classifier As we optimize ✓ for each task at inference, we want our method to add as little computational load as possible. In this regard, we employ a simple linear classifier with learnable parameters

✓ (t) = {w (t) , b (t)
}, with t the current step of the optimization procedure, w (t) 2 R C the foreground prototype and b (t) 2 R the corresponding bias. Thus, the probabilities p (t) k and p (t) Q at iteration t, for pixel j 2 can be obtained as follow:

p (t) ⇧ (j) := 1 s (t) ⇧ (j) s (t) ⇧ (j) ! , (2)
where s

(t) ⇧ (j) = sigmoid ⌧ ⇥ cos z ⇧ (j), w (t) b (t) ⇤ , ⌧ 2
R is a temperature hyper-parameter and cos the cosine similarity. The same classifier is used to estimate the support set probabilities p k and the query predicted probabilities p Q . At initialization, we set prototype w (0) to be the average of the foreground support features, i.e. w (0) = 1

K| | P K k=1 P j2 e y k (j) 1 z k (j)
, with e y k (j) 1 the foreground component of the one-hot label of image x k at pixel j. Initial bias b (0) is set as the mean of the foreground's soft predictions on the query image: b

(0) = 1 | | P j2 p Q (j) 1 .
Then, w (t) and b (t) are optimized with gradient descent. The computational footprint of this pertask optimization is discussed in Section 4.

Joint estimation of B/F proportion ⇡ Without additional information, we leverage the model's label-marginal distribution over the query image b p

(t)
Q in order to learn ⇡ jointly with classifier parameters. Note that minimizing Eq. ( 1) with respect to ⇡ yields

⇡ (t) = b p (t)
Q . Empirically, we found that, after initialization, updating ⇡ only once during optimization, at a later iteration, t ⇡ was enough:

⇡ (t) = 8 < : b p (0) Q 0  t  t ⇡ b p (t⇡) Q t > t ⇡ . (3)
Intuitively, the entropy term H helps gradually refine initially blurry soft predictions (third column in Fig. 1), which turns b p

(t)
Q into an improving estimate of the true B/F proportion. A quantitative study of this phenomenon is provided in Section 4.3. Therefore, our inference can be seen as a joint optimization over ✓ and ⇡, with D KL serving as a selfregularization that prevents the model's marginal distribution b p

(t) Q from diverging.
Oracle case with a known ⇡ As an upper bound, we also investigate the oracle case, where we have access to the true B/F proportion in x Q :

⇡ ⇤ = 1 | | X j2 e y Q (j). (4)

Experiments

Datasets We resort to two public few-shot segmentation benchmarks, PASCAL-5 i and COCO-20 i , to evaluate our method. PASCAL-5 i is built from PASCALVOC 2012 [8], and contains 20 object categories split into 4 folds. For each fold, 15 classes are used for training and the remaining 5 categories for testing. COCO-20 i is built from MS-COCO [START_REF] Lin | Microsoft coco: Common objects in context[END_REF] and is more challenging, as it contains more samples, more classes and more instances per image. Similar to PASCAL-5 i , COCO-20 i dataset is divided into 4folds with 60 base classes and 20 test classes in each fold.

Training We build our model based on PSPNet [START_REF] Zhao | Pyramid scene parsing network[END_REF] with Resnet-50 and Resnet-101 [START_REF] He | Deep residual learning for image recognition[END_REF] as backbones. We train the feature extractor with standard cross-entropy over the base classes during 100 epochs on PASCAL-5 i , and 20 epochs on COCO-20 i , with batch size set to 12. We use SGD as optimizer with the initial learning rate set to 2.5e 3 and we use cosine decay. Momentum is set to 0.9, and weight decay to 1e 4. Label smoothing is used with smoothing parameter ✏ = 0.1. We did not use multi-scaling, nor deep supervision, unlike the original PSPNet paper [START_REF] Zhao | Pyramid scene parsing network[END_REF]. As for data augmentations, we only use random mirror flipping.

Inference At inference, following previous works [START_REF] Liu | Part-aware prototype network for fewshot semantic segmentation[END_REF][START_REF] Wang | PANet: Few-shot image semantic segmentation with prototype alignment[END_REF], all images are resized to a fixed 417 ⇥ 417 resolution. For each task, the classifier ✓ is built on top of the features from the penultimate layer of the trained network.

For our model with ResNet-50 as backbone, this results in a 53 ⇥ 53 ⇥ 512 feature map. SGD optimizer is used to train ✓, with a learning rate of 0.025. For each task, a total of 50 iterations are performed. The parameter t ⇡ is set to 10. For the main method, the weights H and KL are both initially set to 1/K, such that the CE term plays a more important role as the number of shots K grows. For t t ⇡ , KL is increased by 1 to further encourage the predicted proportion close to ⇡ (t⇡) . Finally, the temperature ⌧ is set to 20.

Evaluation We employ the widely adopted mean Intersection over Union (mIoU). Specifically, for each class, the classwise-IoU is computed as the sum over all samples within the class of the intersection over the sum of all unions. Then, the mIoU is computed as the average over all classes of the classwise-IoU. Following previous works [START_REF] Liu | Part-aware prototype network for fewshot semantic segmentation[END_REF], 5 runs of 1000 tasks each are computed for each fold, and the average mIoU over runs is reported.

Benchmark results

Main method First, we investigate the performance of the proposed method in the popular 1-shot and 5-shot settings on both PASCAL-5 i and COCO-20 i , whose results are reported in Table 1 and 2. Overall, we found that our method compares competitively with state-of-the-art approaches in the 1-shot setting, and significantly outperforms recent methods in the 5-shot scenario. Additional qualitative results on PASCAL-5 i are shown in the supplemental material.

Beyond 5-shots In the popular learning-to-learn paradigm, the number of shots leveraged during the metatraining stage has a direct impact on the performance at inference [START_REF] Cao | A theoretical analysis of the number of shots in few-shot learning[END_REF]. Particularly, to achieve the best performance, meta-learning based methods typically require the numbers of shots used during meta-training to match those employed during meta-testing. To demonstrate that the proposed method is more robust against differences on the number of labeled support samples between the base and test sets, we further investigate the 10-shot scenario. Particularly, we trained the methods in [START_REF] Tian | Prior guided feature enrichment network for few-shot segmentation[END_REF][START_REF] Yang | Prototype mixture models for few-shot semantic segmentation[END_REF] by using one labeled sample per class, i.e., 1-shot task, and test the models on a 10-shots task. Interestingly, we show that the gap between our method and current state-of-the-art becomes larger as the number of support images increases (Table 3), with significant gains of 6% and 4% on PASCAL-5 i and COCO-20 i , respectively. These results suggest that our transductive inference leverages more effectively the information conveyed in the labeled support set of a given task.

Oracle results

We now investigate the ideal scenario where an oracle provides the exact foreground/background proportion in the query image, such that ⇡ (t) = ⇡ ⇤ , 8t. Reported results in this scenario, referred to as Oracle (Table 1 and 2) show impressive improvements over both our current method and all previous works, with a consistent gain across datasets and tasks. Particularly, these values range from 11% and 14 % on both PASCAL-5 i and COCO-20 i and in both 1-shot and 5-shot settings. We believe that these findings convey two important messages. First, it proves that there exists a simple linear classifier that can largely outperform state-of-the-art meta-learning models, while being built on top of a feature extractor trained with a standard cross-entropy loss. Second, these results indicate that having a precise size of the query object of interest acts as a strong regularizer. This suggests that more efforts could be directed towards properly constraining the optimization process of w and b, and opens a door to promising avenues.

Domain shift

We introduce a more realistic, cross-domain setting (COCO-20 i to PASCAL-VOC). We argue that such setting is a step towards a more realistic evaluation of these methods, as it can assess the impact on performances caused by a domain shift between the data training distribution and the testing one. We believe that this scenario can be easily found in practice, as even slight alterations in the data collection process might result in a distributional shift. We Table 1: Results of 1-way 1-shot and 1-way 5-shot segmentation on PASCAL-5 i using the mean-IoU. Best results in bold. reproduce the scenario where a large labeled dataset is available (e.g., COCO-20 i ), but the evaluation is performed on a target dataset with a different feature distribution (e.g., PASCAL-VOC). As per the original work [START_REF] Lin | Microsoft coco: Common objects in context[END_REF], significant differences exist between the two original datasets. For instance, images in MS-COCO have on average 7.7 instances of objects coming from 3.5 distinct categories, while PASCAL-VOC only has an average of 3 instances from 2 distinct categories.

Evaluation We reuse models trained on each fold of COCO-20 i and generate tasks using images from all the classes in PASCAL-VOC that were not used during train- ing. For instance, fold-0 of this setting means the model was trained on fold-0 of COCO-20 i and tested on the whole PASCAL-VOC dataset, after removing the classes seen in training. A complete summary of all the folds is available in the Supplemental material.

Results

We reproduced and compared to the two best performing methods [START_REF] Tian | Prior guided feature enrichment network for few-shot segmentation[END_REF][START_REF] Liu | Part-aware prototype network for fewshot semantic segmentation[END_REF] using their respective official GitHub repositories. Table 4 summarizes the results for the 1-shot and 5-shot cross-domain experiments. We observe that in the presence of domain-shift, our method outperforms existing methods in both 1-shot and 5-shot scenarios, with again the improvement jumping from 2% in 1-shot to 4% in 5-shot.

Ablation studies

Impact of each term in the main objective While Fig. 1 provides qualitative insights on how each term in Eq. ( 1) affects the final prediction, this section provides a quantitative evaluation of their impact, evaluated on PASCAL-5 i (Table 5). Quantitative results confirm the qualitative insights observed in Fig. 1, as both CE and CE + H losses drastically degrade the performance compared to the proportionregularized loss, i.e., CE + H + D KL . For example, in the 1-shot scenario, simply minimizing the CE results in more than 20% of difference compared to the proposed model. In this case, the prototype w tends to overfit the support sample and only activates regions of the query object that strongly correlate with the support object. Such behavior hampers the performance when the support and query objects exhibit slights changes in shape or histogram colors, for example, which may be very common in practice.

Adding the entropy term H to CE partially alleviates this problem, as it tends to reinforce the model in being confident on positive pixels initially classified with mid or low confidence. Nevertheless, despite improving the naive CE based model, the gap with the proposed model remains considerably large, with 10% difference. One may notice that the differences between CE and CE + H + D KL decrease in the 5-shot setting, since overfitting 5 support samples si-multaneously becomes more difficult. The results from this ablation experiment reinforce our initial hypothesis that the proposed KL term based on the size parameter ⇡ acts as a strong regularizer.

Influence of the parameter t ⇡ In Fig. 2, we plot the averaged mIoU (over 4 folds) as a function of t ⇡ varying over the full range t ⇡ 2 [START_REF] Berthelot | Mixmatch: A holistic approach to semi-supervised learning[END_REF]50]. For 5-shot, the performances are stable and remain largely above SOTA for all t ⇡ . As for the 1-shot case, the range [START_REF] Dengyong | Learning with local and global consistency[END_REF]15] yields roughly similar results.

While selection of optimal t ⇡ would lead to performance gains in each setting, in the paper, we used a single value of t ⇡ = 10 for all the settings. Influence of parameter ⇡ misestimation Precisely knowing the foreground/background (B/F) proportion of the query object is unrealistic. To quantify the deviation from the exact B/F proportion ⇡ ⇤ , we introduce the relative error on the foreground size:

(t) = ⇡ (t) 1 ⇡ ⇤ 1 1, (5)
where ⇡ ⇤ 1 represents the exact foreground proportion in the query image, extracted from its corresponding ground truth, and ⇡ (t)

1 our estimate at iteration t, which is derived from the soft predicted segmentation. As observed from Fig. 1, the initial prototype often results in a blurred probability map, from which only a very coarse estimate of the query proportion can be inferred and used as ⇡ (0) . The distribution of over 5000 tasks is presented in Fig. 3a. It clearly shows that the initial prediction typically provides an overestimate of the actual query foreground size, while finetuning the classifier ✓ for 10 iterations with our main loss (Eq. 1) already provides a strictly more accurate estimate, as conveyed by the right box plot in Fig. 3a, with an average around 0.7. Now, a natural question remains: how good does the estimate need to be in order to approach the oracle results? To answer this, we carry out a series of controlled 0) and after 10 gradient iterations (10) .

(b) Mean-IoU versus enforced relative foreground size error in the parameter ⇡ (0) .

Figure 3: Experiments on ⇡ misestimation. Both figures are computed using 5 runs of 1000 1-shot tasks, each on the fold-0 of PASCAL5 i . experiments where, instead of computing ⇡ (t) with Eq.

(3), we use a -perturbed oracle at initialization, such that ⇡

(t) 1 = ⇡ ⇤ 1 (1+ ).
Each point in Fig. 3b represents the mIoU obtained over 5000 tasks for a given perturbation . Fig. 3b reveals that exact B/F proportion is not required to significantly close the gap with the oracle. Specifically, foreground size estimates ranging from -10% to +30% with respect to the oracle proportion are sufficient to achieve 70%+ of mIoU, which represents an improvement of 10% over the current state-of-the art. This suggests that more refined size estimation methods may significantly increase the performance of the proposed method.

Computational efficiency We now inspect the computational cost of the proposed model, and compare to recent existing methods. Unlike prior work, we solve an optimization problem at inference, which naturally slows down the inference process. However, in our case, only a single prototype vector w 2 R C , where we recall C is the feature channel dimension, and a bias b 2 R need to be optimized for each task. Furthermore, in our setting C = 512, and therefore the problem can still be solved relatively efficiently, leading to reasonable inference times. In Table 6, we summarize the FPS rate at inference for our method, as well as for two competing approaches that only require a forward pass. We can observe that, unsurprisingly, our method reports lower FPS rates, without becoming unacceptably slower. The reported values indicate that the differences in inference times are small compared to, for example, the approach in [START_REF] Tian | Prior guided feature enrichment network for few-shot segmentation[END_REF]. Particularly, in the 1-shot scenario, our method processes tasks 3 FPS slower than [START_REF] Tian | Prior guided feature enrichment network for few-shot segmentation[END_REF], whereas this gap narrows down to 0.7 FPS in the 5-shot setting.

Conclusion

Without resorting to the popular meta-learning paradigm, our proposed RePRI achieves new state-of-theart results on standard 5-shot segmentation benchmarks, while being close to best performing approaches in the 1-shot setting. RePRI is modular and can, therefore, be used in conjunction with any feature extractor regardless how the base training was performed. Supported by the findings in this work, we believe that the relevance of the episodic training should be re-considered in the context of few-shot segmentation, and we provide a strong baseline to stimulate future research on this topic. Our results indicate that current state-of-the-art methods may have difficulty with more challenging settings, when dealing with domain shift or conducting inference on tasks whose structures are different from those seen in trainingscenarios that have been overlooked in the literature. These findings align with recent observations in few shot classification [START_REF] Chen | A closer look at few-shot classification[END_REF][START_REF] Cao | A theoretical analysis of the number of shots in few-shot learning[END_REF]. Furthermore, embedding more accurate foreground-background proportion estimates appears to be a very promising way of constraining the inference, as demonstrated with the significantly improved results obtained by the oracle. Our implementation is publicly available online: https://github.com/mboudiaf/ RePRI-for-Few-Shot-Segmentation.

Figure 1 :

 1 Figure 1: Probability maps for several 1-shot tasks. For each task, the two first columns show the ground truth of support and query. Initial column represents the probability map with the initial classifier ✓ (0) , and the last three columns show the final soft predicted segmentation after finetuning with each of the three losses. Best viewed in colors.

Figure 2 :

 2 Figure 2: Average mIoU (over 4 folds) as a function of t ⇡ on PASCAL-5 i (left) and COCO-20 i (right) .

Table 2 :

 2 Results of 1-way 1-shot and 1-way 5-shot segmentation on COCO-20 i using mean-IoU metric. Best results in bold.

	1 shot	5 shot

* We report the results where no additional unlabeled data is employed.

Table 3 :

 3 Aggregated results for 1-way 1-, 5-and 10-shot tasks with Resnet50 as backbone and averaged over 4 folds. Per fold results are available in the supplementary material.

		PASCAL-5 i		COCO-20 i
	Method	1-S	5-S	10-S	1-S	5-S	10-S
	RPMM [38]	56.3 57.3 57.6 30.6 35.5 33.1
	PFENet [33]	60.8 61.9 62.1 35.8 39.0 39.7
	RePRI (our)	59.1 66.8 68.2 34.0 42.1 44.4
	Oracle-RePRI 73.3 77.9 78.6 45.1 55.5 58.7

Table 4 :

 4 Aggregated domain-shift results, averaged over 4 folds, on COCO-20 i to PASCAL-VOC. Best results in bold. Per-fold results are available in the supplementary material.

			COCO ! PASCAL
	Method	Backbone 1 shot	5 shot
	RPMM [38]		49.6	53.8
	PFENet [33]	ResNet50	61.1	63.4
	RePRI (ours)		63.2	67.7
	Oracle-RePRI Resnet-50	76.2	79.7

Table 5 :

 5 Ablation study on the effect of each term in our loss in Eq. (1), evaluated on PASCAL-5 i .

				1 shot					5 shot
	Loss	Fold-0 Fold-1 Fold-2 Fold-3 Mean Fold-0 Fold-1 Fold-2 Fold-3 Mean
	CE	39.7	49.3	37.3	27.5	38.5	56.5	66.4	60.1	49.0	58.0
	CE + H CE + H + DKL	45.7 60.2	61.7 67.0	48.2 61.7	36.4 47.5	48.0 59.1	56.8 64.5	68.5 70.8	61.3 71.7	47.0 60.3	58.4 66.8
	(a) Relative error distribution of
	our current method, at initializa-
	tion

Table 6 :

 6 Number of tasks performed per second, and the corresponding mIoU performances on PASCAL-5 i .

		1-shot	5-shot
	Method	FPS mIoU FPS mIoU
	RPMMS [38] 18.2	51.5	9.4	57.3
	PFENet [33]	15.9	60.8	5.1	61.9
	RePRI (ours)	12.8	59.1	4.4	66.8

Acknowledgement

This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 792464.

A. Domain shift experiment

In Table 7, we show the details of the cross-domain folds used for the domain-shift experiments. Also, in Table 8, the per-fold results of the same experiment are available.

B. Results of the 10-shot experiments

In Table 9, we give the per-fold results of the 10-shot experiments.

C. Qualitative results

In Figure 4, we provide some qualitative results on PASCAL-5 i that show how our method helps refining the initial predictions of the classifier.

Figure 4: Qualitative results on PASCAL 5 i . Initial column refers to the predictions right after initializing the prototypes, while Final column refers to the prediction after running our inference. Best viewed in colors in high resolution.