Michalis Lazarou

Yannis Avrithis

Tania Stathaki

Iterative label cleaning for transductive and semi-supervised few-shot learning

Few-shot learning amounts to learning representations and acquiring knowledge such that novel tasks may be solved with both supervision and data being limited. Improved performance is possible by transductive inference, where the entire test set is available concurrently, and semi-supervised learning, where more unlabeled data is available. These problems are closely related because there is little or no adaptation of the representation in novel tasks.

Focusing on these two settings, we introduce a new algorithm that leverages the manifold structure of the labeled and unlabeled data distribution to predict pseudo-labels, while balancing over classes and using the loss value distribution of a limited-capacity classifier to select the cleanest labels, iterately improving the quality of pseudo-labels. Our solution sets new state of the art on four benchmark datasets, namely miniImageNet, tieredImageNet, CUB and CIFAR-FS, while being robust over feature space pre-processing and the quantity of available data.

Introduction

Few-shot learning [START_REF] Vinyals | Matching networks for one shot learning[END_REF][START_REF] Snell | Prototypical networks for few-shot learning[END_REF] is challenging the deep learning paradigm in that, not only supervision is limited, but data is limited too. Despite the initial promise of metalearning [START_REF] Mishra | A simple neural attentive meta-learner[END_REF][START_REF] Finn | Modelagnostic meta-learning for fast adaptation of deep networks[END_REF], transfer learning [10,[START_REF] Tian | Rethinking few-shot image classification: a good embedding is all you need?[END_REF] is becoming increasingly successful in decoupling representation learning from learning novel tasks on limited data. Semi-supervised learning [START_REF] Lee | Pseudo-label: The simple and efficient semisupervised learning method for deep neural networks[END_REF]5] is one of the dominant ways of dealing with limited supervision and indeed, its few-shot learning counterparts [START_REF] Ren | Meta-learning for semi-supervised fewshot classification[END_REF][START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF] are miniature versions where both labeled and unlabeled are limited proportionally, while representation learning may be decoupled. In this sense, these methods are closer to transductive inference [START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF][START_REF] Pau Rodríguez | Embedding propagation: Smoother manifold for few-shot classification[END_REF], which was a pillar of semi-supervised learning before deep learning [8].

Predicting pseudo-labels on unlabeled data [START_REF] Lee | Pseudo-label: The simple and efficient semisupervised learning method for deep neural networks[END_REF] is one of the oldest ideas in semi-supervised learning [START_REF] Scudder | Probability of error of some adaptive patternrecognition machines[END_REF]. Graphbased methods, in particular label propagation [START_REF] Zhu | Learning from labeled and unlabeled data with label propagation[END_REF][START_REF] Zhou | Learning with local and global consistency[END_REF], are prominent in transductive inference and translate to inductive inference in deep learning exactly by predicting pseudo- labels [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF]. However, with the representation being fixed, the quality of pseudo-labels is critical in few-shot learning [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF]28]. At the same time, in learning with noisy labels [3, [START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF][START_REF] Song | Robust and on-the-fly dataset denoising for image classification[END_REF], it is common to clean labels based on the loss value statistics of a small-capacity classifier.

In this work, we leverage these ideas to improve transductive and semi-supervised few-shot learning. As shown in Figure 1, focusing on transduction, a set of labeled support examples S and unlabeled queries Q are given, represented in a feature space by mapping f . By label propagation [START_REF] Zhou | Learning with local and global consistency[END_REF], we obtain a matrix that associates examples to classes. The submatrix corresponding to unlabeled examples, P , is normalized over examples and classes using the Sinkhorn-Knopp algorithm [START_REF] Arazo | Unsupervised label noise modeling and loss correction[END_REF], assuming a uniform distribution over classes. We extract pseudo-labels from P , which we clean following O2U-Net [START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF], keeping only one example per class. Finally, inspired by [25], we move these examples from Q to S and iterate until Q is empty.

Related work and contributions

Few-shot learning

Meta-learning This is a popular paradigm, where the training set is partitioned in episodes resembling in struc-1 arXiv:2012.07962v1 [cs.LG] 14 Dec 2020 ture the novel tasks [START_REF] Finn | Modelagnostic meta-learning for fast adaptation of deep networks[END_REF][START_REF] Koch | Siamese neural networks for one-shot image recognition[END_REF][START_REF] Snell | Prototypical networks for few-shot learning[END_REF][START_REF] Mishra | A simple neural attentive meta-learner[END_REF][START_REF] Vinyals | Matching networks for one shot learning[END_REF]. Model-based methods rely on the properties of specific model architectures, such as recurrent and memory-augmented networks [START_REF] Mishra | A simple neural attentive meta-learner[END_REF]52,[START_REF] Munkhdalai | Meta networks[END_REF]. Optimization-based methods attempt to learn model parameters that are able to adapt fast in novel tasks [START_REF] Finn | Modelagnostic meta-learning for fast adaptation of deep networks[END_REF][START_REF] Rajeswaran | Meta-learning with implicit gradients[END_REF][START_REF] Zintgraf | Fast context adaptation via meta-learning[END_REF][START_REF] Ravi | Optimization as a model for few-shot learning[END_REF][START_REF] Nichol | On first-order meta-learning algorithms[END_REF][START_REF] Lee | Meta-learning with differentiable convex optimization[END_REF][START_REF] Bertinetto | Meta-learning with differentiable closed-form solvers[END_REF]. Metric-based methods attempt to learn representations that are appropriate for comparisons [START_REF] Snell | Prototypical networks for few-shot learning[END_REF][START_REF] Koch | Siamese neural networks for one-shot image recognition[END_REF][START_REF] Sung | Learning to compare: Relation network for few-shot learning[END_REF][START_REF] Vinyals | Matching networks for one shot learning[END_REF]. Of course, metric learning is a research area on its own [START_REF] Wang | Multi-similarity loss with general pair weighting for deep metric learning[END_REF][START_REF] Kim | Proxy anchor loss for deep metric learning[END_REF] and modern ideas are commonly effective in few-shot learning [START_REF] Liu | Negative margin matters: Understanding margin in few-shot classification[END_REF].

Predicting weights, data augmentation Also based on meta-learning, it is possible to predict new parameters or even data. For instance, it is common to learn to predict data-dependent network parameters in the last layer (classifier) [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF][START_REF] Qi | Low-shot learning with imprinted weights[END_REF]46] or even in intermediate convolutional layers [7]. Alternatively, one can learn to generate noveltask data in the feature space [START_REF] Hariharan | Low-shot visual recognition by shrinking and hallucinating features[END_REF] or in the input (image) space [START_REF] Wang | Low-shot learning from imaginary data[END_REF][START_REF] Qiao | Fewshot image recognition by predicting parameters from activations[END_REF]. The quality can be improved by translating images, similar to style transfer [34]. Such learned data augmentation is complementary to other ideas.

Transfer learning More recently, it is recognized that learning a powerful representation on the entire training set is more effective than sampling few-shot training episodes that resemble novel tasks [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF]10,[START_REF] Tian | Rethinking few-shot image classification: a good embedding is all you need?[END_REF][START_REF] Mangla | Charting the right manifold: Manifold mixup for fewshot learning[END_REF]. In doing so, one may use standard loss functions [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF]10], knowledge distillation [START_REF] Tian | Rethinking few-shot image classification: a good embedding is all you need?[END_REF] or other common self-supervision and regularization methods [START_REF] Mangla | Charting the right manifold: Manifold mixup for fewshot learning[END_REF]. We follow this transfer learning approach, which allows us to decouple representation learning from the core few-shot learning idea and provide clearer comparisons with the competition.

Using unlabeled data

Leveraging unlabelled data is of interest due to the ease of obtaining such data. Two common settings are transductive inference and semi-supervised learning.

Transductive inference In this setting, all novel-class unlabeled query examples are assumed available at the same time at inference [START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF][START_REF] Lichtenstein | TAFSSL: Task-adaptive feature sub-space learning for few-shot classification[END_REF][START_REF] Qiao | Transductive episodic-wise adaptive metric for few-shot learning[END_REF][START_REF] Hu | Exploiting unsupervised inputs for accurate few-shot classification[END_REF]19,[START_REF] Pau Rodríguez | Embedding propagation: Smoother manifold for few-shot classification[END_REF]. These examples give additional information on the distribution of novel classes on top of labeled support examples.

Common transductive inference solutions are adapted for few-shot classification, notably label propagation [START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF] and embedding propagation [START_REF] Pau Rodríguez | Embedding propagation: Smoother manifold for few-shot classification[END_REF], which smooths embeddings as in image segmentation [START_REF] Bertasius | Convolutional random walk networks for semantic image segmentation[END_REF]. Both operations are also used at representation learning, as in meta-learning. Using dimensionality reduction, TAFSSL [START_REF] Lichtenstein | TAFSSL: Task-adaptive feature sub-space learning for few-shot classification[END_REF] learns a task-specific feature subspace that is highly discriminant for novel tasks. Meta-confidence transduction (MCT) [28] meta-learns a data-dependent scaling function to normalize every example and iteratively updates class centers. PT+MAP [19] uses a similar iterative process but also balances over classes. Cross-attention [START_REF] Hou | Cross attention network for few-shot classification[END_REF], apart from aligning feature maps by correlation, leverages query examples by iteratively making predictions and using the most confident ones to update the class representation.

Semi-supervised learning In this case, labeled novelclass support examples and additional unlabeled data are given. A classifier may be learned on both to make predictions on novel-class queries [START_REF] Ren | Meta-learning for semi-supervised fewshot classification[END_REF][START_REF] Wang | Instance credibility inference for few-shot learning[END_REF][START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF].

One of the first contributions uses unlabelled examples to adapt prototypical networks [START_REF] Snell | Prototypical networks for few-shot learning[END_REF], while discriminating from distractor classes [START_REF] Ren | Meta-learning for semi-supervised fewshot classification[END_REF]. Common semi-supervised solutions are also adapted to few-shot classification, for instance learning to self-train [START_REF] Li | Learning to self-train for semi-supervised few-shot classification[END_REF], which adapts pseudo-label [START_REF] Lee | Pseudo-label: The simple and efficient semisupervised learning method for deep neural networks[END_REF] and TransMatch [START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF], which is an adaptation of MixMatch [5]. Instance credibility inference [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] predicts pseudo-labels iteratively, using a linear classifier to select the most likely to be correct and then augmenting the support set. Adaptive subspaces [START_REF] Simon | Adaptive subspaces for few-shot learning[END_REF] are learned from labeled and unlabeled data, yielding a discriminative subspace classifier that maximizes the margin between subspaces.

Contributions

In this work, focusing on the transfer learning paradigm to learn novel tasks given a fixed representation [START_REF] Tian | Rethinking few-shot image classification: a good embedding is all you need?[END_REF][START_REF] Mangla | Charting the right manifold: Manifold mixup for fewshot learning[END_REF], we make the following contributions:

1. We combine the power of predicting pseudo-labels in semi-supervised learning [START_REF] Lee | Pseudo-label: The simple and efficient semisupervised learning method for deep neural networks[END_REF][START_REF] Li | Learning to self-train for semi-supervised few-shot classification[END_REF] with label cleaning in learning from noisy labels [START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF]. 2. According to manifold assumption, we use label propagation [START_REF] Zhou | Learning with local and global consistency[END_REF][START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF] to infer pseudo-labels, while balancing over classes [START_REF] Arazo | Unsupervised label noise modeling and loss correction[END_REF]19] and iteratively re-using pseudolabels in the propagation process [25,[START_REF] Wang | Instance credibility inference for few-shot learning[END_REF]. 3. We achieve new state of the art in both transductive and semi-supervised few-shot learning.

Method

Problem formulation

At representation learning, we assume access to a labelled dataset D base with each example having a label in one of the classes in C base . This dataset is used to learn a mapping f : X → R d from an input space X to a ddimensional feature or embedding space.

The knowledge acquired at representation learning is used to solve novel tasks, assuming access to a dataset D novel with each example being associated with one of the classes C novel , where C novel is disjoint from C base . Examples in D novel may labeled or not.

In few-shot classification [START_REF] Vinyals | Matching networks for one shot learning[END_REF], a novel task is defined by sampling a support set S from D novel , consisting of N classes with K labeled examples per class, for a total of L := N K examples. Given the mapping f and the support set S, the problem is to learn an N -way classifier that makes predictions on unlabeled queries also sampled from D novel . Queries are treated independently of each other. This is referred to as inductive inference.

In transductive inference, a query set Q consisting of M unlabeled examples is also sampled from D novel . Given the mapping f , S and Q, the problem is to make predictions on Q, without necassarily learning a classifier. In doing so, one may leverage the distribution of examples in Q, which is important because M is assumed greater than L.

In semi-supervised few-shot classification, an unlabelled set U of M unlabeled examples is also sampled from D novel . Given f , S and U , the problem is to learn to make predictions on new queries from D novel , as in inductive inference. Again, M > L and we may leverage the distribution of U .

In this work, we focus on transductive inference and semisupervised classification, given f . The performance of f on inductive inference is one of our baselines. We develop our solution for transductive inference. In the semi-supervised case, we follow the same solution with Q replaced by U . Using the predictions on U , we then proceed as in the inductive case, with S replaced by S ∪ U .

Nearest-neighbor graph

We are given the mapping f , the labeled support set S := {(x i , y i )} L i=1 and the query set

Q := {x L+i } M i=1
, where

y i ∈ [N ] := {1, . . . , N }. We embed all examples from S and Q into V = {v 1 , . . . , v T } ⊂ R d , where T := L + M and v i := f (x i ) for i ∈ [T ]
. Following [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], we construct a k-nearest neighbour graph of the features in V , represented by a sparse T × T nonnegative affinity matrix A, with

A ij := [v i v j ] γ + , if i = j ∧ v i ∈ NN k (v j ) 0, otherwise (1)
for i ∈ [T ], j ∈ [N ],
where NN k (v) are the k-nearest neighbors of v in V and γ > 1 is a hyperparameter. Finally, we obtain the symmetric T × T adjacency matrix W := 1 2 (A + A ) and we symmetrically normalize it as

W := D -1/2 W D -1/2 , (2)
where D = diag(W 1 T ) is the T × T degree matrix of W .

Label propagation

Following [START_REF] Zhou | Learning with local and global consistency[END_REF], we define the T × N label matrix Y as

Y ij := 1, if i ≤ L ∧ y i = j 0, otherwise (3)
for i ∈ [T ], j ∈ [N ]
. Matrix Y has one column per class and one row per example, which is an one-hot encoded label for S and a zero vector for Q. Label propagation amounts to solving N linear systems

Z := (I -αW) -1 Y, (4)
where α ∈ [0, 1) is a hyperparameter. The resulting T × N matrix Z can be used to make predictions by taking the maximum element per row [START_REF] Zhou | Learning with local and global consistency[END_REF]. However, before making predictions, we balance over classes.

Class balancing

We focus on the M × N submatrix

P := Z L+1:T,: (5)
(the last M rows) of Z that corresponds to unlabeled queries. We first perform an element-wise power transform

P ij ← P τ ij ( 6
)
for i ∈ [M ], j ∈ [N ]
, where τ > 1, encouraging hard predictions. Parameter τ is analogous to the scale (or inverse temperature) of logits in softmax-based classifiers [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF][START_REF] Qi | Low-shot learning with imprinted weights[END_REF]41], only here the elements of P are proportional to class probabilities rather than logits.

Inspired by [19], we normalize P to a given row-wise sum p ∈ R M and column-wise sum q ∈ R N . Each element p i ∈ [0, 1] of p represents a confidence of example x L+i for i ∈ [M ]; it can be a function of the i-th row of P or set to 1. Each element q j ≥ 0 of q represents a weight of class j for j ∈ [N ]. In the absence of such information, we set

q := 1 N (p 1 M )1 N , (7)
assuming a uniform distribution of queries over classes. The normalization itself is a projection of P onto the set S(p, q) of nonnegative M × N matrices having row-wise sum p and column-wise sum q,

S(p, q) := {X ∈ R M ×N : X1 N = p, X 1 M = q}. (8)
We use the Sinkhorn-Knopp algorithm [START_REF] Arazo | Unsupervised label noise modeling and loss correction[END_REF] for this projection, which alternates between rescaling the rows of P to sum to p and its columns to sum to q,

P ← diag(p) diag(P 1 N ) -1 P (9)
P ← P diag(P 1 M ) -1 diag(q), (10)
until convergence. Finally, for each query

x L+i , i ∈ [M ],
we predict the pseudo-label

ŷL+i := arg max j∈[N ] P ij (11)
that corresponds to the maximum element of the i-th row of the resulting matrix P , for i ∈ [M ].

Label cleaning

The predicted pseudo-labels are not necessarily correct, yet a classifier can be robust to such noise. This is the case when enough data is available to adapt the representation [START_REF] Lee | Pseudo-label: The simple and efficient semisupervised learning method for deep neural networks[END_REF][START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], such that the quality of pseudo-labels improves with training. Since data is limited here, we would like to select pseudo-labeled queries in Q that are most likely to be correct, treat them as truly labeled and add them to the support set S. Iterating this process is an alternative way of improving the quality of pseudo-labels.

We interpret this problem as learning with noisy labels, leveraging recent advances in label cleaning [3,[START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF][START_REF] Song | Robust and on-the-fly dataset denoising for image classification[END_REF]. Assuming that the classifier does not overfit the data, e.g. with small capacity, high learning rate or few iterations, the principle is that examples with clean labels exhibit less loss than examples with noisy labels.

In particular, given the labeled support set S := {(x i , y i )} L i=1 and the pseudo-labeled query set Q := {(x L+i , ŷL+i )} M i=1 , we train an N -way classifier g using a weighted cross-entropy loss

:= - L i=1 log g(x i ) yi - M i=1 p i log g(x L+i ) ŷL+i , ( 12
)
where p i is the confidence weight of example x L+i . Here, the classifier g is assumed to yield a vector of probabilities over classes using softmax and g(x) y refers to element y ∈ [N ] of g(x). In practice, it is obtained by a linear classifier on top of embedding f , optionally allowing the adaptation of the last layers of the network implementing f .

The loss term i := -p i log g(x L+i ) ŷL+i , corresponding to the pseudo-labeled query x L+i , is used for selection. Following O2U-Net [START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF], we use a cyclical schedule of learning rate and collect the average loss li over all epochs, for i ∈ [M ]. In learning with noisy labels, it is common to detect noisy labels based on statistics of this loss value for clean and noisy labels [3,[START_REF] Song | Robust and on-the-fly dataset denoising for image classification[END_REF]. However, this does not work well with predicted pseudo-labels [START_REF] Albert | Relab: Reliable label bootstrapping for semi-supervised learning[END_REF], hence we select queries having the least average loss [START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF][START_REF] Albert | Relab: Reliable label bootstrapping for semi-supervised learning[END_REF]. Since we shall iterate the process, we take the extreme approach of selecting one query example per class:

I := arg min ŷL+i =j ¯ i : j ∈ [N ] . (13)
Finally, we augment the support set S with the selected queries and their pseudo-labels, while at the same time removing the selected queries from Q:

S ← S ∪ {(x L+i , ŷL+i )} i∈I (14) Q ← Q \ {x L+i } i∈I (15)

Iterative inference

Although label propagation and class balancing make predictions on the entire unlabeled query set Q, we apply cleaning to only keep one pseudo-labeled query per class, which we move from Q to the support set S. We iterate the entire process, selecting one pseudo-labeled query per class at a time, until Q is empty and S is augmented with all pseudo-labeled queries. Assuming that the selections are correct, the idea is that treating them as truly labeled in S improves the quality of the pseudo-labels. Sinkhorn-Knopp ( 9),(10)

9

Ŷ ← PREDICT(P ) pseudo-labels [START_REF] Guneet S Dhillon | A baseline for few-shot image classification[END_REF] 10 12),( 13)

I ← CLEAN(f, S, Q, Ŷ , p) label cleaning (
11 (S, Q) ← AUGMENT(S, Q, I) augment support (14),( 15
)
12 until Q = ∅ all queries are predicted Algorithm 1 summarizes this process, called iterative label propagation and cleaning (iLPC). Given S, Q and the embedding f , we construct the nearest neighbor graph represented by the normalized adjacency matrix W (1),( 2) and we perform label propagation on the current label matrix Y (4). Focusing on the unlabeled submatrix P of the resulting matrix Z, we perform power transform [START_REF] Bertinetto | Meta-learning with differentiable closed-form solvers[END_REF] and row/column normalization to balance over classes [START_REF] Da Chen | Self-supervised learning for few-shot image classification[END_REF],(10). We predict pseudo-labels Ŷ from the normalized P (11), which we use along with S and Q to train a linear classifier on top of f with cross entropy loss [START_REF] Finn | Modelagnostic meta-learning for fast adaptation of deep networks[END_REF] and a cyclical learning rate schedule [START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF]. We select one query per class with the least average loss over all epochs [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF], which we move from Q to S as labeled ( 14),(15). With Q, S redefined, we repeat the process until Q is empty.

At termination, all data is labeled in S. The predicted labels over the original queries are the output in the case of transductive inference. In semi-supervised classification, we use S to learn a new classifier and make predictions on new queries, as in inductive inference. in [START_REF] Ravi | Optimization as a model for few-shot learning[END_REF]. All images are resized to 84 × 84.

Experiments

tieredImageNet This is also sampled from ImageNet [27] but has a hierarchical structure. choice in the literature [START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF][START_REF] Li | Learning to self-train for semi-supervised few-shot classification[END_REF][START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF].

In the semi-supervised setting, the unlabeled set U contains an additional number of randomly sampled examples per novel class. This number depends on K. We use two settings, namely 30/50 and 100/100, where the first number (30 or 100) refers to 1-shot and the second (50 or 100) to 5-shot. Again, these are the two most common choices in semi-supervised few shot learning [START_REF] Li | Learning to self-train for semi-supervised few-shot classification[END_REF][START_REF] Wang | Instance credibility inference for few-shot learning[END_REF]28,[START_REF] Ren | Meta-learning for semi-supervised fewshot classification[END_REF][START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF].

Unless otherwise stated, we use 1000 tasks and report mean accuracy and 95% confidence interval on the test set.

Competitors As discussed in Appendix C, there are several flaws in experimental evaluation in the literature, like the use of different networks, training, versions of datasets, dimensionality and feature pre-processing. Fair comparison is impossible, unless one uses public code to reproduce results under exactly the same setup.

In this work, we do provide completely fair comparisons with such reproduced results of three state-of-the-art methods: LR+ICI [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF], PT+MAP [19] and MCT [28]. Only [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] is published, while the other two are pre-prints.

Networks We use publicly available pre-trained backbone convolutional neural networks that are trained on the baseclass training set. We experiment with two popular networks, namely, the residual network ResNet-12 [41] and the wide residual network WRN-28-10 [START_REF] Rusu | Meta-learning with latent embedding optimization[END_REF].

In particular, to compare with [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF], we use pre-trained weights of the ResNet-12 provided by [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF], which we call ResNet-12A, as well as official public code1 for testing. To compare with [19], we use pre-trained weights of a WRN-28-10 provided by [START_REF] Mangla | Charting the right manifold: Manifold mixup for fewshot learning[END_REF] 2 , which are the same used by [19], as well as official public code3 for testing. To compare with [28], we use official public code 4 to train from scratch another version of ResNet-12 used by [28], which we call ResNet-12B, as well as the same code for testing.

Feature pre-processing Each method uses its own feature pre-processing. LR+ICI [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] to reduce ResNet-12A to 5 dimensions. PR+MAP [19] uses element-wise power transform, 2 -normalization and centering of WRN-28-10 features. MCT [28] uses flattening of the output tensor of ResNet-12B rather than spatial pooling. By default, we use the same choices as [19,28] for WRN-28-10 and ResNet-12B. For ResNet-12A however, we use Implementation details We use PyTorch [42] and scikitlearn [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]. Label cleaning is based on a linear classifier on top of f , initialized by imprinting the average of support features per class and then trained using [START_REF] Finn | Modelagnostic meta-learning for fast adaptation of deep networks[END_REF]. We use SGD with momentum 0.9 and weight decay 0.0005. Following [START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF], we use a cyclical schedule of 10 cycles with 20 iterations per cycle for the learning rate, which linearly decreases from a maximum η max to zero. For inductive (resp. semi-supervised) learning, we use logistic regression on support (resp. also pseudo-labeled) examples, learned using scikit-learn [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF]. The row-wise sum p (9) is fixed to 1. Appendix B includes more choices.

Ablation study

Hyperparameters Our hyperparameters include γ and k used in the nearest neighbor graph (1), α in label propagation (4), τ in balancing ( 7) and the maximum learning rate η max of label cleaning. We optimize them on the val-idation set of every dataset. Common choices for k and α are in [15,[START_REF] Huang | O2U-Net: A simple noisy label detection approach for deep neural networks[END_REF] and in [0.5, 0.8], respectively. We set γ = 3, τ = 3 and η max = 0.1. More details and precise choices per dataset are given in Appendix A.

Algorithmic components Table 1 ablates our method in the presence or not of individual components, as well as using alternatives to our components. The use of queries with label propagation immediately gives a gain of transductive over inductive inference, up to 14% in 1-shot. Balancing and iterative label cleaning each bring another gain of 4-5%, especially in 1-shot. Iterative label cleaning is very important because selecting examples based on P instead is even worse than baseline transductive inference. Label propagation is also superior to using a linear classifier in most cases.

Label cleaning: loss distribution

To illustrate our label cleaning, we conduct two experiments, showing the distribution of the loss value [START_REF] Finn | Modelagnostic meta-learning for fast adaptation of deep networks[END_REF]. In the first, shown in Figure 2 The second experiment is on two novel 1-shot transductive tasks, shown in Figure 2(c,d). We use 50 unlabeled queries per class and we predict pseudo-labels according to [START_REF] Guneet S Dhillon | A baseline for few-shot image classification[END_REF]. Label cleaning is more challenging here because the two distributions are more overlapping. This is natural because predictions are more informed than uniform, even if incorrect. Still, a large proportion of clean examples have less loss than the minimal value of noisy ones.

Transductive inference

Table 2 compares our iLPC with LR+ICI [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] and PT+MAP [19]. The truly fair comparison is with our reproductions, indicated by *. Apart from the default networks, we also use WRN-28-10 with LR+ICI [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF], since it is more powerful. Our iLPC is superior or on par with competitors.

The gain over [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] up to 3% on miniImageNet 1-shot. The gain over PT+MAP [19] is around 0.5%.

We also experiment with 50 unlabeled queries per class, or M = 250 in total. As shown in Table 3, the gain over PT+MAP [19] increases up to 3% on miniImageNet 1-shot. This can be attributed to the fact that PT+MAP [19] operates on Euclidean space, while we capture the manifold structure, which manifests itself in the presence of more data. A 10shot experiment, using more labeled support examples, is shown in Table 4. The gain is around 0.5%.

Table 5 shows that PT+MAP [19] is very sensitive to feature pre-processing, losing up to 40% without it, while our iLPC more robust, losing only up to 5%. Similarly, Figure 3 shows that LR+ICI [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] is sensitive to dimension reduction, working best at only 5 dimensions. By contrast, our iLPC is very stable and only fails at 2 dimensions.

Table 6 compares our iLPC with MCT [28]. We reproduce MCT results by training from scratch ResNet-12B using the official code and we test both methods without data augmentation (horizontal flipping) and without meta-learned scaling function. The objective is to compare the two transductive methods under the same backbone network and the same training process, which is clearly superior to ResNet-12A. Under these settings, MCT is slightly better in 5-shot but iLPC outperforms it by a large margin in 1-shot.

Semi-supervised learning

As shown in Table 7, iLPC is superior to LR+ICI [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] in all settings by an even larger margin than in transductive inference, e.g. by nearly 5% in miniImageNet 1-shot. This can be be attributed to capturing the manifold structure of the data, since there is more unlabeled data in this case. Because PT+MAP [19] does not experiment with semi-supervised learning, we adapt it in the same way as ours, using the default WRN-28-10, again outperforming it.

Comparison with the state of the art

Conclusion

Our solution is conceptually simple and combines in a unique way ideas that have been successful in problems related to our task at hand. Label propagation exploits the manifold structure of the data, which becomes important in the presence of more data, while still being competitive otherwise. Class balancing provides a strong hint in correcting predictions when certain classes dominate. Label cleaning, originally introduced for learning with noisy labels, is also very successful in cleaning predicted pseudo-labels. Iterative reuse of few pseudo-labels as true labels bypasses the difficulty of single-shot detection of clean examples.

Importantly, reasonable baselines, like predicting pseudolabels by a classifier or iteratively re-using pseudo-labels without cleaning, fail completely. When compared under fair settings, our iLPC outperforms or is on par with stateof-the art methods on a range of networks and datasets. It is also significantly more robust against feature pre-processing and other tricks on which other methods rely. Finally, there is no data augmentation in our work. Augmentation should naturally bring complentary improvement.

A. Hyperparameters

Table 10 shows the best hyperparameters k (1) and α (4) for every dataset, network and number of support examples per class K ∈ {1, 5}. The hyperparameters are optimized on the validation set separately for each experiment. We carried out experiements in the transductive setting for k ∈ {5, 8, 10, 15, 20, 25, 30, 40, 50, 60} and α ∈ {0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9} and select the combination resulting in the best mean validation accuracy. In the semi-supervised setting, we use the same optimal values.

B. Confidence weights

The Sinkhorn-Knopp algorithm iteratively normalizes a M × N positive matrix P to a row-wise sum p ∈ R M and column-wise sum q ∈ R N . We experiment with two ways of setting the value of p:

1. Uniform. Interpreting the i-th row of P as a class probability distribution for the i-th query, it should be normalized to one, such that p i = 1 uniformly.

2.

Entropy. Because we do not have the same confidence for each prediction, we use the entropy of the predicted class probability distribution of each example to quantify its uncertainty. Following [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], we associate to each example x L+i for i ∈ [M ] a weight

ω i := 1 - H( ẑi ) log (N ) , ( 16
)
where N is the number of classes and ẑi is the

1 -normalized i-th row of Z (4), that is, ẑij := z ij / N k=1 z ik .
We then set the confidence weights p i = ω i . Note that ω i takes values in [0, 1] because log(N ) is the maximum possible entropy.

Given p and assuming balanced classes, q is defined by (7), that is, q j = 1 N M i=1 p i for j ∈ [N ]. In the special case of p i = 1, this simplifies to q j = M N . Table 11 compares the two approaches. Even though using non-uniform confidence weights is a reasonable choice, uniform weights are superior in all settings. This can be attributed to the fact that examples with small weight tend to be ignored in the balancing process, hence their class distribution and consequently their predictions are determined mostly by other examples with large weight. For this reason, examples with small weight may get more incorrect predictions in the case of entropy.

C. Flaws in evaluation

Throughout our investigations we observed that comparisons are commonly published that are not under the same settings. In this section we highlight such problems. 1. In multiple works such as [START_REF] Pau Rodríguez | Embedding propagation: Smoother manifold for few-shot classification[END_REF]19,[START_REF] Yang | Dpgn: Distribution propagation graph network for few-shot learning[END_REF][START_REF] Guneet S Dhillon | A baseline for few-shot image classification[END_REF], comparisons between state-of-the-art methods are made without explicitly differentiating between inductive and transductive methods. This is unfair since transductive methods perform better by leveraging query data.

2. Comparisons use different networks without mentioning so. For example, Table 1 of [START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF] does not indicate what network each method uses. [START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF] uses WRN-28-10, while [START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF] uses a 4-layer convolutional network.

3. In the semi-supervised setting, comparisons use different numbers of unlabelled data without mentioning so.

In Table 4 of [START_REF] Pau Rodríguez | Embedding propagation: Smoother manifold for few-shot classification[END_REF] for example, [START_REF] Pau Rodríguez | Embedding propagation: Smoother manifold for few-shot classification[END_REF] uses 100 unlabelled examples while [START_REF] Li | Learning to self-train for semi-supervised few-shot classification[END_REF] uses 30 for 1-shot and 50 for 5shot, [START_REF] Ren | Meta-learning for semi-supervised fewshot classification[END_REF] and [START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF] use 20 for 1-shot and 20 for 5-shot.

In Table 1 of [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF], the best model of [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] uses an 80/80 split for 1/5-shot while other methods such as [START_REF] Li | Learning to self-train for semi-supervised few-shot classification[END_REF] use a 30/50 split. In Table 1 of [START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF], [START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF] uses 100 or 200 unlabelled examples while [START_REF] Ren | Meta-learning for semi-supervised fewshot classification[END_REF][START_REF] Liu | Learning to propagate labels: Transductive propagation network for few-shot learning[END_REF] use 20/20 split for 1/5-shot.

4. Some methods use different dataset settings when comparing with other methods without explicitly stating so.

In Table 1 of [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] for instance, [START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] uses the bounding box provided for CUB while other methods such as [10, 30] do not.

5. Comparisons using the same network is made but this network has been trained using a different training regimes. Unless the novelty of the work lies in the training regime, this is unfair. As shown in [START_REF] Mangla | Charting the right manifold: Manifold mixup for fewshot learning[END_REF], a better training regime can increase the performance significantly.

6. There are several different variants of the benchmark datasets, coming from different sources. The two most common variants are [10], which uses original image files, and [START_REF] Lee | Meta-learning with differentiable convex optimization[END_REF], which uses pre-processed tensors stored in pkl files. Testing a network on a different variant than the one it was trained on may result in performance drops as large as 5%.

We believe that highlighting these evaluation flaws will help researchers avoid making such mistakes and move towards a fairer evaluation. We encourage the community to compare different methods against the same settings and if otherwise, state clearly the differences. As a contribution towards a fairer evaluation, we intend to make our code publicly available along with the pre-trained networks used in this work.

Figure 1 .

 1 Figure 1. Overview of the proposed method.

Algorithm 1 :

 1 Iterative label propagation and cleaning (iLPC).

2 W 4 Z 6 P

 246 input : embedding f input : labeled support set S with |S| = L input : unlabeled query set Q with |Q| = M output : augmented support set S with |S| = L + M 1 repeat ← GRAPH(f, S, Q; γ, k) adjacency matrix (1),(2) 3 Y ← LABEL(S) label matrix (3) ← LP(W , Y ; α) label propagation (4) 5 P ← Z L+1:L+M,:unlabeled submatrix (5) ← POWER(P ; τ ) power transform[START_REF] Bertinetto | Meta-learning with differentiable closed-form solvers[END_REF] 7 (p, q) ← BALANCE(P ) class balance (7) 8 P ← SINKHORN(P ; p, q)

4. 1 .Figure 2 .

 12 Figure 2. Distributions of loss values (12) for correctly and incorrectly labeled examples, normalized independently. Uniform label noise: (a) 20%, (b) 40%. Pseudo-labels predicted by (11) for two different 1-shot transductive miniImageNet tasks (c,d).

4. 2 .Figure 3 . 1 -

 231 Figure3. 1-shot trasductive inference on miniImageNet, ablation over LR+ICI[START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] pre-processing: dimension reduction by PCA.

  (a,b), we inject label noise uniformly at random to the 20% (a) and 40% (b) of 500 labeled examples. The correctly and incorrectly labeled examples have very different loss distributions. Importantly, while previous work on noisy labels [3, 20, 56] attempts to detect clean examples by an optimal threshold on the loss value, we only need few clean examples per iteration. Examples with minimal loss value are clean.

  CIFAR-FS This dataset is derived from CIFAR-100[START_REF] Krizhevsky | Learning multiple layers of features from tiny images[END_REF], consisting of 100 classes with 600 examples per class. We follow the split provided by [10], with 64 training, 16 validation and 20 test classes. To compare fairly with competitors, we use the original image resolution of 32 × 32 on WRN features to compare with [19] but we resize images to 84 × 84 on ResNet features to compare with[START_REF] Wang | Instance credibility inference for few-shot learning[END_REF].

Classes are partitioned into 34 categories, organized into 20 training, 6 validation and 8 test categories, containing 351, 97 and 160 classes, respectively. This ensures that training classes are semantically distinct from test classes, which is more realistic. We follow the common split of

[START_REF] Da Chen | Self-supervised learning for few-shot image classification[END_REF]

. Again, all images are 84 × 84.

CUB This is a fine-grained classification dataset consisting of 200 classes, each corresponding to a bird species. We follow the split defined by [10, 15], with 100 training, 50 validation and 50 test classes. To compare fairly with competitors, we use bounding boxes on ResNet features following

[START_REF] Triantafillou | Few-shot learning through an information retrieval lens[END_REF]

to compare with

[START_REF] Wang | Instance credibility inference for few-shot learning[END_REF]

but we do not use bounding boxes on WRN

[START_REF] Rusu | Meta-learning with latent embedding optimization[END_REF]

features to compare with

[19]

.

Table 1 .

 1 uses 2 -normalization and PCA Ablation study of algorithmic components of our method iLPC on miniImageNet. Inductive: baseline using only support examples. LP: label propagation. Balance: class balancing (7). iLC: iterative label cleaning, without which we just output predictions[START_REF] Guneet S Dhillon | A baseline for few-shot image classification[END_REF]. iProb: iterative selection of top examples per class directly as column-wise maxima of P (5) instead of iLC. Class: linear classifier used for prediction instead of LP, as in[START_REF] Wang | Instance credibility inference for few-shot learning[END_REF], with balancing still applied on output probabilities. †: default setting of iLPC.

	INFERENCE	COMPONENTS		RESNET-12	WRN-28-10
		LP Balance iLC iProb Class	1-shot	5-shot	1-shot	5-shot
	Inductive				56.30±0.62 75.59±0.47 60.04±0.64 81.35±0.43
	Transductive				61.09±0.70 75.32±0.50 74.35±0.68 84.76 ±0.42
	Transductive				65.04±0.75 76.82±0.50 79.21±0.72 88.00±0.41
	Transductive †				69.79±0.99 79.82±0.55 83.43±0.79 89.10±0.42
	Transductive				58.27±0.91 74.11±0.56 72.28±0.90 85.88±0.46
	Transductive				68.79±0.96 79.93±0.56 82.01±0.85 89.04±0.43
	METHOD	NETWORK	miniIMAGENET	tieredIMAGENET	CIFAR-FS	CUB
			1-shot	5-shot	1-shot	5-shot	1-shot	5-shot	1-shot	5-shot
	LR+ICI [62]	ResNet-12A 66.80	79.26	80.79	87.92	73.97	84.13	88.06	92.53
	LR+ICI [62]*	ResNet-12A 66.85±0.92 78.89±0.55 82.40±0.84 88.80±0.50 75.36±0.97 84.57±0.57 86.53±0.79 92.11±0.35
	iLPC (ours)	ResNet-12A 69.79±0.99 79.82±0.55 83.49±0.88 89.49±0.38 77.14±0.95 85.23±0.55 89.00±0.70 92.74±0.35
	PT+MAP [19]	WRN-28-10 82.92±0.26 88.82±0.13	-	-	87.69±0.23 90.68±0.15 91.55±0.19 93.99±0.10
	PT+MAP [19]* WRN-28-10 82.88±0.73 88.78±0.40 88.15±0.71 92.32±0.40 86.91±0.72 90.50±0.49 91.37±0.61 93.93±0.32
	LR+ICI [62]*	WRN-28-10 80.61±0.80 87.93±0.44 86.79±0.76 91.73±0.40 84.88±0.79 89.75±0.48 90.18±0.65 93.35±0.30
	iLPC (ours)	WRN-28-10 83.43±0.79 89.10±0.42 88.50±0.77 92.99±0.42 87.57±0.78 91.26±0.46 91.53±0.59 94.27±0.30

Table 2

 2

. Transductive inference, comparison with LR+ICI

[START_REF] Wang | Instance credibility inference for few-shot learning[END_REF]

and PT+MAP

[19]

. *: our reproduction with official code on our datasets.

Table 3 .

 3 Transductive inference, 50 queries per class. *: our reproduction with official code on our datasets.

	METHOD		NETWORK	miniIMAGENET	tieredIMAGENET	CIFAR-FS
						1-shot	5-shot	1-shot	5-shot	1-shot	5-shot
	PT+MAP [19]* WRN-28-10 83.79±0.71 88.94±0.33 88.87±0.64 92.01±0.36 87.63±0.66 90.15±0.46
	iLPC (ours)	WRN-28-10 86.30±0.71 89.99±0.32 90.60±0.64 92.97±0.38 88.51±0.69 91.23±0.44
	METHOD	mIN	tIN	CIFAR-FS	CUB
	PT+MAP [19]* 89.97±0.34 93.33±0.34 91.30±0.45 94.24±0.28
	iLPC (ours)	90.51±0.35 93.61±0.38 91.59±0.44 94.75±0.26
	Table 4. Transductive 10-shot inference using WRN-28-10. mIN:
	miniImageNet. tIN: tieredImageNet. *: our reproduction with
	official code on our datasets.
	METHOD	PRE	miniImageNet		tieredImageNet
			1-shot	5-shot		1-shot	5-shot
	PT+MAP [19]*	48.57±0.81 75.67±0.82 49.67±0.77 88.32±0.50
	iLPC (ours)	78.89±0.90 86.80±0.46 86.52±0.47 91.07±0.47
	PT+MAP [19]*	82.88±0.73 88.78±0.40 88.15±0.71 92.32±0.40
	iLPC (ours)	83.43±0.79 89.10±0.42 88.50±0.77 92.99±0.42

Table 5 .

 5 Transductive inference, ablation over PT+MAP [19] preprocessing. PRE: power transform, normalization, centering. *: our reproduction with official code on our datasets.

	METHOD	NETOWRK	miniIMAGENET
			1-shot	5-shot
	MCT (instance,flip) [28] ResNet-12B 78.55±0.86 86.03±0.42
	MCT (no scale) [28]*	ResNet-12B 67.26±0.60 81.90±0.43
	iLPC (ours)	ResNet-12B 75.58±1.16 81.58±0.50
	iLPC (ours)	ResNet-12A 69.79±0.99 79.82±0.55

Table 6 .

 6 Transductive inference, comparison with MCT[28] using ResNet-12B.

*: our reproduction with official code on our datasets, without augmentation and without scaling.

Table 8 and

 8 Table9compare our iLPC with a larger collection of recent methods on the tranductive and semisupervised settings, respectively. Even when the network and data split appears to be the same, we acknowledge that our results are not directly comparable with any method other than our reproductions. As discussed in Appendix C, this is due to the very diverse choices made in the bibliography, e.g. versions of network, training settings, versions of datasets,

	METHOD	NETWORK SPLIT	miniIMAGENET	tieredIMAGENET	CIFAR-FS	CUB
			1-shot	5-shot	1-shot	5-shot	1-shot	5-shot	1-shot	5-shot
	LR+ICI [62]	ResNet-12A 30/50 69.66	80.11	84.01	89.00	76.51	84.32	89.58	92.48
	LR+ICI [62]*	ResNet-12A 30/50 67.64±0.95 80.00±0.55 83.20±0.89 89.15±0.48 76.57±0.96 84.11±0.60 88.36±0.71	-
	iLPC (ours)	ResNet-12A 30/50 72.36±0.93 81.23±0.51 84.34±0.77 89.66±0.49 78.44±0.89 85.69±0.50 89.44±0.62	-
	LR+ICI [62]*	WRN-28-10 30/50 81.77±0.81 88.53±0.44 87.96±0.75 92.11±0.41 86.41±0.77 89.90±0.53 91.29±0.59	-
	PT+MAP [19] † WRN-28-10 30/50 83.47±0.68 89.15±0.38 88.67±0.68 92.44±0.38 87.71±0.67 90.38±0.45 92.05±0.52	-
	iLPC (ours)	WRN-28-10 30/50 84.00±0.74 89.68±0.37 88.87±0.73 92.66±0.40 87.78±0.69 90.58±0.50 92.35±0.52	-

Table 7 .

 7 Semi-supervised few-shot learning, comparison with LR+ICI[START_REF] Wang | Instance credibility inference for few-shot learning[END_REF] and PT+MAP[19]. *: our reproduction with official code on our datasets. †: our adaptation to semi-supervised, based on official code. CUB 5-shot omitted: no class has the required 70 examples.

	METHOD	NETWORK	miniIMAGENET	tieredIMAGENET	CIFAR-FS	CUB
			1-shot	5-shot	1-shot	5-shot	1-shot	5-shot	1-shot	5-shot
	LR+ICI [62]*	ResNet-12A 66.85±0.92 78.89±0.55 82.40±0.84 88.80±0.50 75.36±0.97 84.57±0.57 86.53±0.79 92.11±0.35
	CAN+Top-k [16]	ResNet-12 67.19±0.55 80.64±0.35 73.21±0.58 84.93 ±0.38	-	-	-	-
	DPGN [64]	ResNet-12 67.77±0.32 84.60±0.43 72.45±0.51 87.24±0.39 77.90±0.50 90.20±0.40 75.71±0.47 91.48±0.33
	MCT (instance) [28] ResNet-12B 78.55±0.86 86.03±0.42 82.32±0.81 87.36±0.50 85.61±0.69 90.03±0.46	-	-
	PT+MAP [19]*	WRN-28-10 82.88±0.73 88.78±0.40 88.15±0.71 92.32±0.40 86.91±0.72 90.50±0.49 91.37±0.61 93.93±0.32
	Fine-tuning [11]	WRN-28-10 65.73±0.68 78.40±0.52 73.34±0.71 85.50±0.50 76.58±0.68 85.79±0.50	-	-
	EP [50]	WRN-28-10 70.74±0.85 84.34±0.53 78.50±0.91 88.36±0.57	-	-	-	-
	SIB [17] †	WRN-28-10 70.00±0.60 79.20±0.40 72.90	82.80	80.00±0.60 85.3±0.40	-	-
	SIB+E 3 BM [36]	WRN-28-10 71.40±0.50 81.20±0.40 75.60±0.6	84.30±0.4	-	-	-	-
	iLPC (ours)	WRN-28-10 83.43±0.79 89.10±0.42 88.50±0.77 92.99±0.42 87.57±0.78 91.26±0.46 91.53±0.59 94.27±0.30

Table 8 .

 8 Transductive inference state of the art. *: our reproduction with official code on our datasets. †: tieredImageNet as reported by[START_REF] Liu | An ensemble of epoch-wise empirical bayes for few-shot learning[END_REF].

	METHOD	NETWORK SPLIT	miniIMAGENET
			1-shot	5-shot
	LST [31]	ResNet-12 30/50 70.10±1.90 78.70±0.80
	LR+ICI [62]	ResNet-12A 30/50 69.66	80.11
	MCT (instance) [28] ResNet-12B 30/50 73.80±0.70 84.40±0.50
	k-means [49] †	WRN-28-10 100/100 52.35±0.89 67.67±0.65
	TransMatch [65]	WRN-28-10 100/100 63.02±1.07 81.06±0.59
	iLPC (ours)	WRN-28-10 100/100 87.62±0.67 90.51±0.36

Table 9 .

 9 Semi-supervised few-shot learning state of the art. †: as reported by[START_REF] Yu | Transmatch: A transfer-learning scheme for semi-supervised few-shot learning[END_REF].

	or pre-processing. For instance, ResNet-12 is different than either ResNet-12A or ResNet-12B.
	For this reason, we focus on the best result by each method, including ours. Necessarily, methods experimenting with WRN-28-10 have an advantage. Still, at least among those, iLPC performs best by a large margin in both settings, with the closest second best being PT+MAP [19].

Table 10 .

 10 Selected hyperparameters. mIN: miniImageNet. tIN: tieredImageNet. CFS: CIFAR-FS. 79±0.99 79.82±0.55 83.43±0.79 89.10±0.42 entropy 66.94±1.01 78.34±0.58 81.05±0.90 88.43±0.44

	METHOD	RESNET-12	WRN-28-10
	1-shot	5-shot	1-shot	5-shot
	uniform 69.

Table 11 .

 11 Comparison between ways of setting confidence weights p; transductive inference on miniImageNet. Uniform: pi = 1.

Entropy: pi = ωi (16).

https://github.com/Yikai-Wang/ICI-FSL

https://github.com/nupurkmr9/S2M2_fewshot

https://github.com/yhu01/PT-MAP

https://github.com/seongmin-kye/MCT

-normalization only on transductive inference and we do not use any dimensionality reduction.