Marie Darcheville

Anne-Lise Adenot-Engelvin

Christophe Boscher

Nicolas Vukadinovic

Christophe Lefèvre

André Thiaville

Clément Sanchez

Christophe Lefevre

Evidence of the Superparamagnetic State in the Zero-Field Microwave Susceptibility Spectra of Ferrimagnetic Nanoparticles

Keywords: Nanomagnetics, dynamic magnetic susceptibility, ferromagnetic resonance, nanoparticles, superparamagnetism

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

111Chapitre d'équation 1 Section 1Nanomagnetics __________________________________________ Evidence of the superparamagnetic state in the zero-field microwave susceptibility spectra of ferrimagnetic nanoparticles I.  INTRODUCTION Magnetic iron oxide nanoparticles are widely studied for biomedical applications like markers for magnetic resonance imaging, new nanosystems for hyperthermia, or drug delivery carriers due to their low toxicity, and biocompatibility [START_REF] Xiao | Superparamagnetic nanoparticles for biomedical applications[END_REF]]. Their tunable sizes make them perfect candidates to observe or interact locally with entities at the cell scale without damaging healthy tissues [START_REF] Nikiforov | Magnetic Nanoparticles[END_REF]]. They are designed with a high saturation magnetization to provide a sufficient heating of the particles under the applied AC fields [Bañobre-López 2013] and their size is small enough to ensure a superparamagnetic behavior: for small particle sizes, the thermal energy is close to the difference of magnetic energy between two magnetization directions resulting in fluctuations in the magnetization direction.

Such particles are also suitable for designing new magnetic materials for electromagnetic applications as a constituent of an ink, which will be then ejected by inkjet printing for elaborating wellcontrolled patterned coatings [START_REF] Saleh | 3D inkjet-printed UV-curable inks for multi-functional electromagnetic applications[END_REF]]. The small grain size inhibits complex magnetic textures and thus prevents losses arising from the domain wall resonance. Recently, a one-step synthesis and deposition process of nanoparticle derived from microwave-assisted thermal decomposition synthesis has shown successful compatibility with CMOS inductors enhancing the Q factor 20 times and extending the frequency range on the C band in the case of Mn0.5Zn0.5Fe2O4 composition [START_REF] Sai | High-Q On-Chip C -Band Inductor With a Nanocrystalline MnZn-Ferrite Film Core[END_REF]] or with partially inverted ZnFe2O4 ferrites [START_REF] Sai | Integrated X-Band Inductor With a Nanoferrite Film Core[END_REF]]. Following [START_REF] Song | Exceeding natural resonance frequency limit of monodisperse Fe3O4 nanoparticles via superparamagnetic relaxation[END_REF], the size of the nanoparticle is believed to play a key role on the high frequency susceptibility through the superparamagnetic/ferromagnetic resonance [START_REF] Brown | Thermal Fluctuations of a Single-Domain Particle[END_REF]].

orresponding author: A-L. Adenot-Engelvin (anne-lise.adenot-engelvin@cea.fr). Digital Object Identifier: 10.1109/LMAG.XXXX.XXXXXXX (inserted by IEEE).

Shedding light on the size effect on the microwave susceptibility should allow a better design of these devices. The dynamic magnetic response of nanoparticles has been extensively analyzed by ferromagnetic resonance (FMR) measurements and the effect of superparamagnetic transition has been highlighted for small nanoparticles [Antoniak 2015]. However, in this kind of experiments conducted at a given frequency when sweeping the magnetic field, the magnetization configuration of the nanoparticles assembly depends on the strength of the static magnetic field making the analysis of experimental FMR spectra complex.

In this work, we performed zero-field microwave permeability measurements from 1 MHz up to 20 GHz to characterize the spectral response of Zn0.4Fe2.6O4 nanoparticles and evidence the influence of the superparamagnetic state. The selected chemical composition allows combining a high value of the saturation magnetization with a large electrical resistivity [START_REF] Fujiwara | Permeability and electric resistivity of spin-sprayed Zn ferrite films for high-frequency device applications[END_REF]].

II. SYNTHESIS OF Zn0.4Fe2.6O4 NANOPARTICLES

A microwave assisted thermal decomposition in organic solvents has been chosen [START_REF] Liang | Size-dependent electromagnetic properties and the related simulations of Fe3O4 nanoparticles made by microwave-assisted thermal decomposition[END_REF]] because it allows a good control of the nanoparticle size, the size distribution, and the morphology through the tuning of the decomposition temperature, the ratio between precursors and surfactant, the choice of solvent, precursors and surfactant, the heating rate, and the reaction time [START_REF] Song | Shape Control and Associated Magnetic Properties of Spinel Cobalt Ferrite Nanocrystals[END_REF][START_REF] Lu | Magnetic Nanoparticles: Synthesis, Protection, Functionalization, and Application[END_REF]. In particular, it is really convenient for mixed cubic ferrites MxFe3-xO4 synthesis in comparison with other synthetic routes [START_REF] Pessoa | A comprehensive study of Co1-xNixFe2O4 nanoparticles fabricated via three different synthetic methods[END_REF]]. However, one parameter remains not well controlled and is clearly affected by the synthesis route: the location of the M cations in the spinel sites. Indeed, it governs the magnetic properties of the whole structure, making those systems really hard to analyze [START_REF] Nakagomi | Cation distribution of ZnxCo1-xFe2O4 nanoparticles: A resonant X-ray diffraction study[END_REF]].

Iron acetylacetonate (Fe(acac)3, 97% purity), zinc acetylacetonate ------------------------------------hydrate (Zn(acac)2), oleic acid (OA, 90% purity) and ethyl acetate were purchased (Sigma-Aldrich), as well as ethylene glycol (EG), absolute ethanol (VWR), 1-octadecene (ODE, Merck Millipore) and oleylamine (OAm, 80-90% purity, Acros Organics). The following protocol was applied: 1 mmol Fe(acac)3 and 0.15 mmol Zn(acac)2 were added to a homogeneous mixture of 8 mL OAm, 2 mL OA and 6 mL ODE. EG was introduced only to make the nanoparticles grow. The obtained solution was ultrasound-sonicated during 1 hour before introduction in the microwave reactor Monowave 300 (Anton-Paar). The mixture was at first heated up to 473 K in 5 min and maintained at this temperature for 10 min. Then, the temperature was increased to 543 K in 5 min and maintained for 15 min. The reactor was finally cooled down to 333 K. Finally, the nanoparticles were recovered and washed six times by centrifugation processes with addition of a mixture of ethanol and ethyl acetate. Once all of the organics were removed, the nanoparticles were dried naturally under air. Two categories of nanoparticles were finally studied: small particles (S)P (synthesized without addition of EG in the reactor) and larger ones (L)P (with EG).

III. EXPERIMENTAL TECHNIQUES

The nanoparticle morphology and size were observed by transmission electron microscopy (TEM) using a JEOL JEM-2100F TEM with a GATAN UltraScan 4k camera. The diameter values were obtained from image processing with ImageJ. Sizes were averaged with 5 to 7 samples originating from different synthesis batches and were presented in histograms, applying the Sturges criterion [Aragon 2015]. The average was then calculated from 1000 to 2500 particles by fitting the histograms with a log normal distribution. The phase purity was probed by X-ray diffraction (XRD) with a Malvern PANalytical X'Pert PRO diffractometer using CuKα line radiation in θ/2θ configuration and the size of the diffracting domains was calculated with the Scherrer formula.

The magnetic properties were analyzed using a vibrating sample magnetometer (VSM) (Microsense, model EZ7) at room temperature and also with a VSM-SQUID magnetometer (Quantum Design, MPMS) for in-temperature measurements.

Effective magnetic permeability μeff and dielectric permittivity εeff of a cell (APC7 standard) were determined by means of an APC7 coaxial line (Fig. 1.A-C). From the measured transmission and reflection coefficients of a cell containing the nanoparticles, the μeff and εeff spectra were retrieved using the Nicolson-Ross algorithm [Le [START_REF] Gallou | Technique under compressive stress for determining the magneto-elastic effects in ferrites[END_REF]. Two vectorial network analyzers were used: a PNA N5232A from 1 MHz to 6 GHz and a PNA N5230A from 10 MHz to 20 GHz (both from Agilent Technologies) to cover a wide range of frequencies. The APC7 non-magnetic brass cell consists of two lateral plexiglass slices (Fig. 1.D-E) spaced 3 mm apart with the following properties: a relative permittivity of 2.67 and a thickness of 2.0 mm. The powder is introduced between the two plexiglas slices by keeping one of them as the bottom of the box. The cell is then closed with the second plexiglass slice (Fig. 1.D-E). Measurements of the permeability spectra were also conducted within the [243 K, 343 K] range by means of an HP 8720C network analyzer operating from 50 MHz up to 14 GHz. The sample is placed in a cell (Fig. 1.D-E) and the whole is then loaded in a furnace to control the temperature. The temperature is then monitored with a thermocouple sensor directly inserted inside the cell. Low temperatures are obtained by a water-cooling circuit.

Based on the volume fraction f V of nanoparticles in the cell, the use of a mixing law is required to determine the intrinsic dynamic permeability from the effective one, considering that the cell contains a mixture of nanoparticles with air. In our case, the retained mixing law is a proportional one with the following formula:

(1)

where µint is the intrinsic permeability. The calculation only regards the transverse dynamic susceptibility noted χ⊥, assuming that the parallel susceptibility is negligible. Thus, the relative permeability µr is defined as µr = 1 + χ⊥. The 3/2 prefactor comes from the randomly distributed anisotropy axes within the nanoparticles. It should be remarked that the mixing law does not consider the static and dynamic dipolar coupling between the particles (isolated particles). Taking into account these coupling effects could be achieved through an approach assuming effective demagnetizing factors [START_REF] Ramprasad | Magnetic properties of metallic ferromagnetic nanoparticle composites[END_REF]].

Considering the low level of measured effective imaginary permeability particularly in the case of the superparamagnetic particles, the uncertainty of measurement, about ±0.05 for the cell with Plexiglas sides, has to be taken into account [START_REF] Bardy | Control of Ferrites Microwave Properties at the French Atomic Energy Commission -Manufacturing Process and Characterization[END_REF]].

IV. RESULTS AND DISCUSSION

A. Nanoparticle size and morphology Fig. 2. shows nanoparticles synthesized by thermal decomposition for two ranges of size, namely, SP with a mean diameter of 6.2 ± 1 nm exhibiting spherical shapes (Fig. 2.A) and LP with a 22.4 ± 11 nm diameter with "raspberry" morphologies (Fig. 2.B). LP are assimilated to a binding between crystallites and the continuity of the crystal planes in one raspberry structure is noticed. This framework has already been observed in the case of iron oxide [START_REF] Gerber | Low Oxidation State and Enhanced Magnetic Properties Induced by Raspberry Shaped Nanostructures of Iron Oxide[END_REF]]. The average crystallite size for SP (resp. LP) computed from XRD patterns : 4.5 nm (resp.19.1 nm) is reduced compared to the TEM one : 6.2 ± 1 nm (resp. 22.4 ± 11 nm), as found for Fe3O4 nanoparticles [START_REF] Lima | Spin disorder and magnetic anisotropy in Fe3O4 nanoparticles[END_REF]].

μ int = 1 + 3 2 × f V ( μ eff -1 ) ------------------------------------- A B C 10

B. Static magnetic characterization

In the case of SP the magnetization curve may exhibit, if anisotropy is low enough, a superparamagnetic behavior at room temperature, characterized by a non-saturating signal under high magnetic field. Thus, in the case of a particle assembly, the magnetization direction of each nanoparticle fluctuates over time and the final state is disordered with a zero global magnetization for a measured time above the reversal time of the spins [START_REF] Dormann | Le phénomène de superparamagnétisme[END_REF]]. In this case, a simple model based on the Langevin function has been used to fit the magnetization curve, as follows:

(2)

where <M> is the average theoretical magnetization, L the Langevin function, MS the saturation magnetization, µ0 the permeability of free space, H the applied magnetic field, V the particle volume, k the Boltzmann constant, and T the temperature [START_REF] Dormann | Le phénomène de superparamagnétisme[END_REF]]. This equation allows recovering the intrinsic saturation magnetization and an apparent size. This procedure is well exemplified on SP, for which Fig. 3.A shows the measured and fitted magnetization curve, at room temperature. The fit provides a 5.4 nm diameter which could be interpreted as a magnetically ordered volume which is close to the crystallite size (XRD) and smaller than the TEM size. The superparamagnetic state for Zn0.4Fe2.6O4 SP is clearly exhibited by the zero-field cooling/field cooling (ZFC/FC) curve measured with a VSM-SQUID magnetometer as displayed in Fig. 3.B. At first, the nanoparticles are cooled from room temperature to 2 K at zero field. They are then subjected to a moderate applied field of 500 × 10 3 /(4π) A/m and the magnetization is measured during the increase of temperature up to 300 K (ZFC curve). The same process is done by decreasing the temperature to 2 K (FC curve) under the same field. It is observed that SP change from the blocked state to the superparamagnetic one at approximatively 33 K, determined from the maximum of the ZFC curve. The FC curve is not constant below TB, which means that dipolar interactions can be neglected in this case [Demortière 2016]. An estimate of the effective anisotropy constant K of nanoparticles can be determined from the ZFC curve.

In an uniaxial approximation, the effective anisotropy constant reads [START_REF] Batlle | Magnetic study of M -type doped barium ferrite nanocrystalline powders[END_REF]]:

(3

)
where τm is the measuring time (τm=2s in our case) and τ0 ~1×10 -9 s (case of MnFe2O4 nanoparticles) [START_REF] Rondinone | Determination of Magnetic Anisotropy Distribution and Anisotropy Constant of Manganese Spinel Ferrite Nanoparticles[END_REF]]. This leads to K~1.2×10 5 J/m 3 at TB =33 K for nanoparticles with a 5.4 nm diameter (Fig. 3.A). Moreover, the intrinsic saturation magnetization (65.6 A.m 2 /kg) obtained from ( 2) is consistent with the one determined at 2 K using a VSM-SQUID (64.9 A.m 2 /kg).

C. Dynamic magnetic characterization

1. At room temperature Fig. 4 exhibits the permeability spectra for SP and LP. For superparamagnetic SP, the imaginary part of the permeability spectrum reveals a broad resonance spreading between 0.5 GHz and 3.5 GHz (full width at half maximum, FWHM) with a low amplitude. For LP, the resonance amplitude is multiplied by a factor fourteen and the resonance band width extends from 0.5 GHz up to 5 GHz (FWHM). The level of permeability and the ferromagnetic resonance frequency are drastically modified by the nanoparticles size, hence the superparamagnetic state. This phenomenon could not have been reported by [START_REF] Liang | Size-dependent electromagnetic properties and the related simulations of Fe3O4 nanoparticles made by microwave-assisted thermal decomposition[END_REF] because these permeability measurements were performed only between 2 and 18 GHz. However, the size reduction does not result in a shift of the ferromagnetic resonance towards higher frequencies as suggested by [START_REF] Song | Exceeding natural resonance frequency limit of monodisperse Fe3O4 nanoparticles via superparamagnetic relaxation[END_REF].

At various temperatures

To analyze the dependence of the dynamic permeability through the change of the energy balance between the magnetic anisotropy energy and the thermal energy, permeability measurements have -----------------------------------been carried out within the temperature range 243 K to 343 K.

K = Ln( τm /τ0) (k TB / V ¿ --
The two sizes of nanoparticles have been considered. Fig. 5 displays the frequency dependence of the imaginary part of the permeability for LP. Two measurements at 293 K are reported, to compare the results coming from the previous setup (measurements at room temperature) and those arising from the novel one dedicated to the temperature analysis. Keeping in mind the measurement uncertainty, the data show no clear effect of temperature. For SP (Fig. 6), the imaginary part of the permeability spectrum is significantly modified when the temperature rises from 243 K to 343 K. The resonance frequency is shifted toward the high frequencies for decreasing temperature. The resonance line intensity defined as the area under the µ''(f) curve increases significantly for decreasing temperature, which is not observed for LP particles. These results are consistent with those coming from FMR measurements conducted on magnetite [START_REF] Griscom | Particle size effects in the FMR spectra of fine-grained precipitates in glass[END_REF][START_REF] Guskos | Ferromagnetic resonance and ac conductivity of a polymer composite of Fe3O4 and Fe3C nanoparticles dispersed in a graphite matrix[END_REF]] and maghemite nanoparticles [START_REF] Gazeau | Magnetic resonance of ferrite nanoparticles: evidence of surface effects[END_REF]]. Moreover, the width of the resonance peak (FWHM) seems to decrease for decreasing temperature which is in contrast with most of the FMR results. This point would deserve to be deepened through additional temperature dependent permeability measurements. The measurement quality is nevertheless affected by the temperature measurement set-up, especially for the lowest permeability amplitudes which are noisy, that is why the temperature curves displayed in Fig. 6 correspond to smoothed curves treated with the Savitzky-Golay method.

Interpretation through a dynamic susceptibility model

From the theoretical viewpoint, the high-frequency dynamics of superparamagnetic nanoparticles has been extensively studied. The general theoretical approach combines the Landau-Lifshitz-Gilbert equation for the motion of magnetic moment and the Fokker-Planck continuity equation for the orientational distribution function of the magnetic moment [START_REF] Raikher | Magnetization dynamics of single-domain particles by superparamagnetic theory[END_REF][START_REF] Poperechny | Ferromagnetic resonance in uniaxial superparamagnetic particles[END_REF]. This model has been developed in the context of the FMR where the magnetic moment is subjected to a static polarizing magnetic field. In our experiments, no static magnetic field is applied and the quantity of interest is the zero-field transverse dynamic susceptibility noted χ⊥. As a first approach, the experimental data were interpreted using an approximate analytical model proposed by [START_REF] Raikher | Theory of dispersion of the magnetic susceptibility of fine ferromagnetic particles[END_REF]. The main hypotheses are the following. (i) noninteracting randomly distributed nanoparticles are considered. (ii) each particle is assumed spherical, in a single-domain state and possesses an uniaxial magnetic anisotropy. By introducing the ratio between the magnetic anisotropy energy KV and the thermal energy kT, σ = KV/kT, the zero-field transverse dynamic susceptibility is expressed by: (4) where ω0 = µ0γHK is the angular resonance frequency, γ the gyromagnetic factor, HK the anisotropy field for an uniaxial particle defined as HK = 2K/(µ0MS), R1, R2, R3, R4 coefficients from [START_REF] Raikher | Theory of dispersion of the magnetic susceptibility of fine ferromagnetic particles[END_REF] and depending on σ, and α the Gilbert damping parameter. The temperature dependence of the effective anisotropy field used for fitting the experimental curves is displayed in (C) (black circles) with the K value at TB (red circle). The dashed line is a guide for eyes. Fig. 7.A shows the computed nanoparticle size dependence of the permeability spectra (imaginary part) at room temperature. A transition between a relaxant character and a resonant one is revealed for increasing the nanoparticle size. The model reproduces adequately the experimental curve for the nanoparticle diameter ϕ=5.4 nm (σ≈0.2) corresponding to the one calculated with the Langevin function (Fig. 3.A) and K=10 4 J/m 3 . This last value is one order of magnitude lower than the one obtained at TB. ( ω 0) 2 R 1 -ω 2 +2i ω ω 0 R 2 ------------------------------------dependences of the µ'' spectra. The experimental evolutions are satisfactorily reproduced by the analytical model, if a temperature dependence of the effective anisotropy field is introduced (Fig. 7.C). Such a rapidly decreasing anisotropy profile has been reported for FePt nanoparticles [START_REF] Antoniak | Magnetic anisotropy and its temperature dependence in iron-rich FexPt1 -x nanoparticles[END_REF]]. Consequently, this analytical model appears relevant to capture the main features of the experimentally observed superparamagnetic resonances.

V. CONCLUSION Size effects of Zn0.4Fe2.6O4 nanoparticles have been exhibited in the zero-field dynamic permeability spectra. The main effect is a drastic reduction of the permeability level, whereas a shift in the resonance frequency is not clearly observed.

A dynamic susceptibility model has been used to successfully reproduce the experimental results for the superparamagnetic particles and, in particular, the trend in the temperature dependence of the permeability spectra.

ACKNOWLEDGMENT

  Fig. 1. Set-up for permeability and permittivity measurements (A, B, C), APC7 cell 3D view (D) and cross-sectional view (E).

  Fig. 2. Zn0.4Fe2.6O4 (A) SP and (B) LP obtained by thermal decomposition with sizes of 6.2 ± 1 nm and 22.4 ± 11 nm respectively. (C) Histogram distributions of the particle size fitted with a lognormal function in blue lines.

Fig. 3 .

 3 Fig. 3. Measured and fitted magnetization curves of SP. Mass magnetization σ versus the applied field H (A), ZFC/FC measurement for SP (B).

Fig. 4 .

 4 Fig. 4. Permeability spectra for SP and LP. Dashed lines represent the real part of the permeability µ', and solid lines the imaginary part µ''.

Fig. 5 .

 5 Fig. 5. Evolution of the imaginary part of the permeability spectrum with temperature for Zn0.4Fe2.6O4 LP.

Fig. 6 .

 6 Fig. 6. Evolution of the imaginary part of the permeability spectrum with temperature for Zn0.4Fe2.6O4 SP.

Fig. 7 .

 7 Fig. 7. Computed permeability spectra (imaginary part, µ'') and comparison with experiment. (A) Nanoparticle size-dependence of µ'' at room temperature. The magnetic parameters are: K=10 4 J/m 3 , Ms=3.44 10 5 A/m, γ=1.76×10 11 s -1 T -1 , α=0.13. (B) Temperature dependence of µ'' for ϕ=5.4 nm.The temperature dependence of the effective anisotropy field used for fitting the experimental curves is displayed in (C) (black circles) with the K value at TB (red circle). The dashed line is a guide for eyes.

Fig

  Fig.7.A shows the computed nanoparticle size dependence of the permeability spectra (imaginary part) at room temperature. A transition between a relaxant character and a resonant one is revealed for increasing the nanoparticle size. The model reproduces adequately the experimental curve for the nanoparticle diameter ϕ=5.4 nm (σ≈0.2) corresponding to the one calculated with the Langevin function (Fig.3.A) and K=10 4 J/m 3 . This last value is one order of magnitude lower than the one obtained at TB. Fig.7.B compares the experimental and theoretical temperature

The authors express their appreciation for the support received from Jean-Hugues Le Gallou and Karim Ait Otmane from CEA Le Ripault for the dynamic magnetic measurements, and Silviu Colis from IPCMS for the VSM-SQUID analysis.


Yann Lifchitz

Yannis Avrithis

Sylvaine Picard

Few-Shot Few-Shot Learning and the role of Spatial Attention

Few-shot learning is often motivated by the ability of humans to learn new tasks from few examples. However, standard few-shot classification benchmarks assume that the representation is learned on a limited amount of base class data, ignoring the amount of prior knowledge that a human may have accumulated before learning new tasks. At the same time, even if a powerful representation is available, it may happen in some domain that base class data are limited or non-existent. This motivates us to study a problem where the representation is obtained from a classifier pre-trained on a large-scale dataset of a different domain, assuming no access to its training process, while the base class data are limited to few examples per class and their role is to adapt the representation to the domain at hand rather than learn from scratch. We adapt the representation in two stages, namely on the few base class data if available and on the even fewer data of new tasks. In doing so, we obtain from the pre-trained classifier a spatial attention map that allows focusing on objects and suppressing background clutter. This is important in the new problem, because when base class data are few, the network cannot learn where to focus implicitly. We also show that a pre-trained network may be easily adapted to novel classes, without meta-learning.

I. INTRODUCTION

The ever improving performance of deep learning models, apart from technical progress, is largely due to the existence of large-scale datasets, fully or weakly labeled by humans [START_REF] Russakovsky | ImageNet Large Scale Visual Recognition Challenge[END_REF]- [START_REF] Mahajan | Exploring the limits of weakly supervised pretraining[END_REF]. At the same time, reducing the need for supervision is becoming increasingly important, e.g. by taking advantage of prior learning [START_REF] Bengio | Deep learning of representations for unsupervised and transfer learning[END_REF]- [START_REF] Li | Learning without forgetting[END_REF] or exploiting unlabeled data [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF], [START_REF] Radosavovic | Data distillation: Towards omni-supervised learning[END_REF].

An extreme situation is few-shot learning [START_REF] Koch | Siamese neural networks for one-shot image recognition[END_REF]- [START_REF] Hariharan | Low-shot visual recognition by shrinking and hallucinating features[END_REF], where the problem is to learn novel (previously unseen) classes using only a very small labeled training set, typically not more than 10 examples per class. Here not only the annotation but even the raw data are not available. This problem is often motivated by the ability of humans to learn new tasks from few examples [START_REF] Lake | One shot learning of simple visual concepts[END_REF], [START_REF] Lake | Human-level concept learning through probabilistic program induction[END_REF], which has given rise to metalearning [START_REF] Santoro | Meta-learning with memory-augmented neural networks[END_REF]- [START_REF] Han | Face recognition with contrastive convolution[END_REF], or learning to learn. In this scenario, a training set is treated as a collection of smaller sets where every class has few labeled examples.

However, there is a huge gap between the motivating example of humans learning new tasks and how the few-shot classification task is set up. On one hand, for the sake of simplicity in experiments, the base class datasets where the representation is learned from scratch, contain a few dozen or hundred classes with a few hundred examples each. This is by no means comparable to datasets available to date [START_REF] Kuznetsova | The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale[END_REF], [START_REF] Mahajan | Exploring the limits of weakly supervised pretraining[END_REF], let alone the amount of prior knowledge that a human may have accumulated before learning a new task. On the other hand, for a given domain of novel classes, e.g. bird species [START_REF] Welinder | Caltech-UCSD Birds 200[END_REF], base class data of such size in the same domain may not exist.

In this work, we depart from the standard few-shot classification scenario in two directions. First, we allow the representation to be learned from a large-scale dataset in a domain different than the base and novel class domain. In particular, we model prior knowledge by a classifier that is pre-trained on such a dataset, having no access to its training process. We thus maintain the difficulty of domain gap and the simplicity of experiments (by not training from scratch), while allowing a powerful representation. Second, we assume only few or zero examples per base class. Hence, the role of base classes is to adapt the representation to the domain at hand rather than learn from scratch. This scenario can be seen as few-shot version of few-shot learning. Hence, we call it few-shot few-shot learning (FSFSL).

We treat this problem as a two-stage adaptation process, first on the few base class examples if available and second on the even fewer novel class examples. Because of the limited amount of data, it is not appropriate to apply e.g. transfer learning [START_REF] Bengio | Deep learning of representations for unsupervised and transfer learning[END_REF] or domain adaptation [START_REF] Ganin | Unsupervised domain adaptation by backpropagation[END_REF], in either of the two stages. Because the network is pre-trained, and we do not have access to its training process or data, meta-learning is not an option either. We thus resort to few steps of fine-tuning as in the meta-testing stage of Finn et al. [START_REF] Finn | Model-agnostic meta-learning for fast adaptation of deep networks[END_REF] and Ravi and Larochelle [START_REF] Ravi | Optimization as a model for few-shot learning[END_REF].

Focusing on image classification, we then investigate the role of spatial attention in the new problem. With large base class datasets, the network can implicitly learn the relevant parts of the images where to focus. In our setup, base class data are few, so our motivation is that a spatial attention mechanism may help the classifier in focusing on objects, suppressing background clutter. We observe that although the prior classes of the pre-trained classifier may be irrelevant to a new task, uncertainty over a large number of such classes may express anything unknown like background. This is a class-agnostic property and can apply to new tasks.

In particular, given an input image, we measure the entropybased certainty of the pre-trained classifier in its prediction on the prior classes at every spatial location and we use it to construct a spatial attention map. This map can be utilized in a variety of ways, for instance weighted spatial pooling or weighted loss per location; and at different situations like the two adaptation stages or at inference. By exploring different alternatives, we show that a pre-trained network may be easily adapted to novel classes, without meta-learning.

In summary, we make the following contributions:

• We introduce a new few-shot learning setting, called fewshot few-shot learning (FSFSL), where base class data is few and its role is to adapt the representation rather than learn from scratch. • We demonstrate that an off-the-shelf network pre-trained on large-scale out-of-domain data brings impressive improvement compared to methods specifically designed for few-shot learning. • We show that, given a large number of classes, a pretrained classifier can be used as a class-agnostic spatial attention mechanism, suppressing background clutter. • We show that spatial attention helps in adapting the representation, particularly when base class examples are few. In the following, we provide a detailed problem formulation and related background in section II, then we describe our spatial attention mechanism in section III and its use in fewshot classification in section IV. We provide experimental results in section V, and we conclude in section VI. The objective now is to learn a classifier on these support data, a process that we call adaptation. This classifier should map a new query example from X to a prediction in C . Few-shot few-shot classification. Few-shot classification assumes there is more data in base than novel classes, and a domain shift between the two, in the sense of no class overlap. Here we consider a modified problem where n can be small or zero, but there is another set C • = [c • ] of prior classes with even more data X • and labels y • with n n • := |X • | and a greater domain shift to C, C . Again, the most common setting is k examples per base class, so that n = kc. We are using a classifier that is pre-trained on this data but we do not have direct access to either X • , y • , or its learning process. The objective of base training is now to adapt the representation to the domain of C, C rather than learn it from scratch; but we still call it base training.

In the remaining of this section we present general background on few-shot classification that typically applies to base classes C or novel classes C , but may also apply to both, in which case the symbols c, C and c , C may be used interchangeably. Prior classes and the pre-trained network are only considered in the following sections.

Classifier. The classifier is a function f θ,W : X → R c (resp. R c ) with learnable parameters θ, W , mapping a new example x ∈ X to a vector of probabilities p := f θ,W (x) over c (resp. c ) base (resp. novel) classes. The classifier prediction is the class of maximum probability

π(p) := arg max j p j , (1)
where p j is the j-th element of p. The classifier is built on top of an embedding function φ θ : X → R r×d . Given an example x ∈ X , this function yields a r × d feature tensor φ θ (x), where r represents the spatial dimensions and d the feature dimensions. For X comprising 2d images for instance, the feature is a w×h×d tensor that is the activation of the last convolutional layer, where r = w × h is the spatial resolution. The embedding is a vector in R d in the special case r = 1.

The embedding parameters θ may be updated into θ in the adaptation process, in which case we have an embedding function φ θ and classifier f θ ,W . Again θ, θ may be used interchangeably.

Cosine classifier. A simple form of classifier that was introduced in few-shot learning independently by Qi et al. [START_REF] Qi | Low-shot learning with imprinted weights[END_REF] and Gidaris and Komodakis [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF] is a parametric linear classifier that consists of a fully-connected layer without bias on top of the embedding function φ θ followed by softmax. If W := (w j ) c j=1 is the collection of class weights with w j ∈ R r×d , the classifier is defined by

f θ,W (x) := σ τ [s(φ θ (x), w j )] c j=1 (2)
for x ∈ X , where σ : R m → R m is the softmax function σ(u) := (e u 1 ,...,e uc ) j e u j for u ∈ R c , τ ∈ R + is a trainable scale parameter and s is cosine similarity1 . We minimize the cost function

J(X, y; θ, W ) := n i=1 (f θ,W (x i ), y i ) (3)
over θ, W at base training, where (p, y) := -log p y for p ∈ R c + , y ∈ C is the cross-entropy loss. Prototypes. An alternative classifier that is more appropriate during few-shot adaptation or at testing is a prototype classifier proposed by Snell et al. [START_REF] Snell | Prototypical networks for few-shot learning[END_REF] and followed by Qi et al. [START_REF] Qi | Low-shot learning with imprinted weights[END_REF] and Gidaris and Komodakis [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF] too. If S j := {i ∈ [n ] : y i = j} denotes the indices of support examples labeled in class j, then the prototype of this class j is given by the average features

p j = 1 |S j | i∈Sj φ θ (x i ) (4)
of those examples for j ∈ C . Then, denoting by P := (p j ) c j=1 the collection of prototypes, a query x ∈ X is classified as f θ ,P (x), as defined by (2).

Dense classifier. This is a classifier where the loss function applies densely at each spatial location of the feature tensor rather than by global pooling or flattening as implied by [START_REF] Kuznetsova | The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale[END_REF]. The classifier can be of any form but a cosine classifier [START_REF] Qi | Low-shot learning with imprinted weights[END_REF], [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF] is studied by Lifchitz et al. [START_REF] Lifchitz | Dense classification and implanting for few-shot learning[END_REF]. In particular, the embedding φ θ (x) is seen as a collection of vectors [φ (q) (x)] r q=1 , where φ (q) (x) ∈ R d is an embedding of spatial location q ∈ [r]. The classifier ( 2) is then generalized to f θ,W : X → R r×c , now mapping an example to a vector of probabilities per location, defined by

f θ,W (x) := σ τ [s(φ (q) θ (x), w j )] c j=1 r q=1 (5)
for x ∈ X , while the class weights W are shared over locations with w j ∈ R d . Cross-entropy applies using the same label y i for each location of example x i , generalizing the cost function (3) to Contribution. The problem we consider is a variant of fewshot learning that has not been studied before. It involves sequential adaptation of a given network in two stages, each comprising a limited amount of data. There are many ways of exploiting prior learning to reduce the required amount data and supervision like transfer learning [START_REF] Bengio | Deep learning of representations for unsupervised and transfer learning[END_REF], [START_REF] Yosinski | How transferable are features in deep neural networks?[END_REF], domain adaptation [START_REF] Mallya | Piggyback: Adapting a single network to multiple tasks by learning to mask weights[END_REF], [START_REF] Ganin | Unsupervised domain adaptation by backpropagation[END_REF], [START_REF] Rebuffi | Learning multiple visual domains with residual adapters[END_REF], or incremental learning [START_REF] Li | Learning without forgetting[END_REF], [START_REF] Rebuffi | iCaRL: Incremental classifier and representation learning[END_REF], [START_REF] Yoon | Lifelong learning with dynamically expandable networks[END_REF]. However, none applies to the few-shot domain where just a handful of examples are given.

J(X, y; θ, W ) := n i=1 r q=1 (f (q) θ,W (x i ), y i ). (6
Attention has been studied as a core component of several few-shot learning and meta-learning approaches [START_REF] Vinyals | Matching networks for one shot learning[END_REF], [START_REF] Gidaris | Dynamic few-shot visual learning without forgetting[END_REF], [START_REF] Mishra | A simple neural attentive meta-learner[END_REF], [START_REF] Ren | Incremental few-shot learning with attention attractor networks[END_REF], but it always referred to examples (e.g. images) as a unit. Spatial attention on the other hand refers to neurons at different spatial locations. It is ubiquitous in several problems, for instance weakly supervised object detection [START_REF] Zhou | Learning deep features for discriminative localization[END_REF]- [START_REF] Zhu | Soft proposal networks for weakly supervised object localization[END_REF] and non-local convolution [START_REF] Hu | Squeeze-and-excitation networks[END_REF]- [START_REF] Chen | Aˆ2-nets: Double attention networks[END_REF], but has not been applied to few-shot learning until recently [START_REF] Wertheimer | Few-shot learning with localization in realistic settings[END_REF], [START_REF] Liu | Largescale long-tailed recognition in an open world[END_REF]. In [START_REF] Wertheimer | Few-shot learning with localization in realistic settings[END_REF], attention maps are computed by a module trained on base class data, which is not directly accessible in our setup. In [START_REF] Liu | Largescale long-tailed recognition in an open world[END_REF], attention encourages different classes to use different contexts, which helps maintain the discrimination between head and tail classes. By contrast, we are discriminating between foreground and background. Our spatial attention mechanism is extremely simple, based on the pre-trained network, without training on the base or novel class data.

III. SPATIAL ATTENTION FROM PRE-TRAINING

We assume a pre-trained network with an embedding function φ θ • : X → R r×d followed by global average pooling (GAP) and a classifier that is a fully connected layer with weights W

• := (w • j ) c • j=1 ∈ R d×c • and biases b • ∈ R c • , denoted jointly by U • := (W • , b • ).
Without re-training, we remove the last pooling layer and apply the classifier densely as in 1 × 1 convolution, followed by softmax with temperature T . Then, similarly to (5), the classifier f θ • ,U • : X → R r×c • maps an example to a vector of probabilities per location, where classifier parameters U • are shared over locations:

f θ • ,U • (x) := σ 1 T W • φ (q) θ • (x) + b • r q=1 . (7)
We now want to apply this classifier to examples in set X (resp. X ) of base (resp. novel) classes C (resp. C ) in order to provide a spatial attention mechanism to embeddings obtained by parameters θ (resp. θ ). We formulate the idea on X, C, θ in this section but it applies equally to X , C , θ . In particular, given an example x ∈ X, we use the vector of probabilities p (q) := f entropy function for this purpose, H(p) :=j p j log(p j ). We map the entropy to [0, 1], measuring the certainty of the pre-trained classifier in its prediction on the prior classes C • :

w (q) (x) := 1 - H(f (q) θ • ,U • (x)) log c • (8)
for q ∈ [r], where we ignore dependence on parameters θ • , U • to simplify notation, since they remain fixed. We use this as a weight for location q assuming that uncertainty over a large number of prior classes expresses anything unknown like background, which can apply to a new set of classes. We then 1 -normalize the weights w(x) := [w (q) (x)] r q=1 ∈ R r as ŵ(x) := w(x)/ w(x) 1 . We call ŵ(x) the spatial attention weights of x.

The weights are applied in different ways depending on the problem. If the embedding φ θ (x) is normally a vector in R d obtained by GAP on a feature tensor Φ θ (x) ∈ R r×d as 1 r q∈[r] Φ (q) θ (x) for x ∈ X , then GAP is replaced by global weighted average pooling (GwAP):

φ θ (x) := q∈[r] ŵ(q) (x)Φ (q) θ (x). (9)
for x ∈ X . We recall that this applies equally to θ in the case of novel classes. Figure 1 shows examples of images with spatial attention maps. Despite the fact that there has been no training involved for the estimation of attention on the particular datasets, the result can still be useful in suppressing background clutter.

IV. SPATIAL ATTENTION IN FEW-SHOT CLASSIFICATION

Here we discuss the use of attention maps at inference on novel classes, as well as learning on novel classes. In the latter case, the weights are pre-computed for all training examples since the pre-trained network remains fixed in this process. In summary, we either replace GAP by GwAP [START_REF] Koch | Siamese neural networks for one-shot image recognition[END_REF] in all inputs to the embedding network, or use dense classification [START_REF] Mallya | Piggyback: Adapting a single network to multiple tasks by learning to mask weights[END_REF].

A. Base class training

Starting from a pre-trained embedding network φ θ • , we can either solve new tasks on novel classes C directly, in which case θ = θ • , or perform base class training, fine-tuning θ from θ • . Adaptation may involve for instance fine-tuning the last layers or the entire network, applying a spatial attention mechanism or not. Recalling that φ θ • is still needed for weight estimation (8), the most practical setting is to fine-tune the last layers, in which case φ θ shares the same backbone network with φ θ • . Following MAML [START_REF] Finn | Model-agnostic meta-learning for fast adaptation of deep networks[END_REF], we perform few gradient descent steps with low learning rate.

We use a dense classifier f θ,W : X → R r×c (5) with class weights W . Given the few base class examples X and labels y, we learn W at the same time as fine-tuning θ by minimizing [START_REF] Li | Learning without forgetting[END_REF].

B. Novel class adaptation

Optionally, given the few novel class support examples X and labels y , we can further adapt the embedding network, while applying our attention mechanism to the loss function. As in subsection IV-A, φ θ shares the same backbone with φ θ , being derived from it by fine-tuning the last layers. We perform even fewer gradient descent steps with lower learning rate

We use a prototype classifier where vector embeddings φ θ (x ) of support examples x ∈ X are obtained by GwAP with φ θ defined as in ( 9) and class prototypes P := (p j ) c j=1 are obtained per class by averaging embeddings of support examples as defined by ( 4) and updated whenever θ is updated. The classifier f θ ,P : X → R c is a standard cosine classifier (2) and the loss function is standard cross-entropy J(X, y; θ, P ) (3) with embedding φ θ obtained by GwAP [START_REF] Koch | Siamese neural networks for one-shot image recognition[END_REF]. Attention weights apply to embeddings of all inputs to the network, each time focusing on most discriminative parts. In case of no adaptation to the embedding network, we fix θ = θ. Computing the prototypes P (4) is then the only learning to be done and we can proceed to inference directly.

C. Novel class inference

At inference, as in subsection IV-B, we adopt a prototype classifier where vector embeddings φ θ (x ) of support examples x ∈ X are obtained by GwAP with φ θ defined as in [START_REF] Koch | Siamese neural networks for one-shot image recognition[END_REF] and class prototypes P := (p j ) c j=1 are obtained per class by averaging embeddings of support examples as defined by [START_REF] Bengio | Deep learning of representations for unsupervised and transfer learning[END_REF]. Then, given a query x ∈ X , we similarly obtain a vector embedding φ θ (x) by GwAP [START_REF] Koch | Siamese neural networks for one-shot image recognition[END_REF] and predict the class π(f θ ,P (x)) of the nearest prototype according to cosine similarity where π is given by (1) and f θ ,P by ( 2). We thus focus on discriminative parts of both support and query examples, suppressing background clutter.

V. EXPERIMENTS A. Experimental setup

Pretrained Network. We assume that we have gathered prior knowledge on unrelated visual tasks. This knowledge is modeled by a deep convolutional network, trained on a large-scale dataset. In our experiments, we choose to use a ResNet-18 [START_REF] He | Deep residual learning for image recognition[END_REF] pre-trained on the Places365-Standard subset of Places365 [START_REF] Zhou | Places: A 10 million image database for scene recognition[END_REF]. We refer to this subset as Places. This subset contains around 1.8 million images across 365 classes. The classes are outdoor and indoor scenes. We select this dataset for its large scale, diversity of content and different nature than other popular datasets like CUB-200-2011 (see below). Images are resampled to 224×224 pixels for training. We choose ResNet-18 as the architecture of the pre-trained model as it is a powerful network that is also used in other few-shot learning studies [START_REF] Chen | A closer look at few-shot classification[END_REF], [START_REF] Dvornik | Diversity with cooperation: Ensemble methods for few-shot classification[END_REF], which helps in comparisons. We make no assumption on the pre-training of the network. We do not access either the pre-training process or the dataset. We rather use a publicly available converged model that has been trained with a fully-connected layer as a classifier, as assumed in section III.

Datasets. We apply our method to two standard datasets in few-shot learning. The first is CUB-200-2011 [START_REF] Wah | The Caltech-UCSD Birds-200-2011 Dataset[END_REF], referred to as CUB, originally meant for fine-grained classification, and subsequently introduced to few-shot learning by Hilliard et al. [START_REF] Hilliard | Few-shot learning with metric-agnostic conditional embeddings[END_REF]. This dataset contains 11,788 images of birds across 200 classes corresponding to different species. We use the split proposed by Ye et al. [START_REF] Ye | Learning embedding adaptation for few-shot learning[END_REF], where 100 classes are used as base classes and the remaining 100 as novel, out of which 50 for validation and 50 for testing. CUB images are cropped using bounding box annotations and resampled to 224×224.

The second dataset is miniImageNet [START_REF] Vinyals | Matching networks for one shot learning[END_REF], a subset of the ImageNet ILSVRC-12 [START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF] containing 100 classes with 600 images per class. Following the split from Ravi and Larochelle [START_REF] Ravi | Optimization as a model for few-shot learning[END_REF], 64 classes are used as base classes and 36 as novel, out of which 16 for validation and 20 for testing. Originally, miniImageNet images have been downsampled from the ImageNet resolution to 84×84. In this work, similarily to [START_REF] Chen | A closer look at few-shot classification[END_REF], [START_REF] Dvornik | Diversity with cooperation: Ensemble methods for few-shot classification[END_REF], we resample to 224×224 instead, which is consistent with the choice of pre-trained network.

Dataset overlap. Contrary to CUB, miniImageNet has some non-negligible overlap with Places. Some classes or even objects appear in both datasets. To better satisfy our assumption of domain gap, we remove the most problematic overlapping classes from miniImageNet: 3 base classes, 1 validation class and 2 novel classes. We refer to this pruned dataset as modified miniImageNet. For the sake of comparison and because this overlap can happen in practice, we also experiment on the original miniImageNet.

In Evaluation protocol. To adapt to our few-shot version of fewshot learning, we randomly keep only k images per base class. We experiment with k ∈ {0, 1, 5, 10} and k ∈ {0, 20, 50} respectively for the CUB and miniImageNet. This is because CUB classes refer to bird species, while miniImageNet classes to broad object categories, hence have a lot more variability. For novel classes, we use the standard setting k ∈ {1, 5}. We generate a few-shot task on novel classes by selecting a support set X . In particular, we sample c classes from the validation or test set and from each class we sample k images. In all experiments, c = 5, i.e. 5-way classification. For each task we additionally sample 30 novel class images per class, to use as queries for evaluation. We report average accuracy and 95% confidence interval over 5,000 tasks for each experiment. Baselines. We evaluate experiments with the network being either pre-trained on Places or randomly initialized. In both cases, we report measurements for different number k of examples per base class, as well as all examples in X. In the latter case (randomly initialized), we do not use the option k = 0 because then there would be no reasonable representation to adapt or to perform inference on, given a few-shot task on novel classes. Furthermore, in this case we do not apply the attention mechanism as it is based on the pretrained classifier. In all cases, we compare to the baselines of using no adaptation and no spatial attention. When learning from scratch, spatial attention is not applied as we do not have access to the pre-trained classifier. In the case of random initialization, and using all examples in X, we compare to Baseline++ [START_REF] Chen | A closer look at few-shot classification[END_REF] and prototypical networks [START_REF] Snell | Prototypical networks for few-shot learning[END_REF], as reported in the benchmark by Chen et al. [START_REF] Chen | A closer look at few-shot classification[END_REF], as well as category traversal (CTM) [START_REF] Li | Finding taskrelevant features for few-shot learning by category traversal[END_REF] and ensembles [START_REF] Dvornik | Diversity with cooperation: Ensemble methods for few-shot classification[END_REF], all using ResNet-18. They can only be compared to our randomly initialized baseline when using base training on all data. Implementation details. At base training, we use stochastic gradient descent with Nesterov momentum with mini-batches of size 200. At adaptation, we perform a maximum of 60 iterations over the support examples using Adam optimizer with fixed learning rate. In both cases, the learning rate, schedule if any and number of iterations are determined on the validation set. The temperature used by [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF] for the computation of the entropy is fixed per dataset, again on the validation set. In particular, we use T = 100 and T = 2.6 respectively for CUB and modified miniImageNet.

NOVEL: k = 1 NOVEL: k = 5

B. Results

We present results in Tables II and III respectively for CUB and modified miniImageNet. The original miniImageNet is discussed separately at the end of this section. Effect of base training. For fine-grained few-shot classification (CUB), base training is extremely important in adapting to the new domain, improving the baseline 1-shot accuracy by more than 40% with no adaptation and no spatial attention. On object classification in general (modified miniImageNet), it is less important, improving by 4.5%. It is the first time that experiments are conducted on just a small subset of the base class training set. It is interesting that 50 examples per class are bringing nearly the same improvement as all examples, i.e. hundreds per class. Effect of (novel class) adaptation. Fine-tuning the network on k novel class examples per class, even fewer than k in the case of base classes, comes with the risk of over-fitting. We still show that a small further improvement is possible with a small learning rate. The improvement is more significant when k is low, in which case, more adaptation of the embedding network to the novel class domain is needed. In the extreme case of CUB dataset without base training, adaptating on only the 25 images of the 5-way 5-shot tasks brings an improvement of 8.20%.

Effect of spatial attention. Spatial attention allows focusing on the most discriminative parts of the input, which is more beneficial when fewer examples are available. The extreme case is having no base class images and only one image per novel class. In this case, most improvement comes on modified miniImageNet without base training, where spatial attention improves 5-way 5-shot classification accuracy by 1.5% after adaptation. The attention maps appear to be domain independent as they improve CUB accuracy even when no images from the bird domain have been seen (k = 0).

Original miniImageNet results. The original miniImageNet dataset partially overlaps Places. We use T = 2.4 for the temperature in [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF]. The remaining setup is as for modified miniImageNet. Results are shown in Table IV.

  II. PROBLEM, BACKGROUND, RELATED WORK AND CONTRIBUTION Few-shot classification. We are given a set of training examples X := {x i } n i=1 ⊂ X , and corresponding labels y := (y i ) n i=1 ⊂ C n where C := [c] := {1, . . . , c} is a set of base classes. The objective is to learn a representation on these data, a process that we call base training, such that we can solve new tasks. A new task comprises a set of support examples X := {x i } n i=1 ⊂ X and labels y := (y i ) n i=1 ⊂ (C ) n , where n n and C := [c ] is a set of novel classes disjoint from C. The most common setting is k examples per novel class, so that n = k c , referred to as c -way, k -shot classification.

Fig. 1 .

 1 Fig. 1. Examples of images from CUB (top) and miniImageNet (bottom) overlaid with entropy-based spatial attention maps obtained from (8) using only the predicted class probabilites from ResNet-18 pre-trained on Places. See section V for details on datasets and networks.

  The base class training set X contains k examples per base class in C. Each experiment can be seen as a few-shot classification task on few base class examples.

  particular, we measure, for each miniImageNet class, what is the most frequent prediction among Places classes by the pre-trained Resnet-18 classifier and what proportion of examples are classified in this class. Ranking miniImageNet classes by that proportion, we check class names on both datasets and manually inspect examples in the top-ranking classes. We remove only the most clearly overlapping classes, that is, identical classes and classes depicted in images of a Places class. For instance, most images from the bus stationindoor class of Places contain a bus, so we chose to remove the school bus class from miniImageNet. Table I lists the classes removed from miniImageNet in this way.

TABLE II

 II Average 5-way k -shot novel class accuracy on CUB. WE USE RESNET-18 EITHER PRE-TRAINED ON PLACES OR WE TRAIN IT FROM SCRATCH ON k BASE CLASS EXAMPLES. PROTONET [11] IS AS REPORTED BY CHEN et al. [42]. FOR ENSEMBLE [43], WE REPORT THE DISTILLED MODEL FROM AN ENSEMBLE OF 20. BASELINES AS REPORTED IN THE LITERATURE, WITHOUT ATTENTION OR ADAPTATION; TO BE COMPARED ONLY TO RANDOMLY INITIALIZED WITH k = ALL.

			NOVEL: k = 1			NOVEL: k = 5
	Attention
	Adaptation
	BASE				PLACES
	k = 0	61.66±0.30	63.36±0.29	62.09±0.30	63.56±0.30	78.86±0.22	80.15±0.22	80.38±0.22	81.05±0.22
	k = 20	62.95±0.29	63.15±0.28	63.11±0.29	63.33±0.29	78.41±0.21	78.53±0.21	79.67±0.21	79.82±0.21
	k = 50	65.07±0.29	65.10±0.29	65.18±0.29	65.24±0.29	79.94±0.20	79.99±0.20	80.88±0.20	80.96±0.20
	ALL	66.20±0.29	65.94±0.29	66.23±0.29	66.06±0.29	80.37±0.21	80.24±0.21	81.56±0.20	81.50±0.20
	BASE				RANDOMLY INITIALIZED
	k = 20	33.43±0.21	-	33.35±0.21	-	43.83±0.21	-	44.21±0.21	-
	k = 50	41.03±0.24	-	41.05±0.24	-	54.68±0.22	-	54.92±0.22	-
	ALL	55.99±0.28	-	56.13±0.28	-	72.43±0.22	-	73.10±0.21	-

TABLE III

 III Average 5-way k -shot novel class accuracy on modified miniImageNet. WE USE RESNET-18 EITHER PRE-TRAINED ON PLACES OR WE TRAIN IT FROM SCRATCH ON k BASE CLASS EXAMPLES. BASELINES ONLY SHOWN IN

  TABLE IV ON THE ORIGINAL miniIMAGENET.

For matrices u, v ∈ R r×d , s(u, v) := u, v /( u v ) with •, • and • being the Frobenius inner product and norm respectively.

θ • ,U • (x) corresponding to spatial location q ∈ [r] to compute a scalar weight w (q) (x), expressing the discriminative power of the particular location q of example x.Since x belongs to a set of classes C different than C • , there is no ground truth to be applied to the output of the pre-trained classifier f θ • ,U • . However, the distribution p (q) can still be used to evaluate how discriminative the input is. We use the

Compared to the results of modified miniImageNet (Table III), performances are nearly uniformly increased by 3-4% and conclusions remain the same. The increase in performance is due to having more training data, as well as putting back easily classified classes in the test dataset. Observe that, unlike CUB (cf . Table II), CTM [START_REF] Li | Finding taskrelevant features for few-shot learning by category traversal[END_REF] and ensembles [START_REF] Dvornik | Diversity with cooperation: Ensemble methods for few-shot classification[END_REF] perform better than our randomly initialized baseline. Our objective is not to improve the state of the art of the standard few-shot setup, but rather to study the new problem using a network pre-trained on a large-scale dataset. In this respect, our simple baseline better facilitates future research.

VI. CONCLUSION

In this paper we address the problem of few-shot learning when even base classes images are limited in number. To address it, we use a pre-trained network on a large-scale dataset and a very simple spatial attention mechanism that does not require any training on the base or novel classes. We consider two few-shot learning datasets: CUB and miniImageNet, with different domain gaps to our prior dataset Places. Our findings indicate that even when the domain gap is large between the dataset used for pre-training and the base/novel class domains, it is still possible to get significant benefit from base class training even with a few examples, which is very important as it reduces the need for supervision. The gain from spatial attention is more pronounced in this case.