X Zhou

D Cattiaux

R R Gazizulin

A Luck

O Maillet

T Crozes

J-F Motte

O Bourgeois

A Fefferman

E Collin

On-chip thermometry for microwave optomechanics implemented in a nuclear demagnetization cryostat

Keywords: Mechanics, Condensed Matter Physics, Quantum Physics

We report on microwave optomechanics measurements performed on a nuclear adiabatic demagnetization cryostat, whose temperature is determined by accurate thermometry from below 500 µK to about 1 Kelvin. We describe a method for accessing the on-chip temperature, building on the blue-detuned parametric instability and a standard microwave setup. The capabilities and sensitivity of both the experimental arrangement and the developed technique are demonstrated with a very weakly coupled silicon-nitride doubly-clamped beam mode of about 4 MHz and a niobium on-chip cavity resonating around 6 GHz. We report on an unstable intrinsic driving force in the coupled microwave-mechanical system acting on the mechanics that appears below typically 100 mK. The origin of this phenomenon remains unknown, and deserves theoretical input. It prevents us from performing reliable experiments below typically 10-30 mK; however no evidence of thermal decoupling is observed, and we propose that the same features should be present in all devices sharing the microwave technology, at different levels of strengths. We further demonstrate empirically how most of the unstable feature can be annihilated, and speculate how the mechanism could be linked to atomic-scale two level systems. The described microwave/microkelvin facility is part of the EMP platform, and shall be used for further experiments within and below the millikelvin range.

I. INTRODUCTION

Advances in clean-room technologies within the last decades make it possible to create mechanical elements with one (or more) dimensions below a micron, namely NEMS (Nano-Electro-Mechanical Systems) [START_REF] Cleland | Fabrication of high frequency nanometer scale mechanical resonators from bulk Si crystals[END_REF]. These objects can be embedded in electronic circuits, and today even within quantum electronic circuits [START_REF] O'connell | Quantum ground state and singlephonon control of a mechanical resonator[END_REF][START_REF] Palomaki | Coherent state transfer between itinerant microwave fields and a mechanical oscillator[END_REF][START_REF] Pirkkalainen | Hybrid circuit cavity quantum electrodynamics with a micromechanical resonator[END_REF]. Indeed a breakthrough has been made with microwave optomechanics, essentially shifting the concepts of optomechanics into the microwave domain [START_REF] Regal | Measuring nanomechanical motion with a microwave cavity interferometer[END_REF]. These quantummechanical devices can then be thought of as a new resource for quantum electronics and quantum information processing, with the realization of e.g. quantumlimited optical-photon/microwave-photon converters and non-reciprocal microwave circuits [START_REF] Rueda | Efficient microwave to optical photon conversion: an electro-optical realization[END_REF][START_REF] Higginbotham | Harnessing electro-optic correlations in an efficient mechanical converter[END_REF][START_REF] Metelmann | Nonreciprocal Photon Transmission and Amplification via Reservoir Engineering[END_REF][START_REF] Gabriel | Demonstration of efficient nonreciprocity in a microwave optomechanical circuit[END_REF][START_REF] Bernier | Nonreciprocal reconfigurable microwave optomechanical circuit[END_REF][START_REF] Barzanjeh | Mechanical on-chip microwave circulator[END_REF]. Profound quantum concepts are also under study, with for instance mechanical motion squeezing [START_REF] Wollman | Quantum squeezing of motion in a mechanical resonator[END_REF] and recently mechanical entanglement of separate objects [START_REF] Ockeloen-Korppi | Stabilized entanglement of massive mechanical oscillators[END_REF].

In order to operate at the quantum limit, the mechanical mode in use has to be initially in its quantum ground state; for experiments on microwave optomechanics based on megahertz motion, this shall rely on a strong red-detuned pump signal that actively cools the mode, with an environment remaining "hot" [START_REF] Bernier | Nonreciprocal reconfigurable microwave optomechanical circuit[END_REF][START_REF] Massel | Multimode circuit optomechanics near the quantum limit[END_REF]16]. For most applied issues this is not a problem, even if it brings an additional complexity with a microwave tone that has to be kept on, which could lead to heating of the circuit and mixing with other signals involved. However, for all experiments aiming at a careful characterization of the mechanical decoherence due to the thermal bath, this is impractical: the whole system is required to be in thermodynamic equilibrium. As such, theoretical proposals testing the grounds of quantum mechanics have been released in the literature, both on microwave setups using quantum circuits [START_REF] Armour | Probing the quantum coherence of a nanomechanical resonator using a superconducting qubit: I. Echo scheme[END_REF] and conventional optomechanics (i.e. with lasers) [START_REF] Kleckner | Creating and verifying a quantum superposition in a micro-optomechanical system[END_REF].

Indeed, the first attempt to use nuclear adiabatic demagnetization for optomechanics is due to Dirk Bouwmeester, building on the technology of Leiden Cryogenics [START_REF] Paul | Micro-Optomechanical Systems for Quantum Optics[END_REF]. This ambitious setup was aimed at cooling a macroscopic moving mirror actuated/detected by optical means. On the other hand, a microwave-based optomechanical system hosting a NEMS is much less demanding; the heat loads from both the photon field and the internal heat release of materials are much less. As such, these setups have already proven to be compatible with dilution (millikelvin) technology [3-6, 11, 13, 14].

We have thus built a microwave platfrom for optomechanics on a nuclear adiabatic demagnetization cryostat. The microwave circuitry has been kept as basic as possible so far for demonstration purposes; no JPA (Josephson Parametric Amplifier) has been used, and we rely only on intrinsic properties of optomechanics for the measurement [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]. The cryostat reaches temperatures below 500 µK, and is equipped with accurate thermometry from the lowest temperatures up to about 1 K: using a noise SQUID-based (Superconducting QUantum Interference Device) thermometer [START_REF] Shibahara | Primary current-sensing noise thermometry in the millikelvin regime[END_REF] plus a 3 He-fork thermometer [START_REF] Blaauwgeers | Quartz Tuning Fork: Thermometer, Pressureand Viscometer for Helium Liquids[END_REF].

We report on measurements realized on this platform with a very weakly coupled doubly-clamped beam flex- ural mode embedded in an on-chip cavity. Beam-based devices are indeed a tool of choice for applications like ultimate sensing (e.g. single molecule mass spectrometry [START_REF] Naik | Towards single-molecule nanomechanical mass spectrometry[END_REF]), while the smallness of the photon-phonon coupling enables demonstrating the sensitivity of the methods used: we build on the optomechanical blue-detuned pumping instability to extract the on-chip temperature.

On the other hand, popular drumhead devices achieve routinely much higher couplings [START_REF] Palomaki | Coherent state transfer between itinerant microwave fields and a mechanical oscillator[END_REF][START_REF] Wollman | Quantum squeezing of motion in a mechanical resonator[END_REF] (from ×10 to ×100); the very same methods can then be applied with much less power, therefore limiting heating problems. We report not only on the thermalization of the mechanical mode itself, but also on the bulk of the mechanical element with the temperature of its constitutive TLSs (Two-Level Systems). Below about 100 mK, the mechanical device starts to be self-driven out-of-equilibrium by a strong stochastic force of unknown origin. We report on this phenomenon with a detailed account of the observed features, which have been confirmed in different laboratories but never documented to our knowledge [24][START_REF] Rocheleau | Quantum-Limited Mechanical Resonator Measurement and Back-Action Cooling to near the Quantum Ground State[END_REF][START_REF] Lehnert | [END_REF]. This thorough description is calling for theoretical understanding, and we speculate on the possible ingredients that could be underlying this effect. We demonstrate that the strongest events can be canceled by applying a DC voltage onto the transmission line. However, for beam-based devices the remaining (small) events seem to limit the measurement capabilities to about 10 mK, roughly the lowest achievable temperature for dilution cryostats. This is actually an order of magnitude better than what has been reported so far for doubly-clamped beams [START_REF] Massel | Multimode circuit optomechanics near the quantum limit[END_REF][START_REF] Teufel | Nanomechanical motion measured with an imprecision below that at the standard quantum limit[END_REF][START_REF] Rocheleau | Preparation and detection of a mechanical resonator near the ground state of motion[END_REF]. Looking also at the spread in the reported thermalization temperatures of drum-like devices, obtained in completely similar conditions, we suspect that this genuine phenomenon should be present in all devices sharing the same microwave readout scheme and constitutive materials, at different levels of expression. In this respect, the present description is extremely relevant for the community, and calls for further experiments on other types of devices below 10 mK.

By carefully characterizing the self-heating due to the continuously injected microwave power, we infer that for the extremely weakly coupled mode used here the technique is suitable down to temperatures of the order of 1 -2 milliKelvin; for much larger couplings (like in drum structures) where less power is needed, in the absence of any uncontrolled intrinsic drive the technique should be functional down to the lowest achievable temperatures. As a comparison, similar experiments in the millikelvin range using optics could only be performed in pulsed mode [START_REF] Hauer | Two-level system damping in a quasi-onedimensional optomechanical resonator[END_REF].

II. EXPERIMENT

The microwave setup that we have chosen for this demonstration is relatively basic, and resembles the one described in Ref. [START_REF] Regal | Measuring nanomechanical motion with a microwave cavity interferometer[END_REF]. Details on the microwave wiring can be found in Appendix A. The measurement is performed in transmission, with a coplanar transmission line coupled on-chip to microfabricated microwave cavities (see Fig. 1 left). The practical aspect of this design is multiplexing: in Fig. 1 one cavity hosts our on-chip thermometer while the others can be used for more complex experiments.

The microwave cavities are realized through laser lithography and RIE (Reactive Ion Etching) of a 120 nm thick layer of niobium (Nb). A typical resonance curve at about ω c /(2π) ≈ 6 GHz is shown in Fig. 1 center, displaying a (bi-directional) coupling rate of κ ext /(2π) ≈ 100 kHz and a total damping rate of about κ tot /(2π) ≈ 150 kHz. The NEMS mechanical element we use as onchip thermometer is made from 80 nm thick high-stress silicon-nitride (SiN, 0.9 GPa), grown on top of silicon. It is a 50 µm long doubly-clamped beam of width 300 nm. It is covered by a 30 nm layer of aluminum (Al), capacitively coupled to the cavity through a 100 nm gap. The aluminum part has been patterned using standard e-beam lithography and lift-off, while the beam was released through RIE etching of the silicon-nitride followed by a selective XeF 2 silicon etching [START_REF] Defoort | Stressed Silicon Nitride Nanomechanical Resonators at Helium Temperatures[END_REF]. The siliconnitride has not been removed below the niobium layer. The mechanical resonance of the first flexural mode we use is shown in Fig. 1 right, around Ω m /(2π) ≈ 4 MHz, with damping rate γ m /(2π) of order 10 Hz. Such a mechanical mode still hosts about n thermal ≈ 6 phonons around 1 mK, large enough to be considered in the classical limit and thus well adapted to mode thermometry.

The microwave setup has been carefully calibrated by electronic means; we therefore quote injected powers in Watts applied on-chip, and detected spectra in Watts/Hz (expressed in number of 6 GHz photons) at the output port of the chip. These units are adapted to both issues of heat leaks (cryogenics) and signal-to-noise (electonics) discussed in this Article. Two setups have been used: a dry commercial BlueFors R (BF) machine for preliminary experiments (base temperature 7.5 mK), and then the home-made Grenoble nuclear adiabatic demagnetization cryostat (operated down to 400 µK) [START_REF] Bäuerle | The new Grenoble 100 microKelvin refrigerator[END_REF]. Particular care has been taken in the construction of the two cryostats' temperature scales, and details are given in Appendix A.

The opto-mechanical interaction arises from the force exerted by light onto movable objects [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]. It corresponds to the transfer of momentum carried by light (i.e. photon particles) to the surfaces on which it reflects. In a cavity design with one movable mirror, the retarded nature of this so-called radiation pressure force (due to the finite lifetime of light inside the cavity) leads to damping or anti-damping of the motion, depending on the frequency detuning of the input light with respect to the cavity [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]. Our moving end mirror is the NEMS beam capacitively coupled to the microwave resonator of Fig. 1 (the cavity) [START_REF] Regal | Measuring nanomechanical motion with a microwave cavity interferometer[END_REF]. The standard schemes we use are illustrated in Fig. 2, with an input wave of power P in at frequency ω c + ∆. We are in the so-called resolved-sideband regime, with γ m , κ tot Ω m [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]. When we pump power in the system exactly at the frequency of the cavity ω c (detuning ∆ = 0), the mechanical motion leads to a phase shift of the light [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]. The Brownian motion of the mechanical element thus imprints two equivalent sidebands in the spectrum that we can measure. For a red-detuned pump (frequency detuning ∆ = -Ω m , Fig. 2), energy quanta from the mechanics (phonons) can be transferred to the optical field (photons). This leads to the well-known sideband cooling technique [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF][START_REF] Jasper Chan | Laser cooling of a nanomechanical oscillator into its quantum ground state[END_REF][START_REF] Teufel | Sideband cooling of micromechanical motion to the quantum ground state[END_REF]. The so-called anti-Stokes sideband peak in the spectrum is favored, and the mechanical mode is damped by light. On the other hand for a blue-detuned pump (∆ = +Ω m , Fig. 2), the optical field generates a parametric instability in the mechanics [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF][START_REF] Marquardt | Dynamical Multistability Induced by Radiation Pressure in High-Finesse Micromechanical Optical Cavities[END_REF]. The Stokes sideband peak is favored, and the mechanics is amplified through anti-damping. Eventually one can reach at high pumping powers the self-oscillation regime [START_REF] Kippenberg | Analysis of Radiation-Pressure Induced Mechanical Oscillation of an Optical Microcavity[END_REF][START_REF] Singh | Optomechanical coupling between a multilayer graphene mechanical resonator and a superconducting microwave cavity[END_REF].

We start by calibrating the optomechanical interaction. The optical damping and anti-damping are linear in applied power P in , see Fig. 3. From a fit [Eq. ( 3) below], we can infer the so-called single photon coupling strength

g 0 = 1 2 ω c 1 C
dC dx x zpf with x zpf the zero-point-motion [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]. This is essentially a geometrical parameter, arising from the modulation dC dx of the microwave mode capacitance C by the beam motion [START_REF] Regal | Measuring nanomechanical motion with a microwave cavity interferometer[END_REF]. We find g 0 /(2π) ≈ 0.55±0.1 Hz, corresponding to the out-of-plane flexure. This coupling is particularly small, the idea being to take advantage of that to demonstrate the sensitivity of our method. The magnitude of the output power is fit to theory [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF], leading to a calibration of the measured phonon mode population/temperature [performed at 210 mK, parameter M in Eq. (1) and Fig. 14]. More details on the optomechanics measurements can be found in Appendix B.

III. METHOD

The method we propose builds on the parametric instability of the blue-detuned pumping scheme. When the pump tone is applied at ω c , the size of the two equivalent sideband peaks (their measured area A 0 , in photons/s) is simply proportional to injected power P in and mode temperature T mode [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]:

A 0 = M P in T mode . (1)
This optomechanics scheme alters neither the measured position of the sideband peaks (detuned by ±Ω m [T beam ]), nor their linewidth γ m (T beam ): both are determined by mechanical properties, which depend on the beam temperature T beam . The lineshapes are Lorentzian. We introduce the number of stored photons in the cavity n cav , function of both P in and ∆:

n cav (P in ) = P in κ ext /2 (ω c + ∆) (∆ 2 + κ 2 tot /4) . (2)
On the other hand for blue-detuned pumping, as we increase the injected power P in (but keep it below the FIG. 4: Main: gain of the parametric amplification method based on the blue-detuned pumping scheme (210 mK data, brown squares), as a function of Pin. As in Fig. 3, the arrow indicates the position of the threshold P thr towards selfsustained oscillations. Inset: resonance line (blue trace) measured at very large gains, demonstrating its lorentzian lineshape (linewidth of order 0.9 Hz, close to instability). The black lines are fits, and we report about 20 dB amplification of the Brownian signal. Finite error from both statistics and fluctuations in mechanical parameters (see text).

instability threshold), the area A of the Stokes peak is amplified. The blue/red-detuned pumping expressions write [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]:

γ eff (P in ) = γ m -Sign (∆) 4g 2 0 n cav (P in ) κ tot , (3)
A = A 0 × γ m γ eff (P in ) , (4)
in the limit of negligible cavity thermal population. For ∆ > 0, the last term in Eq. ( 4) after the × sign is a gain, illustrated in Fig. 4. It arises from the anti-damping, with γ eff the linewidth of the Lorentzian peak. Controlling the applied power P in , from the knowledge of system parameters one can straightforwardly recalculate the value of A 0 , and thus of T mode (i.e. the temperature of the mode in absence of optomechanical pumping). In Fig. 4 we demonstrate 18.5 dB gain, which is greater than the previously reported maximum for a similar setup using a graphene device [START_REF] Singh | Optomechanical coupling between a multilayer graphene mechanical resonator and a superconducting microwave cavity[END_REF]. Essentially only γ m depends on temperature, and has to be known to apply Eq. ( 4). It can be obtained easily from a measurement of the mechanical effective damping (the linewidth of the Lorentzian Stokes peak, Fig. 3), by either extrapolating to P in → 0 or defining the position of the threshold P thr ∝ γ m [with Eq. ( 3) at γ eff = 0, see Figs. 3, 4 and 15]. More details are given in Appendix B. Obviously, the main requirement for this T mode estimate is the stability of experimental parameters. The mechanical mode itself happens to be the limiting element (see Fig. 3 and Fig. 5 below), leading to finite error bars at large gains

1 0 1 0 0 1 0 0 0 1 1 0 L i n e w i d t h ( H z ) T c r y o ( m K ) 1 0 1 0 0 1 0 0 0 -1 0 0 -5 0 0 F r e q . s h i f t ( H z )
FIG. 5: Main: mechanical damping parameter γm as a function of cryostat temperature Tcryo. Inset: resonance frequency shift from Ωm(T * ) of the flexural mode in same conditions. The black lines are fits following the TLS model (α ≈ 0.65, T * ≈ 1.1 K see text). Note the scatter in the data (also in Fig. 3), further discussed in Section V.

in Fig. 4; fluctuations are further discussed in Sections IV and V, see Fig. 8.

The measured mechanical damping rates γ m and resonance frequencies Ω m are shown in Fig. 5. The displayed dependencies are characteristic of NEMS devices in the millikelvin range: a damping γ m ∝ T α with 0.3 < α < 2.5 and a logarithmic frequency shift ∝ ln (T /T * ) with T * a characteristic temperature (see fits in Fig. 5). For all materials (from monocrystalline to amorphous) this behavior is understood as a signature of TLSs (Two-Level Systems) [START_REF] Hauer | Two-level system damping in a quasi-onedimensional optomechanical resonator[END_REF][START_REF] Mohanty | Intrinsic dissipation in high-frequency micromechanical resonators[END_REF][START_REF] Hoehne | Damping in high-frequency metallic nanomechanical resonators[END_REF][START_REF] Venkatesan | Dissipation due to tunneling two-level systems in gold nanomechanical resonators[END_REF][START_REF] Lulla | Evidence for the role of normal-state electrons in nanoelectromechanical damping mechanisms at very low temperatures[END_REF][START_REF] Hamoumi | Microscopic Nanomechanical Dissipation in Gallium Arsenide Resonators[END_REF][START_REF] Maccabe | Phononic bandgap nano-acoustic cavity with ultralong phonon lifetime[END_REF]: either defects (e.g. for monocrystalline Si, or polycrystalline Al), or constitutive of the atomic arrangement (for amorphous SiN). Direct coupling of the first flexural mode to the phonon bath (i.e. clamping losses) [START_REF] Cross | Elastic wave transmission at an abrupt junction in a thin plate with application to heat transport and vibrations in mesoscopic systems[END_REF] is negligible for these structures at millikelvin temperatures.

Within the TLS model the mechanical mode is coupled to the two level systems, which are themselves coupled to the external bath: the electrons and the (thermal) phonons present in the moving structure [START_REF] Lulla | Evidence for the role of normal-state electrons in nanoelectromechanical damping mechanisms at very low temperatures[END_REF]. For superconducting materials, the electronic contribution is negligible and the TLSs temperature should reflect the phononic temperature in the beam (i.e. the temperature of high frequency modes well-coupled to the clamping ends), which we simply define as T beam . By inverting the fits in Fig. 5 it is thus straightforward to extract T beam .

The aim of our work is thus to compare the temperature of the cryostat T cryo to T beam and T mode . These results are analyzed in Section IV; however it is mandatory to quantify beforehand the impact of the microwave pump power on the defined temperatures. For this purpose we use the "in-cavity" pumping scheme (Fig. 2). We measure, at a given temperature T cryo , the mechanical characteristics γ m , Ω m and the area A 0 of the two 0 . 0 2 . 0 x 1 0 F i t t e m p e r a t u r e ( m K )

n c a v ( p h o t o n s ) 1 0 1 0 0 1 0 0 0 1 0 -6 σ ( m K / p h o t . ) T c r y o ( m K )
FIG. 6: Main: microwave heating as a function of ncav performed at 210 mK, with temperatures recalculated from the measured width and position (corresponding to T beam ) and peak area (T mode ). The line is a linear fit, leading to the microwave-heating coefficient σ. Inset: heating coefficient σ versus cryostat temperature Tcryo; the dashed line is a power law guide to the eye.

sideband peaks as a function of injected microwave power P in . Using respectively the fits of Fig. 5 and Eq.

(1), we can recalculate the expected temperatures T beam and T mode under microwave irradiation. Since the local heating should be proportional to the local electric field squared confined onto the NEMS, we discuss these results as a function of n cav . The properties of the cavity itself as a function of the pump settings are discussed in Appendix B.

A typical result obtained at 210 mK is shown in Fig. 6 (main graph). Both T beam (obtained equivalently from damping and frequency shift) and T mode display the same linear dependence on n cav , and the two sidebands are equivalent: this demonstrates that the effect is indeed thermal. Defining the slope of the fit as σ, we can extract this coefficient as a function of T cryo (Fig. 6 inset). This temperature-dependence is non-trivial, and no heating model is provided here: such a model should take into account the microwave absorption in the materials, the energy flow in the beam plus the clamping zone slab (suspended by the fabrication undercut), and finally the anchoring to the bulk of the chip. Nonetheless, we can use this graph to estimate the NEMS heating for a given T cryo and n cav in the blue-detuned pumped scheme. As a result, we extrapolate that applying a power of order P thr at 1 mK would heat the beam by about 1 mK; above 10 mK, the heating is essentially negligible (see error bars on T axis of Fig. 15). Knowing the smallness of the coupling g 0 employed here, this demonstrates the capabilities of the method. Furthermore, because of this microwave-heating it is obviously meaningless to report experiments below about T cryo ≈ 1 mK for this first "ultimate" cooling attempt.

IV. IN-EQUILIBRIUM RESULTS

From fits to Eq. ( 4) of the power-dependent Stokes peak area, we thus extract T mode . Reversing the fits of the mechanical parameters γ m , Ω m (Fig. 5) we obtain T beam . Both are displayed as a function of T cryo in Fig. 7, for our two experimental setups. We demonstrate a thermalization from about 10 mK to 1 K of the mode and of the whole beam; the device is in thermal equilibrium over 2 orders of magnitude in T cryo . Reported lowest thermodynamic temperatures in the literature lie all within the range 10 -30 mK [3-6, 11-14, 34, 37]; however one work reports a potential mode temperature for an Al-drum of order 7 mK, consistent with base temperature of dry dilution cryostats [START_REF] Suh | Mechanically Detecting and Avoiding the Quantum Fluctuations of a Microwave Field[END_REF]. Similarly, a lowest temperature of 7 mK is reported for a gigahertz phononic crystal [START_REF] Maccabe | Phononic bandgap nano-acoustic cavity with ultralong phonon lifetime[END_REF]; but obviously such a mode cannot be used for phonon thermometry at millikelvin temperatures.

As far as thermalization between measured T cryo and T mode , T beam is concerned, Fig. 7 reproduces the stateof-the-art, but does not go yet beyond even though the cryostat cools well below 7 mK. The reason for this is discussed in Section V: below typically 100 mK, the system displays huge amplitude fluctuations which hinder the measurements (Fig. 8). These features have been seen by other groups for beam-based microwave optomechanical devices containing an Aluminum layer, but never reported so far [24][START_REF] Rocheleau | Quantum-Limited Mechanical Resonator Measurement and Back-Action Cooling to near the Quantum Ground State[END_REF][START_REF] Lehnert | [END_REF][START_REF] Barzanjeh | [END_REF]. Until recently, this essentially prevented experiments from being performed on these types of devices below physical temperatures of order 100 mK [START_REF] Massel | Multimode circuit optomechanics near the quantum limit[END_REF][START_REF] Teufel | Nanomechanical motion measured with an imprecision below that at the standard quantum limit[END_REF][START_REF] Rocheleau | Preparation and detection of a mechanical resonator near the ground state of motion[END_REF]; remarkably however, (Al covered) ladder-type Si beams [START_REF] Barzanjeh | Mechanical on-chip microwave circulator[END_REF][START_REF] Barzanjeh | [END_REF] seem to be less susceptible to this problem than simple doubly-clamped beams. On the other hand, large signal fluctuations have not been observed to date for (Al) drum-like structures [24,[START_REF] Lehnert | [END_REF], and do not show up in schemes which do not involve microwaves (e.g. magnetomotive measurements of SiN and Al beams [START_REF] Hoehne | Damping in high-frequency metallic nanomechanical resonators[END_REF][START_REF] Maillet | Stochastic and non-linear processes in nano-electro-mechanical systems[END_REF], or laser-based measurements of Si beams [START_REF] Hauer | Two-level system damping in a quasi-onedimensional optomechanical resonator[END_REF]). This is what enabled nano-mechanical experiments to be conducted at base temperature of dilution cryostats.

Up to now, the only possibility to deal with these large events was post-selection, which is extremely timeconsuming and even stops being useable at all at the lowest temperatures. The origin of this phenomenon remains unknown, and we can only speculate on it in Section V hereafter. Note that there is no evidence of thermal decoupling in Fig. 7.

More conventional frequency Ω m and damping γ m fluctuations [START_REF] Maillet | Measuring frequency fluctuations in nonlinear nanomechanical resonators[END_REF] are also present (see e.g. Fig. 8). These features have been reported for essentially all micro/nano mechanical devices, as soon as they were looked for; their nature also remains unexplained, and their experimental magnitude is much greater than all theoretical expectations [START_REF] Sansa | Frequency fluctuations in silicon nanoresonators[END_REF]. Frequency noise essentially leads to inhomogeneous broadening [START_REF] Maillet | Classical decoherence in a nanomechanical resonator[END_REF]. It does not alter the area A measurement, but does corrupt both frequency and linewidth estimates. This noise comes in with a 1/ftype component [START_REF] Maillet | Measuring frequency fluctuations in nonlinear nanomechanical resonators[END_REF][START_REF] Sansa | Frequency fluctuations in silicon nanoresonators[END_REF], plus telegraph-like jumps [START_REF] Defoort | Non-linear dynamics in nanoelectromechanical systems at low temperatures[END_REF]. It leads to the finite error bars in Fig. 5 inset; below 10 mK, the mechanical parameters Ω m and γ m cannot be measured accurately. Similar damping fluctuations [START_REF] Maillet | Measuring frequency fluctuations in nonlinear nanomechanical resonators[END_REF] are more problematic, since the amplification gain Eq. ( 4) depends on γ m . The error bars of Fig. 5 (main graph) are essentially due to this; they translate into a finite error for the estimate of the gain, Fig. 4, which itself limits the resolution on T mode (Fig. 7, top).

V. UNSTABLE DRIVE FORCE FEATURES

In Fig. 8 we show a typical series of spectral acquisitions as a function of time, around 1 mK. We see very large amplitude fluctuations which start to appear around 100 mK, and get worse for lower temperatures (regardless of the scheme used): the "spikes" grow even larger, but more importantly their occurrence increases.

We studied these events in the whole temperature range accessible to our experiment. Their statistics seems to be rather complex, and shall be the subject of another Article. Key features are summarized in this Section.

For blue-detuned pumping the spikes worsen as pump power increases, while for red-detuned pumping it is the opposite, suggesting that the effective damping of the mode plays an important role. With in-cavity pumping, spiky features are also present at very low powers, but not at high powers when the NEMS physical temperature exceeds about 100 mK. The recorded heights can be as large as equivalent mode temperatures in the Kelvin range. Around 10 -30 mK, post-selection becomes impossible.

With the aim of searching for the origin of this effect, we have characterized it in various situations. We first realized that cycling the system from the lowest temperatures to above 100 mK was producing a sort of "reset". But very quickly (a matter of hours), after cooling down again the large spikes happen to dominate the signal again. We then tried to apply a small magnetic field to the system; this was not very conclusive. However, applying a DC voltage had a drastic effect on these random features. This is illustrated in Fig. 9: with a few volts on the chip's coplanar transmission line all the large features disappear. The averaged signal (deeply buried into the noise) recovers a reasonable Lorentzian lineshape (see fit in Fig. 9), while the shape of the spikes is not resolved (Fig. 8). Details on the DC voltage biasing are given in Appendices A and B.

In discussing the source of this feature, a few com- ments have to be made. What is shown in Fig. 8 is primarily fluctuations of the output optical field. These are detected only on the Stokes and Anti-Stokes peaks, for any of the schemes shown in Fig. 2. Furthermore, the threshold to self-oscillation in the blue-detuned pumping scheme displays a large hysteresis (certainly due to nonlinearities in the system, see Fig. 15, Appendix B). We have noticed that when the microwave power applied (at frequency ω c + Ω m ) lies within this hysteresis, the spiky events seem to be able to trigger the self-oscillation. This would not be possible if the amplitude fluctuations measured were only in the detected signal, at the level of the HEMT. We thus have to conclude that we see genuine mechanical amplitude fluctuations. However, these cannot be due to damping fluctuations alone that could trigger self-oscillations, since we do see the same type of features when pumping red-detuned or in-cavity.

If these fluctuations were due to the input field it-self, from Fig. 6 we would reasonably conclude that the NEMS beam would be heated to rather high temperatures, leading to broad and very shifted in frequency (see Fig. 5) Stokes/Anti-Stokes peaks. This is not compatible with the measurement of Fig. 8. The only reasonable conclusion seems thus to be that we do suffer from a genuine extra stochastic force acting on the mechanical element. This is consistent with a stronger sensitivity to the phenomenon when the effective damping of the mode is small (bue-detuned scheme). Since a DC voltage applied only to the cell can drastically modify the measured features, the source has to be on-chip. Noting furthermore than with an in-cavity pumping, it disappears when the beam temperature exceeds 100 mK, we conclude that it should even be within the mechanical element. But the mechanism remains mysterious: citing only documented effects in other areas of research, is it linked to vortex motion in the superconductor [START_REF] Song | Microwave response of vortices in superconducting thin films of Re and Al[END_REF], trapped charges [START_REF] Zorin | Background charge noise in metallic single-electron tunneling devices[END_REF], adsorbed molecules [START_REF] De Graaf | Suppression of low-frequency charge noise in superconducting resonators by surface spin Comm. and two-level system noise in superconducting microresonators[END_REF] or to atomic-size Two-Level-Systems in dielectrics (beyond the standard friction model) [49]?

The low temperature properties of NEMS are described within the tunneling model of Two-Level-Systems (TLSs): for damping, frequency shifts, and phase fluctuations [START_REF] Hauer | Two-level system damping in a quasi-onedimensional optomechanical resonator[END_REF][START_REF] King | Frequency and phase noise of ultrahigh Q silicon nitride nanomechanical resonators[END_REF][START_REF] Maillet | Measuring frequency fluctuations in nonlinear nanomechanical resonators[END_REF]. It is thus natural to consider strongly coupled individual TLSs as the most probable source of our problems. Besides, while the actual nature of these microscopic defects remains elusive in most systems, they could be generated in many ways beyond the standard atomic configuration argument [START_REF] Bilmes | Electronic Decoherence of Two-Level Systems in a Josephson Junction[END_REF]; an electron tunneling between nearby traps would be a TLS strongly coupled to its electromagnetic environment, among other possibilities [START_REF] De Graaf | Suppression of low-frequency charge noise in superconducting resonators by surface spin Comm. and two-level system noise in superconducting microresonators[END_REF]. For Al-based NEMS, these would create (only a few) defects present in (or on) the Al layer; they should carry a dipole moment, which couples them to the microwave drive as well as to the electric field generated by the applied DC voltage. This field distorts their potentials, such that they could get locked in one state and "freeze". Furthermore, our results seem to be very similar to those of Ref. [START_REF] Nazaretski | Effect of Low-Level Radiation on the Low Temperature Acoustic Behavior of a-SiO2[END_REF] obtained with a macroscopic mechanical glass sample, where "spiky" events were demonstrated to be originating in the interaction with low-level radioactivity (gamma rays). These results suggest a parametric coupling to TLS at Giga-Hertz frequencies mediated by the microwave drive, but were the energy corresponding to the large peaks would be provided by the external radiation.

The reason why the mechanism should be dependent on the low phononic dimensionality or size of the device (typ. width about 100 nm, much smaller than the phonon wavelength at 10 mK) is nontrivial. One simple argument could be that the spring constant of the modes under consideration are very different: about 1 N/m for megahertz beams and 100 N/m for drumheads. This could justify why beam-based structures are more reactive to external force fluctuations; an immediate consequence of this argument is then that membrane-based Al devices are not truly immune to force fluctuations, but are just less sen- sitive: if that should be true, cooling them to low enough temperatures would eventually revive the same features as for beam-based NEMS.

To conclude, let us concentrate on the measurements performed at ultra-low temperatures with a DC voltage bias (of the type of Fig. 9, bottom). Even with the help of the in-built parametric amplification, the signal is very small and requires decent averaging, typically here about 30 minutes for reasonable error bars. Even if the resonance peak is Lorentzian, below typically 20 mK the measured area A does not correspond to the actual cryostat's temperature T cryo : it is always bigger, but the actual value presents large fluctuations in magnitude. This is demonstrated in Fig. 10, with identical measurements performed at 23 mK and 7 mK. What is shown is how the measured area of the Stokes peak evolves over time, performing a sliding average over the whole set of acquired data. In the former case, we see that the fluctuations of the measured area are not more than about +60 %; they are much smaller for higher temperatures, leading to proper estimates of T mode . However, for the latter these are greater than 300 %. Besides, fluctuations happen to have an extremely slow dynamics: while spikes switch on/off faster than our acquisition time, their overall occurrence fluctuates over a day (Fig. 10). By no means could this behavior be explained by a thermal decoupling of the device from the cryostat. As a consequence, even the calm zones in Fig. 8 are corrupted by the phenomenon shown in Fig. 10. This is essentially why no reliable data could be acquired below 10 mK; but from the DC biasing and the continuous monitoring of the Stokes peak, thermal equilibrium has been demonstrated at about ten times lower temperature than previously reported in microwave doubly-clamped NEMS experiments [START_REF] Massel | Multimode circuit optomechanics near the quantum limit[END_REF][START_REF] Teufel | Nanomechanical motion measured with an imprecision below that at the standard quantum limit[END_REF][START_REF] Rocheleau | Preparation and detection of a mechanical resonator near the ground state of motion[END_REF].

VI. CONCLUSION

We presented measurements of a microwave optomechanical system performed on a nuclear adiabatic demagnetization cryostat, able to reach temperatures well below the 10 mK limit of conventional dilution machines. Relying on a fairly standard microwave wiring and the in-built parametric amplification provided by a blue-detuned pumping, we devised a method providing accurate thermometry of both the mechanical mode and its on-chip environment (the Two-Level Systems to which it couples). The experiment was conducted on a beam Nano-Electro-Mechanical System embedded in an on-chip microwave cavity. The efficiency of the method is demonstrated with a very low opto-mechanical coupling. Thermalization is shown from 10 mK to 1 K with no sign of thermal decoupling.

However at very low temperatures we report strong fluctuations in the signal amplitude which prevented any experiments to be conducted at ultra-low temperatures. These features appear around 100 mK and have been observed in different laboratories, but had never been studied in details so far. We demonstrate the characteristics of these fluctuations, and argue that they are due to an extra stochastic driving force of unknown origin. Microwave irradiation seems to trigger the phenomenon. Applying a DC voltage of a few Volts on-chip cancels the large spiky events, but a small component of this extra random drive persists, with variations over a typical timescale of about a day.

It is unclear if all the fluctuations characteristics present in the devices (amplitude, frequency, damping) are linked to the same underlying mechanism. One could even imagine that temperature-dependent non-linear effects could impact the phonon-photon coupling, beyond the lowest (geometrical) order g 0 . However, it is likely that these effects are present in all experimental systems at different levels of expressions, since all NEMS/MEMS share the same overall characteristics (especially damping, frequency shifts and phase noise typical of Two-Level Systems physics). It is thus tempting to relate this stochastic force to a mechanism mediated by some kind of microscopic TLSs, driven by microwaves but blocked under DC voltage biasing. This stochastic driving force can mimic to some extent a thermal decoupling, and could explain why some drumhead devices in the literature refuse to cool down below typically 20-30 mK. While being a limitation for experimentalists, this phenomenon definitely deserves theoretical investigations. The present work also calls for further experiments at lower temperatures, using other types of devices (e.g. drums). This shall be performed in the framework of the European Microkelvin Platform (EMP) [START_REF] Pickett | The European Microkelvin Platform[END_REF].

Note added in proof: following the present work, a laboration Aalto University and Institut Néel has started with the aim of cooling down a drumhead Al device as low as possible on our adiabatic nuclear demagnetization platform. We have evidence that the same features as for beams are present, at a different level of expression. This shall be published elsewhere.

( †) Corresponding Author: eddy.collin@neel.cnrs.fr APPENDIX A: SETUP Two similar microwave setups have been used in these experiments. Their common features are described in Fig. 11. This wiring is basic and can also be found within the literature [START_REF] Regal | Measuring nanomechanical motion with a microwave cavity interferometer[END_REF]. Essentially, they are built around a cryogenic HEMT (High Electron Mobility Transistor) placed at about 4 K and two circulators mounted on the mixing chamber of the dilution units. On the BlueFors R (BF) machine, the HEMT is a Low Noise Factory R 4-8 GHz bandwidth, with a measured noise of about 3 K (10 photons) at 6 GHz. On the nuclear adiabatic demagnetization cryostat, it is a Caltech 1-12 GHz bandwidth with a measured noise of about 15 K (50 photons at 6 GHz). The (dashed green) boxed component below the HEMT in Fig. 11 represents a power combiner used to realize an opposition line. On the BF setup, this is mandatory to avoid saturation of the cryogenic HEMT from the strong blue-detuned pump tone. On the demag. cryostat, the cryogenic HEMT is linear enough so this protection is not necessary. This choice has been made because of space constrains: feeding an extra microwave opposition line in the nuclear adiabatic demagnetization cryostat would be very demanding. The filtering of the injection lines (DC and microwave) is also described in Fig. 11.

Gains and noise levels of the full chain have been carefully checked with respect to HEMT working point. Besides, each component has been tested at 4 K prior to mounting. The whole setup has then been calibrated, using an Agilent R microwave generator EXG N5173B and an Agilent R spectrum analyser MXA N9020A. The measurements presented in the core of the paper have been realized using a Zurich Instruments R UHFLI lockin detector operating in spectrum mode. The signal is mixed down with a Local Oscillator (LO) and detected at frequency ±Ω m + 2 MHz (the shift avoiding overlap of Stokes/Anti-Stokes signals). The generators used were from Agilent R or Keysight R brands leading to equivalent data quality. The absolute error in the calibrations is estimated at about ±2 dB over the whole set of realized runs; within these error bars, the two cryogenic platforms gave the same quantitative results.

Particular care has been taken in thermalization issues. The experimental cell is thus made of annealed high-purity copper. It is mounted on a cold finger either bolted onto the mixing chamber plate of BF chine, or connected through silver wires to the bottom of the nuclear stage of the demag. cryostat. This stage is of laminar type, about 1 kg of high-quality copper [START_REF] Bäuerle | The new Grenoble 100 microKelvin refrigerator[END_REF]. On the top it is connected through a Lancaster-made Al heat switch [START_REF] Lawson | A simple heat switch for use at millikelvin temperatures[END_REF] to the mixing chamber of the home-made dilution unit. Both dilution cryostats reach base temperatures of order 7-10 mK depending on cooling power settings. The PCB board mounted inside the cell is hollow in its center; the chip is pressed there directly onto the copper block, by means of a copper/indium spring. At very large microwave powers, we do see heating at the intermediate stages where attenuators are anchored. However, the nuclear stage thermometers do not show any heating, even at the highest powers used. F r e q . ( k H z )

FIG.

12: Calibration measurements performed on the 3 He thermometer, from the lowest operated temperature (here 400 µK, raw data fork resonance curve in inset with Lorentzian fit) to about 100 mK. The superfluid transition Tc is clearly visible around 1 mK and can be used as a temperature fixed point; the line is the expected behavior from 3 He viscosity, see text.

Thermometry below typically 30 mK is no easy task; below 10 mK it requires the expertise of ultra-low temperature laboratories (see e.g. EMP partners [START_REF] Pickett | The European Microkelvin Platform[END_REF]). In the graphs of Fig. 7, the x axis is as important as the y one; this point is not always emphasized in the literature. We thus paid particular care in providing an almost primary temperature scale for our two microwave platforms. The use of the word almost here should be understood as follows: we also used resistive thermometers (RuO 2 , carbon Speer-type) calibrated against primary devices, and our primary thermometers (which do follow known temperature dependencies) are calibrated at a single point in temperature for practical reasons. Our laboratory has a long history of working for the construction of the ultra-low temperature scale [START_REF] Rusby | Dissemination of the Ultra-Low Temperature Scale PLTS-2000[END_REF]. In our case, the BF cryostat has been equipped with a Magnicon R MFFT noise thermometer and a CMN paramagnetic salt (Cerium Magnesium Nitrate) thermometer. The MFFT was bolted at mixing chamber level while the CMN was directly mounted on the cold finger. On the demag. cryostat, an MFFT is mounted at the top of the nuclear stage while a 3 He thermometer is connected at the bottom; the exerimental cell is actually between the nuclear stage and this thermometer. When their working range overlap, the thermometers agree within typically 2 -5 %; the error bars on x axes of the temperature plots also take into account thermal gradients and slow drifts.

The nuclear adiabatic demagnetization process requires large magnetic fields to be used (here, 8 T). One usually starts at T ini ≈ 10 mK with the superconduct-ing coil surrounding the stage fully magnetized at B ini (heat switch open), and then demagnetizes to about B f in ≈ 100 mT to reach the lowest possible temperatures (in the present runs, we worked down to 400 µK, Fig. 12). The cryostat can then stay cold over a week (heat leak < 100 pW); another cycle needs then to be initiated with a pre-cooling of the system. The process being adiabatic, the nuclear spins of copper are cooled according to B ini /B f in = T ini /T f in [START_REF] Pobell | Matter and methods at low temperatures[END_REF]. Our magnet is compensated on both sides, but nonetheless small stray fields are present when it is magnetized (represented in Fig. 11 by the small boxed coil). In order to make sure that these fields do not introduce any slant in the data, we performed measurements above 10 mK but within a demagnetization cycle: we started at 20 mK insead of 10 mK, and reduced the field by 1/2. This can then be compared to experiments performed without any field, at base temperature of the dilution unit. The microwave cavity does shift in frequency with applied field (see Appendix B), but no other difference could be found. All data taken below 10 mK have been obtained exclusively with the nuclear adiabatic demagnetization technique.

The lowest part of our temperature scale is obtained from a 3 He thermometer (see Fig. 12). It is based on the viscosity measured in the fluid with an immersed probe, a mechanical resonator. In the past, vibrating wires were the best choice for this task [START_REF] Guénault | The mechanical behavior of a vibrating wire in superfluid 3 He-B in the ballistic limit[END_REF][START_REF] Winkelmann | Vibrating wire thermometry in superfluid 3 He[END_REF]. Today, people use quartz tuning forks which are more practical [START_REF] Blaauwgeers | Quartz Tuning Fork: Thermometer, Pressureand Viscometer for Helium Liquids[END_REF][START_REF] Bradley | Thermometry in Normal Liquid He-3 Using a Quartz Tuning Fork Viscometer[END_REF][START_REF] Bradley | The damping of a quartz tuning fork in superfluid He-3-B at low temperatures[END_REF]. Our thermometer is thus a nested cell containing two tuning forks (one in the outside cell, the other in the inside), filled with silver sinters connected to the nuclear stage by silver wires. The outside cell serves as a thermal shield for the inner one, and does not cool down below typically 1 mK. The inner cell is directly connected to the cold finger that hosts the microwave cell.

3 He is a Fermi liquid above about 1 mK, and becomes superfluid below. At 0 bar pressure, 0 T field this state is called 3 He-B, and the properties of this amazing fluid have been extensively studied over the years [START_REF] Vollhardt | The Superfluid Phases of Helium 3[END_REF]. For instance, the viscosity of 3 He is well known for both normal and superfluid states, see e.g. [START_REF] Carless | Vibrating wire measurements in liquid 3 He. I. The normal state[END_REF][START_REF] Carless | Vibrating wire measurements in liquid 3 He. II. The superfluid B phase[END_REF]. In principle, from the knowledge of the geometry of the immersed object and its surface rugosity, one can calculate the friction and thus the broadening of the resonance. In practice, it is much more efficient to calibrate the device by scaling its properties on known measurements [START_REF] Bradley | Thermometry in Normal Liquid He-3 Using a Quartz Tuning Fork Viscometer[END_REF][START_REF] Bradley | The damping of a quartz tuning fork in superfluid He-3-B at low temperatures[END_REF].

In order to perform this scaling, we measured the damping of the quartz tuning fork thermometer (inner cell) as a function of the final field of the demagnetization process B f in . The protocol was to demagnetize by steps, and then remagnetize to B ini in order to verify that the process was indeed adiabatic (by recovering the initial temperature T ini ). It is then straightforward to calculate the temperature of the nuclear spins of copper for each step, which should be in equilibrium with electrons (and finally 3 He) if one waits long enough. The result is shown in Fig. 12. The line is the expected behavior from published results. Note that the superfluid transition T c also acts as a fixed point in the temperature scale.

APPENDIX B: MICROWAVE OPTOMECHANICS

The microwave cavity is the central element of the technique. It happens to be very sensitive to external conditions: cell temperature, pump settings (power, detuning), DC voltage bias and magnetic field. In order to avoid any systematic error, the protocol is thus to measure the cavity for each setting. On the BF cryostat, in order to mimic stray fields from the demagnetization protocol and to study the impact of (small) magnetic fields on the unstable features reported in Section V, a small coil has been installed on top of the cell (boxed element in Fig. 11). It can create perpendicular fields up to about 10 µT. The voltage DC bias is applied only to the cell which is placed within a Bias Tee and a DC Block (see boxed elements in Fig. 11). Above about 4 V, the chip starts to heat (certainly because of currents flowing within the silicon). All reported measurements obtained with a DC bias have thus been acquired at 3 V DC, ramping slowly periodically (every 1/2 hour approximately) the voltage to almost 4 V and back again. At zero bias, after some time the "spikes" eventually reappear. They did not seem to react particularly to the small magnetic fields used. FIG. 13: Photon flux due to occupation of the cavity mode in excess of that expected from the cryostat temperature. The x axis is √ ncav (blue-detuned pump scheme for blue symbols, red-detuned pump for red). Inset: example of resonance peak obtained from spectrum analyzer. The lines are guides to the eye. The numbers in magenta correspond to the calculated intra-cavity population for the last point of each temperature series (from fit value of κext, see text).

The cavity parameters are extracted from a transmission response measurement (amplitude of S 21 component of [S] matrix). This is measured by applying a very weak probe tone (see boxed element in Fig. 11) while keeping the strong pump on. We verified that the probe power was small enough not to alter the measurement. The data (see Fig. 1) is fit to the expression [START_REF] Khalil | An analysis method for asymmetric resonator transmission applied to superconducting devices[END_REF]: The dashed line corresponds to the heating measured in Fig. 6.

(S 12 [ω]) dB = 20 log 1 - Q tot (Q ext + IQ im ) -1 1 + 2IQ tot ω-ωc ωc , (
The fits are always of good quality. κ ext is very stable and reproducible, and does not vary by more than about ±5 %. On the other hand κ tot = κ ext + κ in varies between typically 120 kHz and 190 kHz from one cool-down to the other. κ in represents the internal decay processes, taking into account all microscopic mechanisms. At low pump powers, κ tot happens to be worse (i.e. larger) than at high powers. This is true up to some maximum above which we start to substantially degrade the cavity's properties; in terms of n cav this limit lies in the range of 10 8 -10 9 photons. In the range studied the temperature dependence of κ tot is rather small. It is immune to the magnetic fields and voltages applied within the parameter range studied. However, the resonance position of the cavity is very sensitive to all parameters: so the main issue of the fit is to define ω c as accurately as possible.

The properties of the cavity are too complex to allow an analysis of the type of Fig. 6 (performed for the NEMS), which could tell us by how much the microwave mode (and/or the chip) heats at a given pump power. We therefore measured directly the thermal population of the cavity with respect to P in . We present these data in Fig. 13 as a function of √ n cav : on pure phenomenological grounds, the dependence seems then to be linear (see guides for the eyes in Fig. 13). We could not perform this measurement below typically 400 mK because the signal was too weak. The power-dependence is rather different from Fig. 6 and does not seem to be a true heating of the chip itself; it has thus to be noise fed into the cavity by the pump generator. Indeed, similar features but with different levels of cavity populations have been measured using different brands of generators. The relevant outcome of this graph is the extra (out-of-equilibrium) cavity population induced by the strong pump; this is the number quoted in magenta in Fig. 13 for each temperature, at the largest powers displayed. This number is obtained by dividing the photon flux by κ ext /2 (bi-directional coupling). Injecting it in the theoretical expressions [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF], we find out that this effect remains always negligible with respect to other problems creating the finite error bars of Fig. 7.

From the careful measurement of the cavity at each point, the knowledge of the heating effects in Figs. 6 and 13 we can guarantee that the experiment is performed in the best possible conditions. To be then quantitative, we finally need to be able to convert photon population in the Stokes/Anti-Stokes peaks into phonon populations. To do so, we need to know the parameter M introduced in Eq. [START_REF] Pickett | The European Microkelvin Platform[END_REF]. In principle, one could calculate it from theory, knowing all experimental details of the setup [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF]. However, since all of these parameters are known within some experimental error, the final value obtained for M would be of poor precision. It is thus much more efficient to calibrate it at a given temperature: this is performed in Fig. 14, at sufficiently high T cryo to guarantee a good thermal coupling and enough resolution in signal (210 mK). In this plot we show the signal (area) obtained for the 3 schemes presented in Fig. 2, as a function of P in (they thus all overlap at low powers). The lines are fit to theory Eqs. [START_REF] O'connell | Quantum ground state and singlephonon control of a mechanical resonator[END_REF][START_REF] Palomaki | Coherent state transfer between itinerant microwave fields and a mechanical oscillator[END_REF] [START_REF] Aspelmeyer | Florian Marquardt, Cavity optomechanics[END_REF], with the dashed one taking into account the heating produced by the large n cav reached with the "in-cavity" scheme. We thus obtain M ≈ 1.8 × 10 12 photons/s/W/K for our device. Note that at the highest powers used, in the reddetuned pumping scheme the mode cooled from 210 mK down to approximately 20 mK.

Our aim was to demonstrate the capabilities of the method for temperatures as low as possible and a coupling g 0 which is not especially large. We therefore relied on the out-of-plane flexure of our NEMS beam; in-plane and out-of-plane modes were both characterized first at room temperature using optics [START_REF]A room-temperature vacuum chamber equipped with a laser readout from O. Arcizet and B[END_REF]. The out-of-plane flexural mode was found at 3.660 MHz, shifted about 280 kHz from the in-plane one. Both had a quality factor of about 6 500. At low temperatures, the method then relies on the blue-detuned pump instability: below the threshold it enables amplification of the weak signal, and the position of the threshold itself gives access to the parameter γ m needed for quantitative fits. It is thus in FIG. 15: Main: Power threshold P thr to self-oscillation regime (blue-detuned pump scheme). A large hysteresis is seen, certainly due to nonlinearities in the system. The black line is the calculated value ∝ γm from Eq. ( 3), and the dashed line is a guide to the eyes ∝ γ 2 m . Inset: peak height measured sweeping the power up (full black), and down (empty blue symbols) at 210 mK. The dashed verticals are the threshold positions.

principle not necessary to be able to resolve the signal at low powers; in particular, in these conditions sidebandassymetry thermometry would be impractical.

The definition of γ m from the threshold position is illustrated in Fig. 15. Ramping up the power, from Eq.

(3) we can recalculate the damping rate from the position of the threshold P thr (full symbols and full line). Furthermore, it happens that this threshold towards selfsustained oscillations is hysteretic (Fig. 15 empty symbols). This effect is not yet modeled, but it clearly arises from non-linearities in the system. For microwave optomechanics, this are genuine geometrical nonlinearites, as opposed to thermal nonlinearities seen in optics [START_REF] Baker | Optical Instability and Self-Pulsing in Silicon Nitride Whispering Gallery Resonators[END_REF]. The temperature-dependence of the down-sweep threshold (large amplitude motion) is even stronger than for the up-sweep one (small amplitude). Again, no thermal decoupling can be seen down to 10 mK; the heating effect (Fig. 6) has been taken into account, and has only a small impact on the data.

FIG. 1 :

 1 FIG. 1: Chip arrangement. (a): chip before bonding in PCB (microwave circuit board), displaying a coplanar transmission line coupled to 3 microwave cavities. (b): cavity and resonance [data violet squares, black line is a fit to Eq. (B1)] at 210 mK with blue-detuned pump power 7 nW and probe power 25 fW. (c): NEMS beam structure and resonance spectrum measured on the Stokes peak for blue-detuned pump power 0.5 nW at same temperature (negligible optomechanical back-action, data blue line and black line is a Lorentzian fit with Ωm ≈ 2π × 3.79 MHz, see Appendix B for details).

  q . f r o m Ω m / 2 π ( H z )

  k e s , f r o m w i d t h S t o k e s , f r o m a r e a S t o k e s , f r o m f r e q . A n t i -S t o k e s , f r o m w i d t h A n t i -S t o k e s , f r o m a r e a A n t i -S t o k e s , f r o m f r e q .

  FIG. 7: (a): mode temperature T mode as a function of cryostat temperature Tcryo. (b): beam temperature T beam inferred from the TLS bath as a function of Tcryo. The conditions of the measurements are defined in the legend. The line is the y = x function.

Fr 6 FIG. 8 :

 68 FIG.8: Stokes resonance peak (amplitude in color scale, frequency from Ωm on the left) as a function of time at about 1 mK (0.6 nW applied power, blue-detuned pumping scheme). Huge amplitude jumps are seen, together with frequency (and damping) fluctuations (see text).

FIG. 9 :

 9 FIG.9: Comparing measurements with/without applied DC voltage in similar conditions. (a): resonance lines (blue) obtained at 5 mK with 0.6 nW drive; 2 hours acquisition shown in 560 averaged traces, with no DC voltage bias. (b): resonance lines obtained at 7 mK with 0.8 nW drive; 18 hours in 770 averaged traces, with +3 V DC applied on the transmission line (see text). The scheme used for both data sets is blue-detuned pumping, and the thick black line is a fit of the average curve (in red). Note the 10 4 difference in the vertical axes.

  FIG.10:(a): area of peak extracted from a sliding average performed with a window of 26 minutes at 23 mK, with applied power 0.8 nW (blue-detuned pumping). (b): same measurement performed at 7 mK. The horizontal dashed lines are the thermal population expected values, matched at 23 mK in the stable zone (middle of graph). At the lowest temperatures, we observe very large amplitude fluctuations which cannot be of thermal origin; the measured area remains always larger than the expected value (see text).

FIG. 11 :

 11 FIG. 11: Simplified common wiring of the experimental platforms; the different levels within the cryostats (BF and demag.) are shown with their respective temperature. SS stands for Stainless-Steel, NbTi for Niobium-Titanium and Cu for Copper coaxial cables (50 Ohms impedance). The boxed elements have been added/removed depending on the experimental run (see text for details).

  k l i n e w i d t h ( H z ) T e m p e r a t u r e ( m K )

  B1)with I the pure imaginary, Q tot = ω c /κ tot and Q ext = ω c /κ ext . Q im corresponds to an imaginary component of the external impedance, usually attributed to the inductive bonding wires (leading to the asymmetry of the fit in Fig.1 center).

FIG. 14 :

 14 FIG.14:Measured signal amplitude for the 3 schemes used at 210 mK: blue-detuned pump, red-detuned pump and "incavity" (green, the square and circle symbols stand for Stokes and Anti-Stokes peaks respectively). For blue/red pumping, the fits correspond to Eqs. (1-4), defining the coefficient M. The dashed line corresponds to the heating measured in Fig.6.

  g h t ( p h o t o n s / s ) P i n ( W )

Acknowledgments

We acknowledge the use of the Néel facility Nanofab for the device fabrication, and the Néel Cryogenics facility with especially Anne Gerardin for realization of mechanical elements of the demagnetization cryostat. X.Z. and E.C. would like to thank Olivier Arcizet and Benjamin Pigeau for help in room-temperature characterization of the NEMS devices using optics; E.C. would like also to thank O. Arcizet, M. Sillanpää, K. Lehnert, J. Teufel, S. Barzanjeh, K. Schwab, J. Parpia and M. Dykman for very useful discussions. We acknowledge support from the ERC CoG grant ULT-NEMS No. 647917, StG grant UNIGLASS No. 714692 and the STaRS-MOC project from Région Hauts-de-France. The research leading to these results has received funding from the European Union's Horizon 2020 Research and Innovation Programme, under grant agreement No. 824109, the European Microkelvin Platform (EMP).


Jennifer Vandoni

Emanuel Aldea

Sylvie Le Hégarat-Mascle

EVALUATING CROWD DENSITY ESTIMATORS VIA THEIR UNCERTAINTY BOUNDS

Keywords: density estimation, crowd counting, multi-scale evaluation, uncertainty bounds Layers -part 1 FE Conv

In this work, we use the Belief Function Theory which extends the probabilistic framework in order to provide uncertainty bounds to different categories of crowd density estimators. Our method allows us to compare the multi-scale performance of the estimators, and also to characterize their reliability for crowd monitoring applications requiring varying degrees of prudence.

INTRODUCTION

Understanding crowd systems and predicting their evolution is of paramount importance when considering the world population growth and urbanization rates. One of the reasons for which the urban infrastructure sector has not fully taken advantage of vast available video data is the difficulty to extract accurately microscopic and macroscopic observations in high-density conditions. Although it does not require accurate target localization, density estimation inside crowds is still a challenging problem, due to phenomena such as strong occlusion and visual homogeneity. However, deep learning advancements significantly improved the state-of-the-art performance (see [START_REF] Vishwanath | A survey of recent advances in cnn-based single image crowd counting and density estimation[END_REF] for a comprehensive survey). Recent methods are mostly based on the estimation of a density map whose integral over a region provides the number of people within it, in such a way to incorporate spatial information directly into the learning process (e.g. MCNN [START_REF] Zhang | Single-image crowd counting via multi-column convolutional neural network[END_REF], Cascaded-MTL [START_REF] Vishwanath | Cnn-based cascaded multi-task learning of high-level prior and density estimation for crowd counting[END_REF], Hydra CNN [START_REF] Onoro | Towards perspective-free object counting with deep learning[END_REF], CSRNet [START_REF] Li | Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes[END_REF]). In the vast majority of the proposed works, the estimator evaluation is performed at image scale, with error metrics such as MAE or MSE [START_REF] Vishwanath | A survey of recent advances in cnn-based single image crowd counting and density estimation[END_REF]. However, from the point of view of modelling the crowd as a dynamical system, accurate local densities are required in order to characterize wave-like propagation phenomena. The error metrics above are related to large scale statistics, which do not apply to small scales due to compensation between overestimating and underestimating the density in different areas. An additional limitation of current density estimators is the absence of an uncertainty range provided along with the scalar density. Ranges on the pedestrian count are greatly needed, as the trade-off between safety concerns and optimal use of This work was funded by French ANR grant ANR-15-CE39-0005. infrastructure capacity promotes different levels of congestion in different contexts. In statistical learning theory, the uncertainty of classification or regression processes has been studied based on how models are affected by the skewed distribution and noise of training data, by inaccurate training data labeling or by the regularization policy. SVM output uncertainty has been studied [START_REF] Xu | Evidential calibration of binary svm classifiers[END_REF] due to algorithm's convenient generalization ability coupled to the simple underlying principle. In [START_REF] Gal | Dropout as a bayesian approximation: Representing model uncertainty in deep learning[END_REF] a new strategy was introduced for addressing the epistemic uncertainty estimation in Deep Neural Networks by approximating the probabilistic output distribution using dropout during inference.

In this line of research, we propose a generic approach for evaluating the uncertainty of the output of a crowd density estimator. As a second contribution, we apply the proposed evaluation on a multiscale domain derived from the image lattice, which allows us to characterize the estimator performance locally as well. We show that we are able to compare different learning algorithms across the scale space and to provide density estimations with bounded uncertainties.

EVIDENTIAL CNN-ENSEMBLE

FE+LFE network. Following recent advancements on density estimation [START_REF] Li | Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes[END_REF], we propose a fully convolutional network which makes use of dilated convolutions instead of pooling layers, in order to preserve the output resolution in presence of small targets. However, as highlighted in [START_REF] Hamaguchi | Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery[END_REF], aggressively increasing dilation factors through the network layers is detrimental in aggregating local features. By taking inspiration from this latter work we propose a network which is composed of two parts, i.e. a front end (FE) module with increasing dilation factors to consider larger context around small objects, and a local feature extractor (LFE) module with decreasing dilation factors to enforce the spatial consistency of the output by gathering spatial information. Moreover, unlike [START_REF] Hamaguchi | Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery[END_REF], we add batch normalization before ReLU activation functions for faster convergence. The structure of the proposed FE+LFE network is detailed in Table 1. The number of filters per layer is kept small to avoid overfitting since we intend to be able to train the network with relatively small datasets. Note that we employ a ReLU activation function also after the last layer. This has the effect of a zero-threshold; nevertheless, it has beneficial effects on backpropagation with Table 1: Architecture of the FE+LFE network. F is the number of filters and D is the dilation factor of dilated convolutions. Each convolutional layer is followed by batch normalization (except for the last one) and ReLU activation function.

respect to a simple post-processing thresholding. The local density estimation is therefore enhanced, since the network loses its tendency to add noise to compensate between low and high values.

Finally, a L2 loss function is used between the estimated density map and the ground-truth derived by placing a Gaussian on each head center as in [START_REF] Zhang | Single-image crowd counting via multi-column convolutional neural network[END_REF]. Since we know the geometry of the scene, we apply perspective correction as in [START_REF] Vandoni | Active learning for highdensity crowd count regression[END_REF] instead of geometry-adaptive kernels. Building a CNN-ensemble. Recently ensemble techniques have been successfully exploited by the deep learning community since they allows for more robust predictions as well as for a measure of predictive uncertainty, i.e. the confidence of the network with respect to its prediction (which in our case represents the likelihood of head presence). In the context of Bayesian Neural Networks (BNNs), the authors of [START_REF] Gal | Dropout as a bayesian approximation: Representing model uncertainty in deep learning[END_REF] developed a new theoretical framework called MC-dropout casting dropout [START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF] as approximate Bayesian inference in Gaussian processes. This method overcomes the major limitations of BNNs that generally require prohibitive computational costs [START_REF] Radford | Bayesian learning for neural networks[END_REF][START_REF] Diederik | Auto-encoding variational bayes[END_REF][START_REF] Blundell | Weight uncertainty in neural networks[END_REF]. In [START_REF] Gal | Dropout as a bayesian approximation: Representing model uncertainty in deep learning[END_REF] instead, after training the network, Monte Carlo (MC) methods are used at inference time to draw samples from a Bernoulli distribution across the network weights, by performing T stochastic forward passes through the network with dropout. The ensemble is thus composed by T different realizations given by dropping out different units of the network at each forward pass. Another ensemble approach (although non-Bayesian) has been recently proposed in [START_REF] Balaji Lakshminarayanan | Simple and scalable predictive uncertainty estimation using deep ensembles[END_REF], where a deep ensemble is derived by training the same network on the same data but with different random weight initializations. Compared to MC-dropout, this method has nonetheless the immediate drawback of requiring multiple training of the network.

In this work, we derive a CNN-ensemble relying on MC-dropout. We train the network once and then we sample the posterior distribution over the weights using dropout at inference time, obtaining T different realization maps M1 , . . . , MT , outputs of different dropout-perturbed versions of the original network. Classically, the mean map M µ , given by the mean value evaluated independently for each pixel, would be interpreted as the final prediction map, while the standard deviation map M σ would be interpreted as an estimate of the predictive uncertainty. However, we propose to work in the Belief Function (BF) framework [START_REF] Shafer | A mathematical theory of evidence[END_REF][START_REF] Smets | The transferable belief model[END_REF], that we consider more suited to model the specific imprecision of each different realization obtained with dropout, allowing us to derive the uncertainty bounds for density estimation.

Modeling imprecision with BFT. To handle both the uncertainty provided by the classification and the related imprecision that may exist due to the specific classifier and/or data used in the training process, Belief Function Theory (BFT), also called evidential theory, is designed to handle a larger hypothesis set than the probabilistic one. Denoting by Θ the discernment frame, i.e. the set of mutually exclusive hypotheses of cardinality |Θ|, belief functions are defined on the powerset 2 Θ . In our setting, denoting by H and H the two mutually exclusive (singleton) hypotheses "Head" and "Not Head", the discernment frame is Θ = H, H while

2 Θ = ∅, H, H, H, H . Classically, the mass function m is the Basic Belief Assignment (BBA) that satisfies ∀A ∈ 2 Θ , m(A) ∈ [0, 1], A∈2 Θ m(A) = 1.
The hypotheses associated to non-null mass functions are called focal elements.

BBAs that have only singleton hypotheses as focal elements are called Bayesian BBAs. Now, we want to consider the imprecision that possibly arises performing inference on unknown images with a model learned by a neural network, by modelling the pixelwise classification outputs as BBAs. We therefore exploit the the CNN-ensemble composed by the T realizations obtained with MC-dropout. Firstly, we derive Bayesian BBA maps M B 1 , . . . , M B T , where a BBA is associated to each pixel x of every realization, so that we obtain T maps of BBAs m B

x,t x∈P , where P is the pixel domain and t ∈ {1, . . . , T }. These Bayesian BBA maps are 4-layer images where each layer corresponds to the mass value of any hypothesis in ∅, H, H, Θ respectively. For example, M B t (A) corresponds to the layer image associated to hypothesis A for the realization (source) t. In this preliminary Bayesian BBA allocation, layer images corresponding to non-singleton hypotheses are null by definition, whereas for each source t, with t = 1, . . . , T : M B t (H) = Mt , and M B t (H) = 1 -Mt . In order to account for the reliability of the pixel-wise prediction given by every source, we perform a pixel-wise tailored discounting, namely a generalization of each BBA on the basis of its reliability [START_REF] Shafer | A mathematical theory of evidence[END_REF]. To evaluate this latter, noting that the median has been shown to be a more robust estimator than the average in presence of outliers, for each source t we compute a discounting coefficient map Γ t : {γ x,t } x∈P such that a different coefficient γ x,t is associated to every pixel of each source,

Γ t = α 1 - Mt -median M T 1 . (1)
In this way, we discount more pixels whose value is more distant to the median value among the T realizations, since they are supposed to be less representative (even possibly outliers). The α parameter is a scaling factor which allows us to control the amount of discounting. Applying the proposed discounting, we derive the following BBAs map for every source t:

∀A ∈ H, H ,    M t (∅) = {0} x∈P , M t (A) = Γ t M B t (A), M t (Θ) = {1} x∈P -M t (H) -M t (H), (2)
where M 1 M 2 represents the Hadamard product between matrices M 1 and M 2 .

To combine the T different maps to obtain a single output map M with BBAs associated to each pixel x, i.e. {m x } x∈P , we use the conjunctive combination rule [START_REF] Smets | The transferable belief model[END_REF]. In our case where |Θ| = 2, the analytic result may be easily derived:

∀A ∈ H, H ,          m x (A) = (B1,...,B T )∈{A,Θ} T , ∃t∈[1,T ]s.t.Bt=A T t=1 m x,t (B t ) , m x (Θ) = T t=1 m x,t (Θ) , m x (∅) = 1 -m x (H) -m x H -m x (Θ) . ( 3
)
The result is thus a four-layer map M of BBAs m x , that can be used to derive evidential measures of uncertainty about the network prediction. To this extent, we can obtain the ignorance map as M(Θ), that represents the remaining ignorance which has been decreased by the combination but not completely solved, indicating a lack of sufficient information during training to perform a reliable prediction. Likewise, M(∅) is often interpreted as a conflict map [START_REF] Lachaize | Evidential split-and-merge: Application to object-based image analysis[END_REF], and presents higher values for pixels whose prediction completely disagrees through the various realizations.

Finally, in every pixel x the decision is taken from m x . Pignistic probability [START_REF] Smets | The transferable belief model[END_REF] may be used to give a probabilistic interpretation to the BBAs. Since in our setting

|Θ| = 2, ∀A ∈ H, H , BetP x (A) = 1 1-mx(∅) m x (A) + mx(Θ) 2 .
This allows us to assign a BetP x (H) value to the resulting BBA associated to each pixel x that will be differently normalized on the basis of its conflict value, m x (∅).

Then, other functions are in a one-to-one relationship with m x , and can be used either for decision or for some computations, namely the Plausibility (Pl) and the Belief (Bel) functions. In this particular setting where |Θ| = 2 applying a normalization to the BBAs (so that m x (∅) = 0), they are defined as:

Bel x (A) = 1 1-mx(∅) (m x (A)), and P l x (A) = 1 1-mx(∅) (m x (A) + m x (Θ)
). These functions may also be interpreted as upper and lower probabilities respectively [START_REF] Shafer | A mathematical theory of evidence[END_REF] and they check the duality property: ∀A ∈ 2 Θ , P l x (A) = 1 -Bel x (A) (where A represents the complement of A with respect to Θ).

DENSITY UNCERTAINTY FOR BOUNDING PEDESTRIAN COUNTS

In this work we propose a multiscale evaluation strategy which computes for each considered scale S indicators based on all squared subdomains S ∈ S i . These indicators use the derived upper and lower density bounds s ¯(S), s(S): s ¯(S) = w x∈S Bel x (H) and s(S) = w x∈S P l x (H). The factor w relating the numerical output to the actual pedestrian count is 1 for networks trained on actual density maps, but in the general case it may be determined as in [START_REF] Lempitsky | Learning to count objects in images[END_REF] on a validation set consisting of BetP (H) maps. We then calculate for S i the prediction error probability (PEP) as:

P EP i = {S ∈ S i |g(S) ∈ [s ¯(S), s(S)]} /|S i |, (4)
and the relative imprecision (RI) interval as:

RI i = S∈Si (s(S) -s ¯(S))/g(S) /|S i |, (5)
where g(S) is the ground-truth count over S. In our work, we take S 1 as the set of the largest possible squares which fit the image space, and then we use a scale factor δ to reduce the square side for subsequent scales.

The RI criterion highlights the size of the imprecision interval around the estimated count, while the PEP criterion indicates the error rate of the prediction, namely whether the ground-truth count for the considered region is outside the estimated interval. Thus, a two-axis plot presenting the evolution of RI vs. PEP across multiple scales and for different estimators allows one to compare them and to select an operating point with an explicit uncertainty tied to a desired error rate. In order to compute the values required by Eqs. ( 4) and ( 5), the process may be accelerated significantly by using the Integral Histogram [START_REF] Porikli | Integral histogram: A fast way to extract histograms in cartesian spaces[END_REF] trick, given that the most intensive task is to compute sums over rectangular supports defined in the bounded image space.

EXPERIMENTAL RESULTS

We validated our proposed approach on high-density crowd images acquired at Makkah during Hajj [START_REF] Vandoni | Evidential query-by-committee active learning for pedestrian detection in high-density crowds[END_REF]. Besides evaluating the proposed FE+LFE network, we compared it to U-Net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF], originally introduced for medical image segmentation and very effective even on relatively small training datasets as in our case (35 crowd images). The two networks are trained by using Adam stochastic optimizer with a learning rate of 7×10 -3 (FE+LFE) and of 10 -2 (U-Net). Additionally, we perform data augmentation and early stopping in order to limit overfitting. A CNN-ensemble of size T = 10 is then obtained by applying dropout at inference time in the central layers as in [START_REF] Kendall | Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding[END_REF] with probability p drop = 0.5.

Figures 1a and1b show the results when applying the proposed uncertainty bound evaluation for the FE+LFE and U-Net networks respectively. Ideally, an estimator should predict with a high confidence (low PEP) that the estimated count is within a small RI interval. One may increase the size of the RI interval by decreasing the α parameter in Eq. ( 1), in order to obtain better prediction accuracy (at the expense of a larger RI). We tested different discounting factors, corresponding to horizontally aligned clusters of dots. For each cluster, each dot depicts the performance obtained at a different scale, with a scale factor δ = 1.1, S 1 being the largest scale. Both networks perform better at larger scales, due to error compensation. The proposed FE+LFE network outperforms U-Net, showing the importance of preserving spatial information without pooling operations in presence of small targets, while increasing at the same time the contextual information with dilations.

To stress the independence of the proposed evaluation approach with respect to the classifier used, Fig. 1d shows the results of the density estimation obtained with SVM using active learning (AL) as in [START_REF] Vandoni | Evidential query-by-committee active learning for pedestrian detection in high-density crowds[END_REF], where an SVM-ensemble is built iteratively by training SVMs with different descriptors on selected informative samples. The imprecision derives both from possible errors in the calibration procedure to obtain probability estimates out of SVM scores, and from the score heterogeneity in the image space. Moreover, Fig. 1c shows the results obtained training the proposed FE+LFE network with a smaller amount of data (i.e. the pool of unlabeled samples U available for AL in [START_REF] Vandoni | Evidential query-by-committee active learning for pedestrian detection in high-density crowds[END_REF]). This allows us to perform two different types of analysis. Firstly, we can perform a fairer comparison between the two classifiers. To this extent, we notice that FE+LFE, even when trained on less data, outperforms the SVM-based approach, especially at larger scales. Nonetheless, the two methods exhibit almost identical performance when considering the smaller scales. Secondly, it is interesting to evaluate the same network trained with different amounts of data. According to Figs. 1a and1c, we Fig. 2: Results of the density estimation map (Fig. 2b) along with the estimated uncertainty bounds (Fig. 2c). The input data and the ground truth annotations are shown in Fig. 2a.

see that a larger training set is beneficial for density estimation especially at larger scales. However, considering smaller scales, the performance gap is consistently reduced, indicating thus an implicit limit in the network capacity (increasing the number of layers and/or filters per layer could help, paying attention to overfitting). Figure 2a shows an image patch with corresponding ground-truth count (obtained after Gaussian smoothing). Figure 2b shows the resulting BetP (H) map which represents the scalar density estimation map, while Fig. 2c shows the imprecision map M(Θ) (in our case for pixel x the imprecision value P l x (H) -Bel x (H) is equal to m x (Θ)). The values in M(Θ) may be interpreted as the predictive uncertainty, and provide a bound for the density estimation itself. For the given region S indeed, by integrating over the BetP (H) map we obtain the estimated number of people within it. Similarly, integrating over the M(Θ) map we obtain the imprecision interval s(S) -s ¯(S). Then, the corresponding RI interval is given by (s(S) -s ¯(S))/g(S) = 0.26, so that we can conclude that in S there are 12.01 ± 13% heads, i.e. s(S) ∈ [10.4, 13.6]. Moreover, from Fig. 2c we can notice that, in addition to head edges, ignorance is particularly high on heads with lower gradient on the borders and strong clutter, reflecting in a smaller confidence about the prediction. Finally as expected, we underline the desirable effect of ignorance being higher in circularly-shaped areas (e.g. shoulders, or round dark blobs) which are similar to heads, even if they have a low corresponding score.

CONCLUSION

We proposed a strategy for associating an uncertainty interval to crowd density estimation using BFT. A new evaluation method taking into account the output uncertainty at multiple scales was proposed as well. The results show that our contributions are effective in characterizing the multi-scale performance of different density estimators. Our work opens a promising avenue for crowd safety applications which account for estimation uncertainty during planning and monitoring. Future work will be devoted to applying our evaluation to other widely used density estimation networks such as MCNN or CSRNet across more datasets.

Fig. 1 :

 1 Fig. 1: Density estimator evaluation with the proposed RI vs. PEP plot at multiple scales and with different discounting amounts. Each horizontal cluster corresponds to a different discounting factor.

  (a) Image patch S, g(S) = 12.3 (b) BetP (H) map, s(S) = 12.01 (c) M(Θ) map, s(S) -s ¯(S) = 3.2