F Bartoli

T Aubert

M Moutaouekkil

J Streque

P Pigeat

S Zhgoon

A Talbi

S Hage-Ali

H M'jahed

O Elmazria

AlN/GaN/Sapphire heterostructure for high-temperature packageless acoustic wave devices

Keywords: SAW, Packageless Sensor, TCF, WLAW, High-temperature

Surface acoustic waves (SAW) technology is very promising to achieve wireless sensors able to operate in high temperature environments up to possibly 1000°C. However, there is currently a bottleneck related to the packaging of such sensors. The current high-temperature packaging solutions can withstand 600°C at most. This limitation could be overtaken by the development of packageless devices, based on the waveguiding layer acoustic waves (WLAW) principle. In such devices, the acoustic wave is confined inside an inner layer and is then isolated from undesired surface perturbations like dust deposition. In this paper, we investigate the performance of an AlN/IDT/GaN/Sapphire WLAW device used as a temperature sensor able to operate up to 500°C. After validating a room-temperature GaN material constant set with basic SAW measurements performed on IDT/GaN/Sapphire structure, the AlN/IDT/GaN/Sapphire device is simulated to determine the optimal relative thicknesses of AlN and GaN films in order to obtain a good wave confinement. Based on these calculations, an experimental WLAW device is performed and electrically characterized. The full wave confinement is experimentally confirmed by the lamination of an acoustic absorber on top of the device: no change in the scattering parameters was observed. The WLAW device is then electrically characterized between the ambient temperature and 500°C. A temperature coefficient of frequency (TCF) value of -34.6 ppm/°C is obtained, demonstrating the potential of the WLAW AlN/IDT/GaN/Sapphire structure as a packageless temperature sensor. Finally, the theoretical TCF of the AlN/IDT/GaN/Sapphire structure was numerically calculated by changing the material constants of AlN, GaN and Sapphire according to the temperature coefficients available in the literature. The theoretical and experimental data were found in good accordance.

INTRODUCTION

Surface acoustic wave (SAW) devices are massively used as RF filters in mobile communication devices. As SAW devices are very sensitive to environmental parameters, they are also intensively studied for sensor applications [START_REF] Ballantine | Acoustic Wave Sensors[END_REF]. Moreover, SAW devices being passive components, they can be remotely interrogated without embedded electronics or on-board power source. This last property makes this technology particularly suitable for high temperature applications. Currently, conventional wireless sensors are based on silicon integrated electronics, which are limited to 200°C. Many studies have been conducted in the last two decades to identify piezoelectric materials and metals for interdigital transducers (IDT) able to operate at high temperatures. Conventional SAW substrates like quartz, congruent lithium niobate or lithium tantalate are not suitable for temperatures above 400-450°C [START_REF] Hornsteiner | Surface acoustic wave sensors for hightemperature applications[END_REF].

Actually, very few piezoelectric materials can withstand high temperatures beyond this threshold. So far, the most advanced candidate is langasite (La3Ga5SiO14; LGS) whose surface is stable up to 1000°C for weeks under air atmosphere [START_REF] Aubert | Stability of langasite regarding SAW applications above 800°C in air atmosphere[END_REF][START_REF] Da Cunha | Pt-Ni / Pt-Zr electrodes for stable SAW resonator operation during repeated temperature cycling up to 1000[END_REF]. Moreover, 4-inches LGS wafers of different orientations are commercially available and reliable high-temperature elastic constants of LGS have been recently published, allowing the prediction of the frequencytemperature law of a given LGS-based SAW device in a large temperature range [START_REF] Nicolay | A Numerical Method to Derive Accurate Temperature Coefficients of Material Constants from High-Temperature SAW Measurements: Application to Langasite[END_REF]. Still, the upper temperature range of wireless interrogation of LGS-based SAW devices is 600-700°C [START_REF] Da Cunha | Wireless Sensing in Hostile Environments[END_REF][START_REF] François | Wireless and Passive Sensors for High Temperature Measurements[END_REF] because the electrical resistivity of this material drops in the range of 10 5 Ω.cm at 700°C [START_REF] Takeda | Structure and electrical properties of aluminum substituted langasite family crystals[END_REF]. In this context, the most promising candidate to achieve wireless SAW sensors able to operate above 700°C is AlN/sapphire bilayer structure, which shows an electrical resistivity five orders of magnitude higher than that of LGS [START_REF] Kim | High-Temperature Electromechanical Characterization of AlN Single Crystals[END_REF], and could be possibly used up to 900°C in the air [START_REF] Aubert | Reliability of AlN/Sapphire bilayer structure for high-temperature SAW applications[END_REF][START_REF] Aubert | In situ high-temperature characterization of AlN-based surface acoustic waves[END_REF][START_REF] Caliendo | Characterization of Pt/AlN/Pt-based structures for high temperature, microwave electroacoustic devices applications[END_REF]. However, the development of a high-temperature packaging solution constitutes a technological bottleneck, the most advanced solutions being limited to 600°C [START_REF] François | High temperature packaging for surface acoustic wave transducers acting as passive wireless sensors[END_REF].

Indeed, SAW devices must be packaged in order to protect the thin film electrodes from accidental scratches and isolate the acoustic wave from surface perturbations such as dust deposition. Several studies have shown the possibility to achieve packageless acoustic wave devices, using the waveguiding layer acoustic waves (WLAW) technology [START_REF] Bhattacharjee | Packageless SAW Devices with Isolated Layer Acoustic Waves (ILAW) and Waveguiding Layer Acoustic Waves (WLAW)[END_REF][START_REF] Legrani | Packageless AlN/ZnO/Si Structure for SAW Devices Applications[END_REF]. WLAW devices are made from a tri-layer structure, in which the inner layer shows a significantly lower acoustic velocity and thus lower acoustic impedance than both surrounding materials (Fig. 1).

Consequently, the acoustic wave is trapped inside the inner layer, and thus is insensitive to surface perturbations but still sensitive to bulk influences like temperature or strain modifications. In practice, at least one of the layers has to be a piezoelectric material in order to generate the acoustic waves. One additional advantage of packageless WLAW sensors is that the electrodes are buried and thus protected from physical and chemical alterations [START_REF] Legrani | AlN/IDT/AlN/Sapphire SAW Heterostructure for High-Temperature Applications[END_REF]. The extreme miniaturization of the device, achieved by removing the package, is also interesting for industrial perspectives. III-Nitride materials as well as sapphire are well known for their robustness regarding harsh environments and their high electrical resistivity [START_REF] Wei | Ultrasonic Al2O3 Ceramic Thermometry in High-Temperature Oxidation Environment[END_REF][START_REF] Ambacher | Thermal stability and desorption of Group III nitrides prepared by metal organic chemical vapor deposition[END_REF]. Moreover, sapphire and AlN are very stiff materials which show high acoustic velocity: the velocity of Rayleigh waves is around 5600 m/s for both c-oriented AlN and sapphire. In the case of gallium nitride (GaN), this velocity is significantly lower, being equal to 3700 m/s [START_REF] Justice | Harmonic Surface Acoustic Waves on Gallium Nitride Thin Films[END_REF]. Consequently, we consider in this paper the AlN/IDT/GaN/Sapphire structure as a promising WLAW structure for hightemperature applications. In order to develop fully operational WLAW sensors based on this layered structure, it is necessary to simulate the device behavior using reliable physical constant sets in a large temperature range for the three constitutive materials. Thus, the robustness of some constant sets available in the literature is checked by comparing experimental and calculated results of basic AlN/Sapphire and GaN/Sapphire SAW devices. The most reliable sets are then used to determine numerically the possibility to generate WLAW in the AlN/GaN/Sapphire structure at room temperature, and to optimize the films' respective thicknesses. Based on these calculations, an AlN/GaN/Sapphire WLAW device is fabricated and electrically characterized between room temperature and 500°C.

METHODS

Model description and selection of material constants sets

The numerical results including dispersion and S11 curves are calculated by a 2D-FEM modeling (software Comsol Multiphysics®), using the general partial differential equations interface, and applying the Bloch periodic boundary conditions to the unit cells (Fig. 2) along the X-axis and considering one period of SAW structure as shown in fig. 5. The used model describes an infinite aperture IDT with infinite number of fingers. Thus, the electromechanical coupling coefficient K², and the wave velocity are calculated using the admittance of the unit cell. Sapphire is a well-known crystal for which reliable room temperature-data, as well as associated first-order temperature coefficients, are available [START_REF] Tsubouchi | Zero-Temperature-Coefficient SAW Devices on AlN Epitaxial Films[END_REF]. Besides, a room-temperature constants set of AlN/Sapphire bilayer structure had already been validated in previous studies [START_REF] Elmazria | High Velocity SAW Using Aluminum Nitride Film on Unpolished Nucleation Side of Free-Standing CVD Diamond[END_REF]. In the present work, the GaN/Sapphire bilayer structure is firstly considered to find reliable room temperature GaN material constant sets. To do so, calculated velocity dispersion curves are compared with the experimental one obtained from the characterization of GaN/Sapphire SAW devices (see next section). Then, the full AlN/IDT/GaN/Sapphire WLAW structure is simulated, using aluminum as metal for IDTs, and using the separately validated constants sets of the four considered materials. Finally, different first-order temperature coefficient sets are tested for GaN and AlN by comparing calculated and experimental temperature coefficient of frequency (TCF) dispersion curves for GaN/Sapphire and AlN/Sapphire SAW bilayer structures. The experimental data come from previous studies in the case of AlN/Sapphire [START_REF] Aubert | Investigations on AlN/Sapphire Piezoelectric Bilayer Structure for High-Temperature SAW Applications[END_REF][START_REF] Blampain | AlN/Sapphire: a promising structure for high temperature and high frequency SAW devices[END_REF][START_REF] Blampain | Pt/AlN/Sapphire SAW resonator operating in GHz range for High température Wireless SAW sensor[END_REF].

Table I summarizes the room temperature physical constants considered for sapphire, Al and AlN, and the three different sets tested for GaN. Table II gives the various sets of first-order temperature coefficients of elastic constants Cij(T), and thermal expansion coefficients αii considered in the calculations. The reference [START_REF] Adachi | Elastic constants of GaN between 10 and 305 K[END_REF] contains multiple constants sets for the GaN material. One of them is calculated by local-density-approximation (LDA), and the other by the generalized-gradientapproximation (GGA).

Experimental

Commercial GaN/Sapphire bilayer structures were purchase from Kyma Technologies, Rayleigh, NC, USA. The 2 µm-thick, c-oriented, non-intentionally doped GaN films were hetero-epitaxially grown by MOCVD on c-cut sapphire substrates. 200 nm-thick aluminum films were deposited onto the GaN/Sapphire structures and patterned by conventional contact ultraviolet (UV) photolithography and wet etching to achieve SAW synchronous single-port resonators. The Al IDTs were made of 100 finger pairs, with a metallization ratio of 50% and an aperture of 40•λ. Different wavelengths comprised between 5 and 24 µm were used. The resonators were equipped with 200 reflectors on each side of the IDT. The propagation path of the acoustic waves was along the X-direction of the sapphire substrates.

Finally, AlN/Al/GaN/Sapphire WLAW structures were fabricated. 12 µm-thick AlN film was grown onto the previously described Al/GaN/Sapphire SAW structures with a wavelength of 5 µm, using reactive magnetron sputtering. The AlN thickness was measured by in-situ reflectometry and its microstructure was determined by X-ray diffraction (XRD) θ-2θ measurements (Bruker D8 Advance CuKα1). The IDTs contacts were protected by a tantalum hard mask during the AlN deposition in order to keep a subsequent electrical access to the device.

Both SAW and WLAW devices were electrically characterized up to 500°C in air atmosphere using a network analyzer (PNA 5230a, Agilent Technologies Inc., Santa Clara, CA)

and an RF probe station (S-1160, Signatone Corp., Gilroy, CA) equipped with a S-1060 series Signatone thermal probing system and water-cooled RF probes (Z-Probe, Süss Microtech AG, Garching, Germany) so that it can withstand temperatures up to 600°C. Before all measurements, the RF setup was conventionally calibrated in order to obtain reliable measurements.

In order to verify experimentally the confinement of the wave in the inner layer of the WLAW device, an elastomeric material was placed on the surface of the top AlN film (Fig. 3).

Whereas a SAW should suffer from strong viscous losses in the soft medium placed at the surface of the structure, a WLAW should be insensitive to it [START_REF] Floer | AlN/ZnO/LiNbO3 packageless structure as a low-profile sensor for on-body applications[END_REF]. To make this test as selective as possible, a particularly soft elastomer (Solaris®, Smooth-on, USA, EYoung = 172 kPa) has been chosen. The results of this test are shown on figure 10.

RESULTS AND DISCUSSION

Room Temperature results

The comparison between the calculated Rayleigh wave dispersion curves and the ones obtained from the experimental characterization of GaN/Sapphire SAW devices is shown in Fig. 4. A good agreement is obtained in a large range of GaN relative thicknesses, with the roomtemperature constant calculated by the local-density-approximation (LDA). To simulate the whole AlN/Al/GaN/Sapphire structure we consider the sets from references [START_REF] Tsubouchi | Zero-Temperature-Coefficient SAW Devices on AlN Epitaxial Films[END_REF], [START_REF] Elmazria | High Velocity SAW Using Aluminum Nitride Film on Unpolished Nucleation Side of Free-Standing CVD Diamond[END_REF] and [START_REF] Adachi | Elastic constants of GaN between 10 and 305 K[END_REF] respectively for AlN, Sapphire and GaN. As the trapping of the acoustic wave inside the inner layer becomes more efficient when the respective thicknesses of the surrounding layers increase, a thick AlN overlayer of 20 µm was firstly considered to ensure the hypothetical confinement of the wave. Simulations show that the expected WLAW mode can propagate in the structure under certain conditions (Fig. 5). The phase velocity dispersion curve of the AlN/Al/GaN/Sapphire structure was calculated for fixed AlN thickness (20 µm); figure 6 shows the phase velocity and K² dispersion curves. Both experimental and theoretical values of K 2 was determined using from admittance versus frequency curves considering resonance (fr) and antiresonance (fa) frequencies and using the following formula [START_REF] Erhart | Piezoelectric ceramic resonators; Book, Topics in mining, metallurgy and materials engineering[END_REF][START_REF] Varshney | Theoretical and experimental analysis of high Q SAW resonator transient response in wireless sensor interrogation application[END_REF]:

𝐾 " (%) = 2 × 𝑓 * -𝑓 , 𝑓 * × 100
One can observe that the maximum value of K 2 , namely 0.21%, is reached when the relative thickness of the GaN layer khGaN is in the range between 2.6 and 2.9, with a WLAW velocity between 5100 and 4900 m/s respectively. Taking into account that the actual GaN thickness of the samples is 2 µm, obtaining the maximum K 2 requires a wavelength close to 5 µm.

The figure 7 shows the particles displacement and their position inside the three-layer structure following the thickness of the AlN thin film. As we can see in this figure, the particles displacement at the surface of the device decreases when the AlN thickness increases. For subsequent calculations, the GaN thickness was set to 2 µm, the wavelength to 5 µm, and the AlN overlayer thickness was steadily reduced in order to find the minimum value allowing the propagation of the WLAW, i.e. an acoustic wave trapped inside the GaN inner layer. We used as numerical criterion for the wave confinement, a surface displacement below 1% of the wave's maximum displacement. Using this criterion, the minimum thickness of AlN for our structure was found to be equal to 12 µm. Then, it was decided to grow a 12 µm-thick AlN film onto the Al/GaN/Sapphire SAW devices, where The AlN thin film have been grown using a sputtering power of 200 W. The gas used was composed with only N2 at 5 mTorr pressure, with a substrate-target distance of 5 cm. The substrate was not subject to any heating. The growth rate obtained with those parameters was of 100 nm in 12 min. The sapphire substrate (stars) shows a clear (006) orientation of two peaks, the first close to 42° is the (006) orientation, the second, next to 91° being the (0012) orientation. An additional (024) orientation is visible. This could indicate some defect into the substrate, but the intensity of this (024) peak is small compared to the principle (006) orientation. Thus it does not affect our experiments.

The (002) expected orientation for the epitaxial GaN film is confirmed. This orientation can be observed through the (002) and (004) peaks, respectively located at 34.7° and 73°. As for the sapphire substrate, the GaN layer can contain some defects illustrated by the weak (103) orientation close to 64.7°, which is not an issue in our case.

The AlN thin film shows polycrystalline behavior. Most of AlN usual orientations are visible, which means that the deposed AlN won't show any monocrystallic behavior. Thus, the elastic constants of the experimental AlN won't fit the simulated constants. A difference of wave velocities between the simulated structure, and the experimental one is therefore expected.

However, the AlN films shows a significant (002)-texture, by the (002) and (004) peaks respectively located at 36.2° and 76.7°. This kind of structure is still interesting for the hoped oxygen diffusion barrier behavior. expected velocity (4250 m/s). After deposition, the peak shifted towards 953 MHz, while the Q-factor strongly decreased to 30 from 600. The corresponding phase velocity is close to 4800 m/s, in good agreement with the expected 5000 m/s for the calculated WLAW mode.

The confined nature of this mode was finally experimentally confirmed by the specific test described in the experimental section: the S11 signal remains unchanged after the lamination of the soft Solaris elastomer onto the sample (Fig. 10).

The slight discrepancy of 200 m/s between the experimental and the calculated velocity for this WLAW mode could come from the polycrystalline nature of the AlN film. We attribute the reduction in Q to both the decrease of the reflection coefficient of the resonator electrodes and to the dielectric loss in the AlN deposited over the electrodes. The former feature is common to devices in which the resonator is coated by different layers. In filters dense metal electrodes are used (Cu) to oppose this phenomenon [START_REF] Abbott | Temperature compensated saw with high quality factor[END_REF]. In high-temperature sensors, electrodes based on Pt and Ir are often the only choice [START_REF] Taguett | Ir-Rh thin films as high-temperature electrodes for surface acoustic wave sensor applications[END_REF] and can efficiently fit the purpose. For moderate temperature sensors with Al electrodes, the number of electrodes in reflecting gratings should be increased accordingly. So that this feature can be taken care-of with known methods. The latter feature is observed as a substantial increase in the electrical loss level. This effect manifests itself as a visible decrease of S11 (from about -0.1 dB to about -0.7 dB) outside of the WLAW excitation region. We can attribute it to the dielectric loss in AlN deposited over the IDT electrodes. The damage of Al electrodes cannot be excluded either, but it seems less probable as the deposition temperature was not very high. Future development will have to take care of all of these effects.

The experimental K² of the Al/GaN/Sapphire SAW structure have been measured and is equal to 0.30%, which is in agreement with the simulated K², equal to 0.39%. The deterioration of the signal after the AlN film deposition is too important to extract an accurate value of K² for the WLAW structure. II).

Figure 11 shows the experimental and calculated dispersion of the TCF with the piezoelectric layer relative thickness for the AlN/Sapphire SAW structures. A good correlation is obtained for this structure when the AlN constant set from Ref. [START_REF] Bjurström | Temperature compensation of liquid FBAR sensors[END_REF] are used. Thus, this set is considered as the most accurate and it is selected for the high-temperature study of the complete WLAW structure. The reliable constants coming from Ref. [START_REF] Tsubouchi | Zero-Temperature-Coefficient SAW Devices on AlN Epitaxial Films[END_REF] are used for sapphire.

The same study was conducted regarding the GaN/Sapphire SAW devices. All the measured TCF values are reported on figure 12 and compared to the calculated ones. In that case, no set of GaN first-order temperature coefficients of elastic constants and thermal expansion coefficients leads to a good convergence in the full range of GaN normalized thicknesses considered here. Consequently, all the four GaN sets are considered in the subsequent simulation study of the AlN/Al/GaN/Sapphire WLAW device.

AlN/Al/GaN/Sapphire WLAW device

As for the GaN/Sapphire SAW devices, the complete WLAW device was electrically characterized between the ambient temperature and 500°C. After cooling, the devices were characterized a second time in the same temperature range. The results are reproducible for both structures (Fig. 13). The WLAW device shows a linear frequency-temperature law, with a TCF value of -34.6 ppm/°C, corresponding to a slightly larger temperature sensitivity than the GaN/Sapphire SAW devices (TCF = -29.6 ppm/°C). Finally, the theoretical TCF of the AlN/Al/GaN/Sapphire WLAW structure was calculated using the coefficients coming from Ref. [START_REF] Tsubouchi | Zero-Temperature-Coefficient SAW Devices on AlN Epitaxial Films[END_REF] for Sapphire, Ref [START_REF] Bjurström | Temperature compensation of liquid FBAR sensors[END_REF] for AlN, and the four available data for GaN, as explained hereinabove (Tab. II). Calculated TCF values were equal

CONCLUSION

The suitability of the WLAW AlN/Al/GaN/Sapphire structure for the future achievement of high-temperature packageless sensors has been evidenced in this study.

Simulations based on reliable room-temperature constant sets of AlN, GaN and sapphire materials show that a WLAW mode can be generated in this structure under certain conditions.

Based on these calculations, an experimental device has been performed and characterized, confirming the possibility to excite a WLAW mode in the AlN/Al/GaN/Sapphire structure.

Future developments will be focused on the improvement of the resonator device Q-factor which remains currently too weak, likely because of inadequate reflectors design and dielectric losses in the AlN film.

The behavior and the stability of the WLAW device was in situ investigated at temperatures up to 500°C with success. The frequency-temperature law appears to be linear and the temperature sensitivity is large, with a TCF value of -34.6 ppm. This value can be retrieved by simulations means using first-order temperature coefficients of material constant sets for AlN, GaN and sapphire, whose robustness has been checked by basic SAW measurements.

Fig. 1 :

 1 Fig. 1: Schematized WLAW structure.

Fig. 2 :

 2 Fig. 2: Schematic 2D representation of the unit cell of the investigated structures: Al/AlN/Sapphire, Al/GaN/Sapphire and AlN/Al/GaN/Sapphire.

Fig. 3 :

 3 Fig. 3: Measurements of the WLAW scattering parameters, during the Solaris elastomer test.

Fig. 4 :

 4 Fig. 4: Comparison between experimental (yellow stars) and calculated (lines) phase velocity dispersion curves of Rayleigh waves propagating in GaN/Sapphire bilayer structure, made by using the material GaN constants taken from references [27].

Fig. 5 :

 5 Fig. 5: Total displacement in the AlN/Al/GaN/Sapphire structure with different AlN thicknesses: (a) 0 µm, (b) 6 µm, (c) 12 µm and (d) 20 µm.

Fig. 6 :Fig. 7 :

 67 Fig. 6: Phase velocity and K 2 dispersion curves of the WLAW mode in AlN(20 µm)/Al/GaN/Sapphire structures.

Figure 8

 8 Figure 8 shows the θ-2θ XRD spectrum obtained from the new AlN/Al/GaN/Sapphire

Fig. 8 :

 8 Fig. 8: θ-2θ XRD measurement of the AlN/Al/GaN/Sapphire device, where the thicknesses of GaN

Figures 9

 9 Figures 9 shows the magnitude of the S11 scattering parameters measured on a

Fig. 9 :Fig. 10 : 1

 9101 Fig. 9: Room-temperature S11 magnitude of two resonators respectively based on the Al/GaN/Sapphire and the AlN/Al/GaN/Sapphire structures.

Fig. 11 :Fig. 12 :

 1112 Fig. 11: Comparison between the experimental and calculated AlN/Sapphire TCF SAW dispersion curves.

Fig. 13 :

 13 Fig. 13: Experimental frequency-temperature laws of the Al/GaN/Sapphire SAW device and the AlN/Al/GaN/Sapphire WLAW device.

  to -35 ppm/°C, -30 ppm/°C, -19 ppm/°C, and -17 ppm/°C with the GaN constant sets coming from references [27-GGA], [27-LDA],[START_REF] Adachi | Elastic constants of GaN between 10 and 305 K[END_REF], and[START_REF] Dai | Stress evolution in AlN and GaN grown on Si(111): experiments and theoretical modeling[END_REF] respectively. The better agreement is obtained with sets from Ref. and, which were also those giving the most accurate predictions concerning the TCF values of GaN/Sapphire SAW devices with large values of GaN relative thickness (kh ≥ 3), i.e. those in which the acoustic wave is mostly located in the GaN film, which is one of the feature the WLAW mode.

) Ref

	Constant	C11	C12	C13	C14	C33	C44	C66	e15	e31	e33	ε11	ε22	ε33	ρ
	Al	107.3 60.8	X	X	X	28.3	X	X	X	X	X	X	X	2702	[25]
	Sapphire	497	164	111 -23.5 498	147	X	X	X	X	8.28 8.28 0.12	3980	[20]
	AlN	345	125	120	X	395	118 110 -0.48 -0.58 1.55	8	8	9.5	3260	[26]
	GaN	350	140	104	X	376	101 115	0.5	-0.32 0.63	8.9	8.9	8.9	6150	[27]
	GaN	390	145	106	X	398	105 123	0.5	-0.32 0.65	8.9	8.9	8.9	6150	[27]
	GaN	370	145	110	X	390	90	113	0.5	-0.32 0.65	8.9	8.9	8.9	6150	[27]

Tab. II: First-

order temperature coefficient of elastic constants and expansion coefficients of the AlN, GaN and Sapphire.

	Material		Temperature coefficients (ppm/°C)		Thermal expansion coefficients (10 -6 /°C)	Ref
	Constant C11(T) C12(T) C13(T) C14(T) C33(T) C44(T) C66(T) α11(T) α22(T) α33 (T)
	Sapphire	-75	40	-80	-70	-85	-180	X	7.28	7.28	8.11	[20]
	AlN	80	180	160	X	100	50	X	5.27	5.27	4.15	[21]
	AlN	-28	-35	-40	X	-30	-11	X	5.27	5.27	4.15	[28]
	AlN	-37	-1.8	-1.8	X	-65	-50	-57	5.27	5.27	4.15	[26]
	GaN	47.1	94.7	94.7	X	52.5	9.9	16.7	5.2	5.2	3.95	[27]-LDA
	GaN	49.5	97.9	97.9	X	52.9	12.7	19.8	5.2	5.2	3.95	[27]-GGA
	GaN	-15	-16.9	-33.7	X	-23.6	-6.9	-6.9	4.45	4.45	3.03	[29]
	GaN	-42.5	-45.9	-53	X	-59.4	-15.8	-15.8	4.45	4.45	3.03	[28]

This work has been supported by the grant ANR-15-CE08-0015 SALSA, by the Region Grand-Est, by the French PIA project "Lorraine Université d'Excellence", reference ANR-15-IDEX-04-LUE, by Institut Carnot en Loraine (ICEEL) CPER MatDS; and by the Ministry of Science and Education of Russian Federation 8.6108.2017/6.7.


Aymeric Blot
email: aymeric.blot@univ-lille.frme.kessaci

Marie-Éléonore Kessaci

Laetitia Jourdan
email: laetitia.jourdan@univ-lille1.fr

Patrick De Causmaecker
email: patrick.decausmaecker@kuleuven.be

Adaptive Multi-Objective Local Search Algorithms for the Permutation Flowshop Scheduling Problem

Automatic algorithm configuration (AAC) is an increasingly critical factor in the design of efficient metaheuristics. AAC was previously successfully applied to multi-objective local search (MOLS) algorithms using offline tools. However, offline approaches are usually very expensive, draw general recommendations regarding algorithm design for a given set of instances, and does generally not allow per-instance adaptation. Online techniques for automatic algorithm control are usually applied to single-objective evolutionary algorithms. In this work we investigate the impact of including control mechanisms to MOLS algorithms on a classical bi-objective permutation flowshop scheduling problem (PFSP), and demonstrate how even simple control mechanisms can complement traditional offline configuration techniques.

Introduction

Designing and tuning metaheuristics is a great challenge in the optimisation field. Offline configurators have been proposed to automatically configure optimisation algorithms according to a single indicator [?,?] and more recently, according to several indicators [?]. However, offline configuration requires a large amount of algorithm executions to test the different configurations and then, the output tuned algorithm is adapted to the training instances only. In this paper, we are interested in online configuration, also called parameter control, that adapts the algorithm during the execution [?,?,?].

Multi-objective local search (MOLS) algorithms are metaheuristics designed to solve multi-objective combinatorial optimisation problems. MOLS algorithms require the definition of several strategies such as the selection of the solution to explore, the exploration of the neighbourhood, the reference set to accept candidate neighbours, or the archive maintenance. They have successfully been used to tackle multi-objective permutation problems such as the travelling salesman problem, the quadratic assignment problem and the permutation flowshop scheduling problem [?,?,?,?]. Parameter control has mostly been applied to bioinspired algorithms for single objective optimisation. In this paper, we propose to control multi-objective local search algorithms to solve the bi-objective permutation flowshop scheduling problem.

This paper is organised as follows. First, a literature review on adaptive metaheuristics is presented. Section 3 details the static and the adaptive versions of the multi-objective local search algorithms. Section 4 presents the experimental setup and Section 5 gives and discusses the results. Finally, Section 6 concludes the paper and draws some perspectives.

Adaptive Metaheuristics

Parameter control mechanisms are generally classified between deterministic, adaptive and self-adaptive approaches [?]. In the following, we focus on adaptive approaches that are parameter and algorithm-independent.

Adaptive search

In their paper [?], Pisinger and Ropke describe a large neighbourhood search heuristic for a pick up and delivery problem which adapts during search through the selection probability of the neighbourhoods or "subheuristics" as the authors call them. The adaption strategy is steered by the history during the search and a reinforcement learning mechanism is employed to adjust the weight of each neighbourhood after a given number or a "segment" of iterations has been applied. The weight determines the probability of selecting a specific neighbourhood. Extra stochasticity is introduced through noise in some of the basic heuristics to create less deterministic behaviour and increase exploration.

An example of off-line learning can be found in [?] e.g., a multi-mode scheduling problem is approached through an automaton model. Reinforcement learning agents learn the decision functions for these automatons off-line through observation of the history. The result is a fast constructive heuristic adapted to the real world situation from which problem instances originate. Off-line learning has taken on recently resulting in very strong results and frameworks. [?,?,?,?,?,?] and recently, tuning the tuner, in [?]. On-line as well as off-line learning are extensively used in hyperheuristics, see [?] for an overview.

Multi Armed Bandit

A good introduction on the subject of multi armed bandits (MAB), with the right balance between fundamental understanding and practical understanding, and without any unnecessary detail, can be found in [?]. MAB's, metaphorically referring to the infamous gambling machines, model a decision problem where the only (or main in some versions) source of information is history of previous selections. The only decision to be taken is which arm to pull next. The aim is to maximise the expected outcome of a finite series of decisions. This has been the subject of a fascinating piece of research leading to a large number of convergence results for various policies and their variations. In an adaptive local search setting, multi armed bandits can be used to model the decisions to be taken on which subalgorithm or neighbourhood to select in the next step given the history of the current search. MAB's have been applied in evolutionary algorithms, see e.g. [?] as well as in evolution based hyperheuristic settings [?]. An early example of an application to combinatorial optimisation in networks is [?]. The design of algorithms using MAB's was studied in [?,?].

-greedy

MAB's, having only history to learn from, need to make decisions that both optimise the immediate result and optimise the lessons learned for better future results. Handling this exploitation/exploration dilemma is what makes a strategy for a MAB. A simple strategy is to pick the decision that has delivered the best result on average in the past. The MAB can then eventually start with an exploration phase where every handle is tried once or a predetermined number of times, after which the greedy strategy is used, adjusting the averages after every decision. In dynamical situations, the average may be weighted to introduce a bias towards recent history. These approaches are termed "greedy". In a slightly more explorative approach, a probability is introduced to allow for random selection of an arm, independently of its average success rate. This kind of approaches are called -greedy, see e.g. [?]. One possibility is to select at time t + 1 the best performing arm with a probability of (1 -ε), leaving a probability of ε to uniformly select an arm at random (Eq. ??).

p i (t + 1) = (1 -ε) + ε/N, if i = arg max j r j (t) ε/N, otherwise (1)

Random

The most simple random control mechanism is based on uniform distribution of arms. However, other random approaches exist, such as Softmax algorithms, that use a specific distribution to select the next arm. Arms with better results on average in the past are assigned a higher probability to be selected. One possibility is to use a Boltzman distribution with the probability to select arm i at time t+1 is given in terms of the average rewards r i (t) at time t as [?,?] (Eq. ??).

p i (t + 1) = e ri(t)/τ arms j e rj (t)/τ , for all arms i

where τ is a temperature parameter that can be taken constant or decreasing [?,?,?,?].

Adaptive pursuit

Pursuit algorithms relate to classical techniques from machine learning in that the probability to select any arm is adjusted after each selection of a specific arm. Given a learning rate β, the probabilities at t + 1 are adapted after t as [?,?,?] (Eq. ??).

p i (t + 1) = p i (t) + β(1 -p i (t)), if i = arg max j r j (t) p i (t) + β(0 -p i (t)), otherwise (3)
The probabilities at t = 0 are all equal. This works well for static systems, with convergence to one for the probability of selecting the best actions. It does not work well for dynamic systems exactly because of this convergence property. In order to alleviate this situation, adaptive pursuit was conceived [?]. In adaptive pursuit, a minimum and a maximum (P min and P max ) for the probabilities p i is introduced to leave room for exploration (Eq. ??).

p i (t + 1) = p i (t) + β(P max -p i (t)), if i = arg max j r j (t) p i (t) + β(P min -p i (t)), otherwise (4)
Often, this is combined with a weighted average for the expected performances r i coping better with dynamic situations.

Other approaches

Other approaches are reinforcement comparison [?,?] and upper confidence bound [?]. These approaches offer more possibilities or a better behaviour in case of uncertainty at the cost of somewhat higher complexity.

In the following we will concentrate on the simpler methods mentioned above. These result in acceptable behaviour at a relatively low implementation cost which is important for local search algorithms.

Adaptive Multi-Objective Local Search Algorithms

In this paper, we consider and solve multi-objective combinatorial optimisation (MOCO) problems using the Pareto dominance to compare two solutions s and s : s is said to dominate s if and only if s is better or equal to s according to all criteria, and s is strictly better than s at least for one criterion. If neither s dominates s nor s dominates s, both solutions are called incomparable. In order to handle the Pareto dominance, multi-objective metaheuristics, such as bio-inspired algorithms and multi-objective local search algorithms, can be applied to solve MOCO problems. The last ones have shown very good performance on multi-objective permutation problems such as the travelling salesman problem, the quadratic assignment problem and the permutation flowshop scheduling problem [?,?,?]. Therefore, we decided to focus on multi-objective local search algorithms in the rest of the paper. We present first a static version of multiobjective local search algorithms that will be used in the experiments, and then an adaptive version in which control mechanisms have been integrated into.

Static Multi-Objective Local Search Algorithms

Multi-objective local search (MOLS) algorithms are originally adapted from stochastic local search algorithms [?] where the neighbourhood of a solution is explored to successively move to better and better solutions of the search space. Contrary to the single-objective case where a single current solution is considered only, an archive of solutions is maintained to store solutions found during the search. This archive is generally a set of Pareto solutions i.e., all the solutions of the archive are incomparable. In the literature, many variants of multi-objective local search algorithms were proposed, such as PLS [?] and its numerous variants, iterated PLS [?], the stochastic PLS [?], and the anytime PLS [?]; or the DMLS [?].

The variants differ by several aspects: the selection strategy, the neighbourhood exploration strategy and the set of candidate neighbours to enter in the archive. Indeed, one, or several, or even all solutions of the archive can be explored according to the selection strategy. The neighbourhood of each solution selected in the previous phase can be fully or partially explored following the exploration strategy where a reference is used to compare the neighbours with the solution being explored. This reference is generally either the solution being explored or the entire archive. During the exploration, a policy can be used to accept neighbours as candidate solutions. This policy may be different from the termination of the neighbourhood exploration. For example, when the exploration is stopped as soon as a dominating neighbour has been found, both this neighbour and all the incomparable neighbours visited can be candidates.

Following the literature review presented in Section ??, the on-line adaptation of an algorithm is mainly done by changing one or two strategies only during the execution contrary to off-line adaptation where thousand combinations can be tested. In this paper, we limit our study to three different termination conditions in the neighbourhood exploration and fix the other strategies. Only one solution is randomly selected in the archive to be explored. The neighbourhood exploration of the selected solution stops when one neighbour verifies the condition where the reference is fixed being the entire archive. Three strategies will be considered in this study: imp, stops when the visited neighbour dominates one of the solutions of the archive; this neighbour only is candidate, imp ndom, stops when the visited neighbour dominates one of the solutions of the archive; this neighbour and all the visited neighbours not dominated by the solutions of the archive are candidates, ndom, stops when the visited neighbour is not dominated by the solutions of the archive; this neighbour only is candidate. Finally, through the combine() function, all the candidate solutions are added to the archive, and only the incomparable solutions of the two sets are kept in the archive for the next iteration. Algorithm ?? outlines the basic multi-objective described above. The termination criterion of the algorithm can be an allocated time or number of evaluations for example.

The main drawback of the stochastic local search is they focus their search in a small part of the search space, and then mechanisms have to be integrated to enable them to visit several parts the search space. The iterated local search (ILS) algorithm [?] gives a way to do that for single-objective optimisation by adding a phase to perturb the current solution and so, diversify the search, and accept or not the new solution found. We give in Algorithm ?? a way to iterate a MOLS in this study, using a static exploration strategy and an initial archive. The final archive will contain the best Pareto solutions of the execution. The kick 1() function perturbs the execution of the mols: it randomly selects one solution from the final archive and it applies three random moves. Then, the algorithm has a chance to visit a new part of the search space. The combine() function merges the archive returned by the basic MOLS and the final archive of s-mols to keep the best Pareto solutions only. This MOLS is called static since all strategies are fixed before the execution.

Control in Multi-Objective Local Search Algorithms

In this paper, we are interested in an adaptive version of the previous static multi-objective local search where mechanisms are added to modify the exploration strategies during the execution. These mechanisms aim at adapting the exploration strategies of the MOLS during the execution. From the static MOLS algorithm, we designed an adaptive MOLS where a control arm method and an update rewards method to compute the rewards associated to each strategy are added. First, the rewards of each strategy have to be initialized: each exploration strategy is tested. Then, at each iteration of the MOLS, the control arm Algorithm 2: Static MOLS (s-mols) method defines the exploration strategy for the next execution of the mols. The kick 1() and combine() functions are the same one as defined in s-mols. At the end of the iteration, the rewards of the exploration strategies are updated.

Experimental Setup

In this section, we present the bi-objective permutation optimisation problem tackled in this work, and detail the experimental protocol.

Permutation Flowshop Scheduling Problem

The Permutation Flowshop Scheduling Problem (PFSP) is a classical permutation problem which involves scheduling a set of N jobs {J 1 , . . . , J N } on a set of M machines {M 1 , . . . , M M }. Each job J i is processed sequentially on each of the M machines, with fixed processing times {p i,1 , . . . , p i,M }; machines can only process one job at a time. The sequencing of jobs is identical on every machine, so that a solution may be represented by a permutation of size N . In the following, we consider the bi-objective PFSP (bPFSP), minimising two widely studied objectives, the makespan C max (Eq. ??), i.e., the total completion time of the schedule, and the flowtime F T (Eq. ??), i.e., the sum of the individual completion times C i of the N jobs.

C max := max i∈{1,...,N } C i (5)
F T := N i=1 C i (6)
Classical permutation neighbourhoods include the exchange neighbourhood, where the positions of two jobs are exchanged, and the insertion neighbourhood, where one job is reinserted at another position in the permutation. In this study, we consider the hybrid neighbourhood defined as the union of the exchange and insertion neighbourhoods, which is known for the bPFSP to lead to better performance than considering each neighbourhood independently [?].

We use the classical PFSP Taillard instances [?], widely used in the literature. They span numbers of jobs N ∈ {20, 50, 100, 200, 500} and numbers of machines M ∈ {5, 10, 20}. There are 12 valid (N, M ) combinations, with 10 available instances each, for a total of 120 instances. In this study, we limit the initialisation of the MOLS algorithms to the use of a simple single-objective greedy algorithm on the two objectives independently. Indeed, using smarter initialisation procedures, the starting solutions would be too close to the optimal Pareto front, which is undoubtedly detrimental to the current study since we aim to emphasise the impact of the control mechanisms over the algorithm itself. To obtain two solutions of reasonable quality, we have chosen the NEH procedure [?] for the two objectives independently. It is often used to seed state-of-the-art bPFSP initialisation procedures (e.g., the 2-phase local search algorithm [?]).

MOLS Algorithms and Control Mechanisms

Section ?? presented the static and the adaptive versions of a MOLS algorithm where only exploration strategies can be set. Indeed, all other strategies are fixed since earlier work [?,?] has shown that the exploration strategy is the most impactful MOLS component for the bPFSP.

In the experiments, we compare the three deterministic instantiations of Algorithm ??, each using a single exploration strategy (denoted simply by imp, imp ndom, and ndom, respectively), to two adaptive algorithms (see Algorithm ??), using a basic random control mechanism or a ε-greedy control mechanism respectively. While many other more sophisticated mechanisms could have been compared (e.g., see [?,?]), they would likely be similar because only three arms were considered, which considerably limits the number of dissimilar decision strategies.

In the random control mechanism decisions are uniformly taken at random, without any feedback from the search, in contrary to the ε-greedy control mechanism that uses feedback to take decisions. This feedback is computed every iteration using the hypervolume difference between the hypervolume of the new archive and the one of the previous iteration. It is then used to update the reward associated to the current strategy using a learning rate α = 0.8 (Eq. ??, with r i the reward of the arm i, and f (t) the feedback obtained using the current arm).

r i (t + 1) = r i (t) + α(f (t) -r i (t)), if i the current arm r i (t), otherwise (7)
In this study, we set ε = 0.1 so that the best performing strategy (i.e., the arm arg max i r i (t)) is chosen with 90% probability, either strategy being selected uniformly at random otherwise.

For both control strategies, we consider four different variants, that differ by the subset of exploration strategies that are available. First, the three exploration strategies are available for both adaptive algorithms (rand 3, greedy 3). Note that it is already known that the imp strategy leads to poorer results on the bPFSP. But, we still decide to make available this bad strategy in order to evaluate the control mechanism without any a priori knowledge. Secondly, we use this expertise and only make available the two strategies imp ndom and ndom for both adaptive algorithms (rand 2, greedy 2). Finally, the last two variants introduce a long-term learning scheme, beginning with the three strategies but switching to only use the two best strategies during the search. Two possibilities are evaluated: either after half the total running time of both adaptive algorithms (rand ltl 50, greedy ltl 50), or after twenty percent of the total running time (rand ltl 20, greedy ltl 20).

The termination criterion of the three static algorithms and the eight variants of the adaptive algorithm is a total running time fixed to n 2 m/500 seconds. The termination criterion of the inner MOLS (see Algorithm ??) is a combination of either n 2 solution evaluations or n iterations without improvement. This criterion is well adapted to the bPFSP since it enables a sufficient number of iterations of both the inner algorithm and the control mechanism. In the following experiments, about 1600 executions of the inner MOLS for instances with 20 jobs have been done, then about 750, 400, 250 and 100 iterations for instances with 50, 100, 200 and 500 jobs, respectively. This decrease of the number of executions when the number of jobs increases is explained by the exploration step that becomes more and more long and challenging as the size of the neighbourhood quickly grows.

Experimental Protocol

In this work, we propose to compare the performance of using traditional deterministic mechanisms (through three static algorithms) with different control mechanisms (through eight adaptive algorithms). Experiments are conducted across all classical bPFSP Taillard instances, separated in twelve benchmarks of 10 instances sharing the same number of jobs and machines. The experimental protocol is reduced to the exhaustive comparison of all approaches on all benchmark instances. Because of the stochasticity of both the algorithm and the control mechanisms, all approaches are run 20 times on each instance, using a given set of 20 random seeds.

In total, eleven approaches are compared, in 4 successive steps, as detailed in Table ??. First, we compare the three deterministic approaches with the two adaptive approaches that use all three explorations strategies. Then, we focus on the two best strategies, and compare the respective two deterministic approaches with the two adaptive approaches that use them only. Finally, we investigate the potential of a long-term learning scheme for the two control mechanisms independently, first by switching from three arms to two arms after half of the runtime has passed (rand ltl 50, greedy ltl 50), then after only twenty percent of the runtime (rand ltl 20, greedy ltl 20).

Experimental Results

Table ?? presents the rankings resulting of the four steps presented in the experimental protocol (see Section ??) for the 12 instance sizes together with the resulting average ranks. For each instance size, the ranking is computed using pairwise Wilcoxon signed rank tests and the Friedman test post hoc analysis checks the statistical equivalence between algorithms ranked 1, and their difference with the others.

First, we focus on the 3-arm adaptive approaches, rand 3 and greedy 3, comparing them to the three respective deterministic approaches imp, imp ndom and ndom. As shown on Table ??, the imp and ndom approaches always perform very poorly and very well, respectively. Meanwhile, the imp ndom approach performs rather poorly in small instances, but achieves very good results on the largest ones. Surprisingly, the two adaptive approaches (rand 3 and greedy 3) equivalently perform. More precisely, they perform very well on the first eight instances (rank 1), but their performance are more debatable on the four largest ones. Indeed, for instances with 100 jobs and 20 machines, they are outperformed by the deterministic approach ndom and equivalently perform with the imp ndom approach. But, for instances with 200 and 500 jobs, they are also outperformed by this latter approach. In these cases, they are still better than the imp approach. This results show that the imp approach affects more the adaptive algorithms when the problem gets harder. Then, in the second step, the imp approach is discarded and we focus on the 2-arm adaptive approaches (rand 2 and greedy 2) that have only the choice between the imp ndom and ndom exploration strategies. Once again, the two adaptive approaches equivalently perform. However, they are rank 1 for all the instances except for the 100-jobs 20-instances (rank 2). Considering only the well performing exploration strategies largely improves the adaptive approaches for the largest instances.

Having validated that the imp arm should not be used on larger machines instances, we finally investigate a long-term learning scheme where arms can be discarded if they are worse than the others. Two approaches have been tested: the discard of the worst strategy is done after either fifty percent (rand ltl 50, greedy ltl 50) or twenty percent (rand ltl 20, greedy ltl 20) of the total running time. In order to effectively analyse this long-term learning scheme, the two adaptive approaches are investigated and ranked separately. Unsurprisingly, the 2-arm versions of both adaptive approaches always statistically outperform their respective 3-arm versions and so for the versions using the long-term learning. Introducing long-term learning to only keep well-performing arms is efficient. The ranking between the two control mechanisms rand and greedy are not the same size by size, but the average ranks show that it is more efficient to discard an arm sooner since rand ltl 20 and greedy ltl 20 are better ranked than rand ltl 50 and greedy ltl 50 respectively. These results demonstrate how control mechanisms can effectively identify and evaluate the performance of strategies during the search.

Table ?? summarises all experiments and shows the overall ranking of the eleven approaches on each instance size and shows the final average ranks. Regarding the three deterministic approaches, the imp approach is always ranked last; the ndom approach is almost always ranked first, only beaten on the 20jobs 20-machines instance where it is outperformed by all adaptive approaches. Regarding the adaptive approaches, both the 2-arm approaches rand 2 and greedy 2 are the best performing, then are ranked the ltl 20 ones and the ltl 50 ones. The approaches using the random control mechanism generally perform worse than the ones using the ε-greedy mechanism, especially for the 3-arm variants and the long-term learning variants. Interestingly, even the random adaptive approach performs really well when considering the two imp ndom and ndom arms, potentially meaning that the adaptive algorithms will achieve very good results as long as there is no critically bad performing arm available. However, the long-term learning variants show that it is possible to identify, remove and recover in such event.

Conclusion

In this paper we presented an adaptive version of MOLS algorithms by introducing control mechanisms. If many control mechanisms can be found in the literature, they have mostly been applied to single objective bio-inspired algorithms; we investigated here their impact on a multi-objective iterative local search algorithm on the classical bi-objective bPFSP. We restrained the control on a single parameter, the exploration strategy, using two different types of control mechanisms: one uniform random and one feedback-based.

Our results show that on the studied problem, taken individually, the three considered explorations strategies performs very differently, and that adaptive approaches can achieve results statistically equivalent to the best strategy. We verified that identifying the best performing strategy before the search, using for example an offline automatic algorithm configurator, can lead to substantial improvement of the algorithm performance. Moreover, we show that very basic control mechanisms can achieve similar outcomes as long as the worst performing strategies are quickly discarded. Identification of the arm qualities continues to critically impact the performance of the overall algorithm, but the long-term learning approaches show that it is possible to introduce this knowledge during the search instead. It means that if a preliminary offline configuration of the algorithm results on multiple high performing mechanisms, it should be possible to simply postpone the decision and to adaptively decide during the search for each tackled instance.

In future work, it would be necessary to first investigate the addition of more possible arms to the MOLS structure, for example by differentiating the exploration reference, in order to compare more complex and potentially more efficient control mechanisms. Moreover, it would be helpful not to fix strategies a priori. Therefore, we would investigate the potential of control mechanisms to simultaneously manage different strategies. Finally, a single bi-objective problem was considered in this paper, in which the performance of each exploration strategy did not vary much. As MOLS algorithms have also been applied to other problems such as the travelling salesman problem or the quadratic assignment problem, for which the optimal MOLS configurations differs, it would be interesting to investigate adaptive MOLS on these problems.

  Algorithm 1: Basic Multi-Objective Local Search (mols)

	Input: A set of solutions, an exploration strategy
	Output: A set of solutions
	archive ← initial set of solutions;
	until termination criterion is met do
	/* Select a random solution */
	selected ← select 1 rand (archive);
	/* Apply the given exploration strategy */
	reference ← archive;
	accepted ← exploration strat (current, reference);
	/* Update the archive with the accepted neighbours */
	archive ← bounded pareto (archive, accepted);
	return archive ;

Table 1 .

 1 Experimentations summary

	Type	Approach 3 arms 2 arms LTL
	Deterministic	imp
	Deterministic	imp ndom
	Deterministic	ndom
	Random	rand 3
	Random	rand 2
	Random	rand ltl 50
	Random	rand ltl 20
	ε-greedy	greedy 3
	ε-greedy	greedy 2
	ε-greedy	greedy ltl 50
	ε-greedy	greedy ltl 20

Table 2 .

 2 Experimental ranking

	Instance			3 arms			2 arms		LTL rand	LTL ε-greedy
	N M	imp	imp ndom	ndom	rand 3	greedy 3	imp ndom	ndom	rand 2	greedy 2	rand 3	rand 2	rand ltl 50	rand ltl 20	greedy 3	greedy 2	greedy ltl 50	greedy ltl 20
	20 5	5 4 1 1 1	4 1 1 1	4 1 3 1	1 1 1 1
	20 10	5 4 1 1 1	4 1 1 1	4 1 1 1	1 1 1 1
	20 20	5 3 3 1 1	3 3 1 1	2 1 2 2	1 1 1 1
	50 5	5 4 1 1 1	4 1 1 1	4 1 1 1	1 1 1 1
	50 10	5 4 1 1 1	4 1 1 1	4 1 1 1	4 1 1 3
	50 20	5 4 1 1 1	4 1 1 1	4 1 1 1	4 1 1 1
	100 5	5 4 1 1 1	4 1 1 1	4 1 3 1	4 1 3 1
	100 10	5 1 1 1 1	4 1 1 1	4 1 3 1	4 1 3 1
	100 20	5 2 1 2 2	4 1 2 2	4 1 3 1	4 1 3 1
	200 10	5 1 1 3 3	4 1 1 1	4 1 2 2	4 1 3 1
	200 20	5 2 1 3 3	4 1 1 1	4 1 3 2	4 1 2 2
	500 20	5 1 1 3 3	1 1 1 1	3 1 3 2	3 1 2 3
	average	5 2.8 1.2 1.6 1.6	3.7 1.2 1.1 1.1	3.8 1 2.2 1.3	2.9 1 1.8 1.4

Table 3 .

 3 Complete ranking

	Instance
	N M	imp	imp ndom	ndom	rand 3	rand 2	rand ltl 50	rand ltl 20	greedy 3	greedy 2	greedy ltl 50	greedy ltl 20
	20 5	11 10 1 9 1 8 1 1 1 1	1
	20 10	11 10 1 9 1 1 1 1 1 1	1
	20 20	11 9 9 6 1 6 6 1 1 1	1
	50 5	11 10 1 9 1 1 1 1 1 1	1
	50 10	11 10 1 9 1 5 5 8 1 1	5
	50 20	11 10 1 7 1 1 1 9 1 7	1
	100 5	11 9 1 10 1 7 5 8 1 5	1
	100 10	11 7 1 10 1 7 1 9 1 6	1
	100 20	11 6 1 9 2 6 2 9 2 8	2
	200 10	11 4 1 9 1 6 6 9 1 6	4
	200 20	11 7 1 10 1 7 1 9 1 6	1
	500 20	11 6 1 9 2 6 2 9 2 8	2
	average	11 8.2 1.7 8.8 1.2 5.1 2.7 6.2 1.2 4.3 1.75


D Chaudy
email: dchaudy@laas.fr

O Llopis

B Marcilhac

Y Lemaitre

O D'

Allivy Kelly

J.-M Hode

Low phase noise cryogenic amplifiers and oscillators based on superconducting resonators

Keywords: microwave oscillator, phase noise, superconductor resonator, cryoelectronics I

A cryogenic low phase noise amplifier and an high Q superconductor resonator at 1 GHz have been designed and realized. A good agreement between the measured and simulated data at 80 K for these two devices is observed. An all cryogenic oscillator has also been designed with the same devices on an alumina substrate. This oscillator is still under test.

INTRODUCTION

High Tc superconductor materials allows the design of ultra high Q resonators in the low microwave range. The design of an oscillator with these resonators requires an accurate modelling of the oscillating loop, which includes not only the resonator but also the transistor and the passive devices (capacitances, resistances, transmission lines). If the loop can be designed for an all-cryogenic operation, the oscillation stability will be increased. However, this is a hard task because any tuning of a device requires a time consuming process of cooling up and down the circuit. Therefore, the model of the loop elements at low temperature must be very accurate. If this step can be overcome, it could open the path to a disruptive technology. Indeed, the simulated phase noise results outperform the best commercially available 1 GHz sources, as presented in our previous work [START_REF] Chaudy | Active devices choice and design of an all cryogenic superconductor resonator oscillator[END_REF].

In a first step, we present the design of a low phase noise cryogenic amplifier. This device is easier to characterize than the closed oscillating loop, and allows us to understand the problems which may appear in the oscillator design. Then, the superconducting resonator characterization and modelling is described. The integrated cryogenic oscillator is currently under test but preliminary measurements show interesting results, with a phase noise level very close to the state of the art quartz based sources..

II. LOW TEMPERATURE AND PHASE NOISE AMPLIFIER

Designing a low phase noise amplifier has never been an easy task and the difficulty is enhanced when all the devices are at cryogenic temperature. Every device must be carefully chosen and modelled. That is why, as presented in the last IFCS conference [START_REF] Chaudy | Active devices choice and design of an all cryogenic superconductor resonator oscillator[END_REF], characterization and modeling of SiGe transistors has been performed at 80 K. An amplifier has been designed on an alumina substrate using this model. Knowing that our final goal is the realization of an oscillator at 1 GHz, we focus our designs and measurements around this frequency.

We have thus designed a single stage amplifier using a common emitter bipolar SiGe transistor. In order to stabilize the amplifier and to optimize its residual phase noise, several resistances and capacitors have been added, particularly to realize two feedback paths: one series feedback on the emitter and one parallel feedback between base and collector. The amplifier topology is presented in Fig 1. To characterize the circuit at cryogenic temperature, the amplifier has been partially immerged in liquid nitrogen. S-parameters and noise measurement have been performed both at ambient and low-temperature. A good agreement is observed between simulations and measurements, as shown in Fig 2 for S-parameters at 80 K. The amplifier residual phase noise has also been measured and is depicted in Fig 3 . The measured phase noise floor is in the range of -170 dBrad 2 /Hz and is very similar to the one simulated. However, an excess noise is observed close to the carrier (below 100 Hz) compared to the simulated 1/f phase noise level. This excess noise may be caused by vibrations and spurious signals. Indeed, the amplifier was not in a shielded housing at the time of the experiment. The next experimental step for the amplifier is to decrease the temperature down to 60K. Up to now, all the measurements have been performed using liquid nitrogen, at approximately 80K. The use of a cryocooler will allow us to reach 60 K. The experiment is ready as shown in Fig. 4 but measurement are still on-going.

III. HIGH-TEMPERATURE SUPERCONDUCTOR RESONATOR

The other important part of oscillator is the resonator. In the past few years, Thales research on high-temperature superconductor has led to the development of planar resonators with quality factors in the range of 200 000 near 1 GHz. After the design of a test alumina for the resonator, we have measured its frequency response at 80 K (Table 1

, 2 and Fig 5).

Four interesting parameters can be extracted from those measurements: the loaded quality factor, which is 55700 at low microwave power, the input/output coupling factors and the transmission losses. From these parameters, the computed unloaded Q is 190000, as expected. A resonator model is then extracted. In a first step, a linear black box model has been used. Then, in a second step, an electrical RLC circuit coupled with two transformers has been implemented. This model allows us to include various nonlinear elements to take into account the RF power dependence of the resonator frequency response. Indeed, as depicted in Table 1 and 2, the insertion losses and the Q factor are degraded at high RF power. This is a well-known behavior in superconducting cavities [START_REF] Kermorvant | [END_REF][START_REF] Oates | Temperature dependence of intermodulation distortion in YBCO: understanding nonlinearity[END_REF], due to normal current generation at high current density. This behavior has first been described by a nonlinear resistor in the RLC circuit model ( 1). However, a distortion of the resonance can also been observed at high power. This effect is also discussed in some papers [START_REF] Oates | Temperature dependence of intermodulation distortion in YBCO: understanding nonlinearity[END_REF][START_REF] Wosik | Microwave power handling capability of HTS superconducting thin films: weak links and thermal effects induced limitation[END_REF] and can be described by adding a non-linear inductor (2).

IV. OSCILLATOR These two devices, amplifier and resonator, have been assembled to realize a cryogenic oscillator (Fig. 7). All the oscillator loop is cooled down to cryogenic temperature. The only part which is not cooled, is the battery based biasing network. For all the experimental investigations, the biasing network is kept outside the cryocooler in order to provide some tuning parameters. However, in the end, it will be included on the alumina substrate so that only one wire (to connect to the battery) and one SMA cable (the RF output) will go out of the cryocooler.

The first simulations on ADS software have led to a phase noise level close to -140 dBc/Hz at 1 kHz offset from 1 GHz carrier. The first experimental measurement, presented in Fig. 8, features an higher noise level than expected, close to -130 dBc/Hz at 1 kHz offset from carrier. After improving our model, the gap between simulation and measurement has been reduced (Fig. 8), but we still have to understand the increase of phase noise compared to the simulated one. One possible cause may be an excessive intra-cavity power in the resonator.

In any case, the oscillator performance can be improved using an higher Q resonator. A new resonator has thus been realized, featuring an higher loaded quality factor thanks to lower coupling factors and higher intrinsic Q, as presented in Table 3. For the moment, the modeling is still in process, but the use of this resonator may allow us to outperform the best commercially available 1 GHz sources. V. CONCLUSION Measurements and simulations of an amplifier and a superconductor resonator at 80 K have been presented. These devices have been assembled to realize a cryogenic oscillator. This circuit is, up to our knowledge, the first low phase noise full cryogenic oscillator at 1 GHz using superconductor resonators. The phase noise performance is good (-130 dBc/Hz at 1 kHz offset from 1 GHz carrier), even if we still do not reach exactly the state of the art of 1 GHz sources. Further improvements are expected thanks to a new resonator and some design improvements.

Fig. 1 . 1

 11 Fig. 1. 1 GHz cryogenic amplifier.

Fig. 2 .

 2 Fig. 2. Amplifier simulated and measured S-parameters curves at IC = 5 mA, VCE = 2 V, T = 80 K.

Fig. 3 .

 3 Fig. 3. Amplifier simulated and measured 1 GHz phase noise at IC = 5 mA, VCE = 2 V, T = 80K.

Fig. 4 . 1

 41 Fig. 4. 1 GHz cryogenic amplifier on a coldfinger.

Fig. 5 .

 5 Fig. 5. Transmition parameters of an HTS-Resonator at 80 K (f0 ~ 1GHz)

Fig. 7. 1

 1 Fig. 7. 1 GHz cryogenic oscillator

TABLE I .

 I HTS-RESONATOR MEASURE CARACTERISTICS

	Input Power (dBm)	S21-Max (dB)	Loaded Q	UnLoaded Q
	-30	-3.7	55 700	190 000
	-20	-3.7	55 500	190 000
	-10	-4.2	51 000	160 000
	0	-8.5	26 100	49 000
	TABLE II.	HTS-RESONATOR SIMULATED CARACTERISTICS
	Input Power (dBm)	S21-Max (dB)	Loaded Q	UnLoaded Q
	-30	-3.7	55 200	180 000
	-20	-3.8	54 500	180 000
	-10	-4.6	48 000	140 000
	0	-8.4	26 700	49 000

TABLE III .

 III NEW HTS-RESONATOR MEASURED CHARACTERISTICS

	Input Power (dBm)	S21-Max (dB)	Loaded Q	UnLoaded Q
	-30	-6.2	190 000	370 000
	-20	-6.3	180 000	350 000
	-10	-7.0	160 000	285 000
	0	-9.4	105 000	160 000

This work is supported by the French MoD, Direction Générale de l'Armement (DGA)