Thierry Hance
email: thierry.hance@uclouvain.be

Muhammad Mubashir Saeed

Kévin Tougeron

Abu Bakar

Muhammad Raza

Muhammad Afzal

Anjum Aqueel

Guillaume Jean Le Goff

François Renoz

Jennifer Pirotte

Maternal effects, diapause and life-history traits

Keywords: dormancy, life history traits, mummy color, photo-thermal regimes, transgenerational plasticity

Diapause is an adaptation that insects have evolved to synchronize their life cycle with that of seasonal climatic changes and resources availability. However, cues for its induction are not always clear and, in some cases, a maternal effect may be involved. At the population level, This article is protected by

Introduction

Adaptive phenotypic plasticity allows organisms to respond to environmental changes at different times of their life-cycle [START_REF] Sinclair | Insects at low temperatures: an ecological perspective[END_REF][START_REF] Pigliucci | Phenotypic plasticity and evolution by genetic assimilation[END_REF]. As temperature is one of the main factors affecting ectotherms development and activity [START_REF] Hallman | Temperature Sensitivity in Insects and Application in Integrated Pest Management[END_REF], facultative diapause has evolved as a particular type of developmental phenotypic plasticity. Winter diapause is usually induced by well-defined environmental cues, such as a photoperiod and temperature decrease, that precede the arrival of unfavorable conditions [START_REF] Tauber | Seasonal Adaptations of Insects[END_REF]. It is distinguished from quiescence which is a direct and short-term response to stressful conditions [START_REF] Tauber | Seasonal Adaptations of Insects[END_REF].

In temperate areas, the proportion of individuals that undergo diapause increases as the time left for growth and development before the arrival of harsh conditions becomes limited. However in a population, not all the individuals will respond to environmental cues inducing diapause with the same intensity (i.e., at the same threshold), resulting in reaction norm variability among genotypes [START_REF] Bradford | Bet hedging and the diapause strategies of the cricket Allonemobius fasciatus[END_REF][START_REF] Hopper | Risk-spreading and bet-hedging in insect population biology[END_REF]. In temperate climates most aphid parasitoids arrest their growth or activity before winter time and lie dormant inside the dead host cuticle [START_REF] Langer | Overwintering strategies and cold hardiness of two aphid parasitoid species (Hymenoptera: Braconidae: Aphidiinae)[END_REF][START_REF] Polgár | Diapause induction in aphid parasitoids[END_REF].

However, among parasitoids associated with anholocyclic cereal aphids under mild winter only a proportion of the population undergoes true diapause, the other part can remain active during winter if climatic conditions allow these parasitoids to terminate their development [START_REF] Polgár | Induction of dormancy in aphid parasitoids: implications for enhancing their field effectiveness[END_REF][START_REF] Langer | Overwintering strategies and cold hardiness of two aphid parasitoid species (Hymenoptera: Braconidae: Aphidiinae)[END_REF]Tougeron et al., 2017b).

The recent work of [START_REF] Andrade | Impact of change in winter strategy of one parasitoid species on the diversity and function of a guild of parasitoids[END_REF] showed that the increase in average winter temperatures could change the composition of parasitoid communities in cereal crops probably because some species undergo changes among the proportions of individuals which enter diapause. A rapid shift in aphid-parasitoid communities was observed in Western France over the last three decades due to modifications in overwintering strategies of insects (Tougeron et al., 2018a) and of top-down and bottom-up effects [START_REF] Chidawanyika | Global climate change as a driver of bottom-up and top-down factors in agricultural landscapes and the fate of host-parasitoid interactions[END_REF], reflecting the loss of diapausing stage by some species, or at least indicating that conditions for diapause induction are no longer encountered. Some parasitoid strains show low levels of diapause, either because they have been maintained in constant summer-like rearing conditions (e.g., commercially available strains), because they have evolved with very mild or no winter conditions in their area of origin, or because of adaptation to climate-warming [START_REF] Gariepy | Why two species of parasitoids showed promise in the laboratory but failed to control the soybean aphid under field conditions[END_REF][START_REF] Tougeron | How climate change affects the seasonal ecology of insect parasitoids[END_REF].

The aphid parasitoid Aphidius ervi (Hymenoptera: Braconidae) is a generalist parasitoid of aphids which can be found in numerous cultivated crops around the world and in which commercial strains are also available for biological control (Van Emden & Harrington, 2017). In A. ervi, diapause occurs at the prepupae instar inside the aphid host cuticle (called a -mummy‖). In this species, continuous exposure to 15 °C at a photoperiod of 9 : 15 h L : D (Light : Dark) for three consecutive generations produced an increase in the number of diapausing individuals up to a maximum of 42.2% [START_REF] Langer | Overwintering strategies and cold hardiness of two aphid parasitoid species (Hymenoptera: Braconidae: Aphidiinae)[END_REF]. In addition to direct perception of environmental cues by the diapausing generation, these results imply the existence of a maternal effect. Such transgenerational effects in which conditions perceived by one generation induce diapause in the following generation have already been described in numerous other species such as in mosquitoes [START_REF] Lacour | When mothers anticipate: Effects of the prediapause stage on embryo development time and of maternal photoperiod on eggs of a temperate and a tropical strains of Aedes albopictus (Diptera: Culicidae)[END_REF] or fly parasitoids [START_REF] Saunders | Larval diapause of maternal origin: induction of diapause in Nasonia vitripennis (Walk.)(Hymenoptera: Pteromalidae)[END_REF]. In the case of A. ervi, a previous paper suggested that diapause levels and life-history traits can be modulated by conditions experienced by precedent generations, due to transgenerational plasticity (TGP) [START_REF] Tougeron | Trans-generational effects on diapause and life-history-traits of an aphid parasitoid[END_REF].

Among other mechanisms, TGP can involve epigenetics [START_REF] Jablonka | Transgenerational epigenetic inheritance: prevalence, mechanisms, and implications for the study of heredity and evolution[END_REF], which is known to act on phenotypic expression, including diapause [START_REF] Reynolds | Epigenetic Influences on Diapause[END_REF]. Acclimation within and across generations is generally thought to be adaptive for insects [START_REF] Basson | Fitness costs of rapid cold-hardening in Ceratitis capitata[END_REF]. Adaptive acclimation effects through TGP on overwintering strategies have been shown in overwintering insects, such as in the grain aphid Sitobion avenae (Hemiptera: Aphididae) [START_REF] Powell | Intergenerational acclimation in aphid overwintering[END_REF].

Life-history traits are also subject to phenotypic plasticity [START_REF] Travis | Evaluating the adaptive role of morphological plasticity[END_REF] and their expression can be influenced by diapausing states or to diapause-inducing cues. If diapause is adaptive to face adverse conditions, it also represents a fitness cost in term of survival of diapausing stages and for emerging adults [START_REF] Hance | Impact of extreme temperatures on parasitoids in a climate change perspective[END_REF]. For instance, the parasitoid Asobara tabida showed a negative relationship between the time spent in diapause and egg load in adults [START_REF] Ellers | A trade-off between diapause duration and fitness in female parasitoids[END_REF]. For the aphid parasitoid Praon volucre Haliday, diapause is also associated with a reduction of adult longevity [START_REF] Colinet | Cold-induced expression of diapause in Praon volucre: fitness cost and morpho-physiological characterization[END_REF]. In that case, diapausing mummies present a higher lipid mass than non diapausing ones. This storage of lipids could be used both to ensure survival during a long period of diapause but also as energy resources for the production of eggs after diapause. We can therefore expect exacerbated trade-offs between survival and fertility as shown for A. tabida [START_REF] Ellers | A trade-off between diapause duration and fitness in female parasitoids[END_REF], or between lipid storage and diapause [START_REF] Varpe | Trade-offs between storage and survival affect diapause timing in capital breeders[END_REF]. Moreover, costs also arise from the need to resist cold exposure during overwintering [START_REF] Langer | Overwintering strategies and cold hardiness of two aphid parasitoid species (Hymenoptera: Braconidae: Aphidiinae)[END_REF][START_REF] Hodkova | Photoperiod, diapause and cold-hardiness[END_REF]. Thus fitness related traits (e.g., female fecundity) may be affected either by cold storage or by diapause itself, even over several generations [START_REF] Ellers | A trade-off between diapause duration and fitness in female parasitoids[END_REF][START_REF] Levie | Consequences of acclimation on survival and reproductive capacities of cold-stored mummies of Aphidius rhopalosiphi (Hymenoptera: Aphidiinae)[END_REF]. Life-history traits such as fecundity, survival, size, energetic reserves and developmental time are therefore usually measured to evaluate parasitoid fitness [START_REF] Godfray | Parasitoids: Behavioral and Evolutionary Ecology[END_REF][START_REF] Roitberg | Fitness, parasitoids, and biological control: an opinion[END_REF] and consequence of cold exposure or diapause [START_REF] Colinet | Cold-induced expression of diapause in Praon volucre: fitness cost and morpho-physiological characterization[END_REF][START_REF] Ismail | Stress intensity and fitness in the parasitoid Aphidius ervi (Hymenoptera: Braconidae): temperature below the development threshold combined with a fluctuating thermal regime is a must[END_REF].

Within insect populations that need to overwinter, color polymorphism has been commonly reported [START_REF] Leather | The Ecology of Insect Overwintering[END_REF]. In the case of A. ervi, mummies have been described as light brown, intermediate and dark brown. Previous research stated that dark brown mummies usually correspond to a state of diapause (Brodeur & McNeil, 1989b;[START_REF] Krespi | A morphometric comparison of aphid mummies containing diapausing vs. non-diapausing larvae of Aphidius rhopalosiphi (Hymenoptera: Braconidae)[END_REF][START_REF] Polgár | Induction of dormancy in aphid parasitoids: implications for enhancing their field effectiveness[END_REF][START_REF] Langer | Overwintering strategies and cold hardiness of two aphid parasitoid species (Hymenoptera: Braconidae: Aphidiinae)[END_REF]. In another aphid parasitoid, Praon volucre, dark colored mummies are always correlated to a significant delay/arrest in development of the parasitoid with also significant reduced metabolic rates whose are characteristic of diapausing insects [START_REF] Colinet | Cold-induced expression of diapause in Praon volucre: fitness cost and morpho-physiological characterization[END_REF]. However, in a related species Aphidius rhopalosiphi, mummy color characteristics was not found as a reliable marker for identifying diapause and is most probably a phenotypic variation only linked to cold resistance (Legrand et al., 2004b, c). In consequence, plasticity in the color of the parasitized mummy seems to be very species-specific, must be taken with caution when considering diapause indicators and must be completed with other, more reliable methods such as mummy dissections.

In this study, we aimed to test whether diapause levels could vary depending on the photothermal regime and how this effect can be amplified during successive generations of a commercial strain of the parasitoid A. ervi. Entering diapause under spring-like or summerlike conditions should be avoided because of expected costs, unless competition is too high (Tougeron et al., 2017a). Transgenerational plasticity (acclimation) in response to environmental cues might be expressed because it should be adaptive and have benefits regarding fitness. Plastic response allows rapid responses to changing conditions and it is this aspect we want to test over a very small number of generations. We explored mummy coloration patterns and emergence delays as indicators of the state of diapause. Moreover, we evaluated how photo-thermal regimes could influence the value of a set of fitness-related parasitoid life-history traits. We hypothesized that low temperatures and photoperiods would increase the number of diapausing parasitoids, and that diapause levels would be higher at the second generation experiencing such conditions. We also predicted that fitness would be reduced at low temperature and short photoperiod, due to the physiological costs induced by these conditions.

Materials and methods

Biological material and insect rearing

Cereal aphids Sitobion avenae (Aphididae: Hemiptera) were obtained from Koppert (The Netherland), and they were maintained on wheat seedlings (Triticum aestivum L.) in 0.3 m 3 cages under laboratory conditions (18 °C, 16 : 8 h L : D, 65% ± 5% relative humidity).

Parasitoids A. ervi were purchased from Viridaxis S.A. (Belgium) and reared on S. avenae under the same laboratory conditions. The experiment was conducted in the laboratory of Earth and life Institute, Biodiversity Research Centre, Université catholique de Louvain, Belgium.

We checked for the presence of the facultative endosymbionts Hamiltonella defensa, Serratia symbiotica and Regiella insecticola in our aphid culture because their presence could affect parasitism rates and diapause [START_REF] Douglas | Nutritional interactions in insect-microbial symbioses: aphids and their symbiotic bacteria Buchnera[END_REF][START_REF] Mushegian | Host-microbe interactions in the context of diapause[END_REF]. The infection status was verified using diagnostic PCRs with primers 16SA1 (5′-AGAGTTTGATCMTGGCTCAG-3′) and PASS cmp (5′-GCAATGTCTTATTAACACAT-3′) (Fukatsu et al, 2000;Attia et al., 2016). None of the facultative symbionts we tested were found in our aphid clone.

Parasitism and photo-thermal regimes

Parasitoid females used in the experiment were less than 48 h old and naive. In order to ensure mating, each female was maintained for 24 h at 18°C with 1 male in an Eppendorf tube (1.5 mL) and fed with 50% honey solution. After mating, each female was presented a batch of 10 aphids S. avenae (3-4 days old, L2 instar) in a glass petri dish (Ø 4.5 cm) and stings were observed directly. Once parasitized, batches of 10 aphids were removed from the petri dish and transferred to a 4-6 days old fresh wheat seedlings cultured in glass vials (centrifuge tube 50 mL). The top of glass vial was closed by mesh cloth for ventilation.

Diapause induction, developmental delay and life-history traits of A. ervi were analyzed under four crossed photo-thermal regimes; 2 temperature regimes (8°C and 16°C) and 2 photoregimes (16 : 8 h L : D and 8 : 16 h L : D), over 2 consecutive generations. These conditions match the range of temperatures and photoperiod parasitoids encounter in the field in Belgium at the end of autumn and induced diapause in a related species Aphidius rhopalosiphi (Legrand et al., 2004a;Tougeron et al., 2017b). Parasitized aphids were distributed in four different incubators and were kept at these conditions until mummy formation and adult emergence, following the protocol of Tougeron et al. (2019b). For each photo-thermal regime and generation, 200 aphids were parasitized (by 20 different females) and we managed to obtain between 80 and 191 mummies. After measuring the desired parameters on the 1 st parasitoid generation, the 2nd generation was obtained from parents that experienced the same treatment in the same way as described above.

Diapause induction, development time and mummy coloration

Mummies were kept upon formation separately in small plastic tubes under their respective photo-thermal regimes. Mummies were characterized as light brown, intermediate and dark brown following the categories described in previously published work [START_REF] Colinet | Cold-induced expression of diapause in Praon volucre: fitness cost and morpho-physiological characterization[END_REF] and presented in Figure 1.

Day-degrees necessary for an individual insect's development was calculated from developmental time of egg laying to mummy formation and after mummy formation to development of adult at different photo-thermal regimes for 2 generations. Developmental time on a day-degree (DD) time scale was calculated as: DD = DT(T-T o ), where DT is the calculated developmental time in days, T is the temperature in ºC, and T o is the threshold temperature for development of A. ervi, which is 2.2ºC [START_REF] Sigsgaard | The temperature-dependent duration of development and parasitism of three cereal aphid parasitoids, Aphidius ervi, A. rhopalosiphi, and Praon volucre[END_REF]. We also measured mummy formation date, time from egg-laying to emergence and time from mummy formation to parasitoid emergence.

Parasitoids which did not emerged 3 weeks after mummy formation were shifted to 16°C. If they did not emerged after 3 more weeks (total of 6 weeks), mummies were dissected to determine if they contained alive larvae still in the state of diapause or dead ones [START_REF] Colinet | Cold-induced expression of diapause in Praon volucre: fitness cost and morpho-physiological characterization[END_REF]Tougeron et al., 2017b). In that case, after dissection, alive larvae were considered to be under diapause as the delay of emergence was over three weeks once placed at 16°C for which quiescence should be naturally interrupted to continue development. This helped to test the reliability of the diapause identification method based on the coloration of the mummy.

Trait measurements

We measured success of emergence, adult longevity, fecundity, and sex ratio at each photo-thermal regime and for each generation. For survival, 20 male and 20 female parasitoid adults were individually placed in Eppendorf tube (1.5 mL) and fed with a 50% honey solution at 20°C, as done by [START_REF] Levie | Consequences of acclimation on survival and reproductive capacities of cold-stored mummies of Aphidius rhopalosiphi (Hymenoptera: Aphidiinae)[END_REF], and for each treatment. Survival of each adult parasitoid was monitored twice a day until the last parasitoid died. To measure potential fecundity of A. ervi, <24h emerged females were freeze-killed at -20°C and dissected (n = 20 for each treatment) and the number of eggs at emergence was counted as described by [START_REF] Lins | Cold storage affects mortality, body mass, lifespan, reproduction and flight capacity of Praon volucre (Hymenoptera: Braconidae)[END_REF].

For morphological traits, 40 additional one-day-old adult parasitoids (20 males and 20 females) were killed by cold exposure at -20°C for 10 minutes and we then measured size, fresh mass, dry mass, fat mass, and fat contents [START_REF] Ismail | Physiological costs of cold exposure on the parasitoid Aphidius ervi, without selection pressure and under constant or fluctuating temperatures[END_REF], for each treatment.

Fresh mass was measured using a Mettler-electrobalance Me22 (sensitivity: 1 µg). Size was measured using the length of the left hind tibia as a standard estimation for aphid parasitoids [START_REF] Godfray | Parasitoids: Behavioral and Evolutionary Ecology[END_REF][START_REF] Ismail | What are the possible benefits of small size for energy-constrained ectotherms in cold stress conditions?[END_REF]. Digital pictures of hind tibia were captured with Nikon 4500 camera mounted on a stereomicroscope. The images captured were analyzed by ImageJ v. 1.48 software (Wayne Rasband, USA). The dry mass was measured after the samples/adults were oven dried for 3 days at 60°C. Each sample was left in Eppendorf tube containing 1 mL of extracting solution (chloroform : methanol solution 2 : 1) on agitator for 2 weeks [START_REF] Terblanche | Metabolic rate variation in Glossina pallidipes (Diptera: Glossinidae): gender, ageing and repeatability[END_REF][START_REF] Colinet | Water relations, fat reserves, survival, and longevity of a cold-exposed parasitic wasp Aphidius colemani (Hymenoptera: Aphidiinae)[END_REF]. Adults were then placed in an air oven for 12 h at 60°C to eliminate residues of the extracting solution and to obtain the lean dry mass. Fat mass and fat contents were calculated using the formulas: Fat mass = Dry mass -Lean dry mass Fat contents = Fat mass / Lean dry mass following [START_REF] Terblanche | Metabolic rate variation in Glossina pallidipes (Diptera: Glossinidae): gender, ageing and repeatability[END_REF], [START_REF] Colinet | Water relations, fat reserves, survival, and longevity of a cold-exposed parasitic wasp Aphidius colemani (Hymenoptera: Aphidiinae)[END_REF].

Water mass was determined as the difference between fresh and dry mass. The water content (WC) was calculated as a proportion of dry mass using the formula:

Water Content = Water mass/ Dry mass following [START_REF] Colinet | Water relations, fat reserves, survival, and longevity of a cold-exposed parasitic wasp Aphidius colemani (Hymenoptera: Aphidiinae)[END_REF]

Statistical analyses

Coloration patterns were compared among generations and photothermal regimes using a chi-square test of independence, by considering the numbers of dark vs. white mummies for each tested condition. Intermediate coloration forms were ignored in this analysis but were counted and displayed in the graphic representation. This was done to analyze the differences in proportions of the two most contrasted colorations (dark vs. white).

Emergence patterns were fitted to a survival logistic-rank model (using package survival) by adding a censoring factor for non-emerged living parasitoids inside mummies. The effects of the generation, of the photoperiod and the temperature regimes, as well as their three-way interaction were considered and models were tested using Wald chisquares using the Anova function of the package car [START_REF] Fox | An R Companion to Applied Regression[END_REF]. Life-history trait data were compared among generations, photoperiod and the temperature regimes and their three-way interaction using GLMs with gaussian distribution and tested using a likelihood-ratio chisquare method [START_REF] Fox | An R Companion to Applied Regression[END_REF]. However, biological interpretation could not be made on two-way interactions of photoperiod and temperature with generation, because parasitoids from G2 are necessarily coming from the same photoperiod x temperature regime as those of G1. Tukey's HSD posthoc tests were performed to differentiate means. Previous work with similar experimental design on the same species did not find any -brood‖ effect, so all female lines and families were treated equally [START_REF] Tougeron | Trans-generational effects on diapause and life-history-traits of an aphid parasitoid[END_REF]. Statistical analyses were performed using R (R Core Team, 2019).

Results

Mummies characteristics

The percentage of brown colored mummies (individuals usually considered under diapause or dormant state) increased from generation 1 to generation 2, all regimes put together (Fig. 2, χ² = 84.542, df = 1, P < 0.001), and it also increased at photoregimes under low photoperiod (8 : 16 h L : D) (Fig. 2, χ² = 162.87, df = 3, P < 0.001). At least 72% of the mummies were brown at the second generation of the most stringent regime of 8°C 8 : 16 h L : D (Fig. 2).

Emergence delay

Emergence patterns were significantly affected by the photoperiod (χ²=350.4, df = 1, P < 0.001), the temperature regime (χ² = 773.9, df = 1, P < 0.001), and were different between generations (χ² = 286.96, df = 1, P < 0.001). There was an interaction effect between photoperiod and temperature (χ² = 157.2, df = 1, P < 0.001) but also between photoperiod and generation (χ² = 55.6, df = 1, P < 0.001), meaning that the photoregime did not affect development time in the same way depending on the considered generation (Fig. 3). Overall, parasitoids took longer to develop at the second generation than at the first one (10.05 ± 0.43 days vs. 5.98 ± 0.26 days, respectively). Parasitoids also took longer to develop at 8°C, 8 : 16 h L : D (15.66 ± 0.84 days) than at all of the other regimes (4.12 ± 0.10, 5.61 ± 0.22 and 6.82 ± 0.19 days for 16°C, 16:8 h LD, 16°C, 8:16 h LD and 8°C, 16 : 8 h L : D, respectively).

At 16°C, all parasitoids from the 180 mummies emerged within three weeks, while a proportion of the mummies did not emerge at 8°C, especially for the second generation placed at 8°C, 8 : 16 h L : D (up to 67.5%) (Table 1). Out of both photoregimes at 8°C, 123 (38 during 1st generation and 85 during 2nd generation) parasitoids did not emerged from their mummies after 3 weeks. After being shifted to 16°C, 52 parasitoids emerged rapidly, 39 parasitoids were found dead and 32 were still alive at the prepupae instar after three other weeks. Parasitoids that were found alive after mummy dissection were considered as diapausing individuals. Diapause levels reached 1.2% and 1.5% at 8°C, 16 : 8 h L : D (for G1 and G2, respectively) and 4.7% and 23.8% at 8°C, 8 : 16 h L : D (for G1 and G2, respectively) (Table 1). All the mummies that were dissected and in which a diapausing parasitoid was found were brown in colour.

At the first generation, emergence peaks occurred at day 3, 4, 5 and 5, with 72, 52, 41 and 33 parasitoids, respectively, at 16°C, 16 : 8 h L : D, 16°C, 8 : 16 h L : D, 8°C, 16 : 8 h L : D and 8°C, 8 : 16 h L : D, respectively. At the second generation, emergence peaks occurred at day 4, day 7, day 6 and day 18, with 35, 28, 25 and 10 parasitoids, respectively, at 16°C, 16 : 8 h L : D, 16°C, 8 : 16 h L : D, 8°C, 16 : 8 h L : D and 8°C, 8 : 16 h L : D, respectively (Fig. 3).

Results revealed that day-degrees for egg-to-mummy formation were the highest at 8°C, 8 : 16 h L : D, whereas lowest day-degree requirements were recorded at 16°C, 16:8 h LD. For mummy-to-adult duration, highest day-degree requirement was observed at 16°C, 8 : 16 h L : D, while lowest day-degrees requirement was observed at 8°C, 16 : 8 h L : D, for both generations (Table 2).

Life-history traits

There were significant differences in mean value of most of the measured life-history traits among photoperiod, temperature regimes, generations, with interaction effects of these factors (GLMs, Table 3,Table 4). For all traits but fat mass, there were significant interaction effects of photoperiod and temperature, meaning that the effect of photoperiod depended on the tested temperature, and reversely (Table 3). No generation effect was found on tibia size, dry weight and water content. Overall and all photoregimes put together, mean trait value of all other traits and longevity of both males and females were higher for the first than for the second generation. Egg-load was the only trait that was observed higher at the second generation. The effect of generation on fresh weight and longevity of both males and females was different depending on the photo-thermal regime tested (thee-way interaction effect). For both generations, mean trait values were overall different among regimes at 16°C and regimes at 8°C, although complex interactions were observed (Table 4).

Noticeably, during the 2nd generation at 8°C, 8 : 16 h L : D, minimum fresh weight (0.15 mg) was recorded. This very low value of fresh weight also corresponds to very low values of dry weight, tibia size and fat-content. Sex-ratio was male-biased for most of the treatments, especially at 8°C, 8 : 16 h L : D (Table 4).

Discussion

Surprisingly, we observed very low levels of diapause in A. ervi after mummy dissections, under all of the photo-thermal regimes tested. As expected, maximum diapause incidence was observed at 8°C, 8 : 16 h L : D, which represents conditions typically inducing diapause in aphid parasitoids from temperate areas [START_REF] Langer | Overwintering strategies and cold hardiness of two aphid parasitoid species (Hymenoptera: Braconidae: Aphidiinae)[END_REF]Tougeron et al., 2017b[START_REF] Tougeron | Trans-generational effects on diapause and life-history-traits of an aphid parasitoid[END_REF]. However, it concerned only around 25% of the mummies exposed to these conditions while Christiansen-Weniger and Hardie (1997) observed 100% diapause induction in fieldcollected populations of A. ervi after one generation at 15°C and 10 : 14 h L : D. The small proportion of diapausing parasitoids we found could be due to the commercial strain of A. ervi we used in this experiment and that has been maintained at constant temperatures and photoperiods for several years for mass release by the Viridaxis Company. Under continuous controlled rearing conditions, it is possible that this population has lost a part of its ability to enter diapause [START_REF] Tougeron | Trans-generational effects on diapause and life-history-traits of an aphid parasitoid[END_REF]. Releasing parasitoids with poor genetic diversity into the fields could impair biological control strategies and long-term establishment of natural enemies, because of low diapause levels and/or low resistance to adverse environmental conditions. Such an observation was already made for Binodoxys communis who showed poor capacity to enter diapause (less than 0.8%) and had probably lost that ability due to long period of quarantine rearing [START_REF] Gariepy | Why two species of parasitoids showed promise in the laboratory but failed to control the soybean aphid under field conditions[END_REF]. Here we used 8°C, but probably lower temperature can be more effective and differences among populations are to be expected due to local adaptations (Tougeron et al., 2018b).

Mummy characteristics such as coloration or cuticle thickness have been used to discriminate between diapausing and non-diapausing mummies but have not always proved to be reliable (Brodeur & McNeil, 1989a;[START_REF] Krespi | A morphometric comparison of aphid mummies containing diapausing vs. non-diapausing larvae of Aphidius rhopalosiphi (Hymenoptera: Braconidae)[END_REF][START_REF] Polgár | Induction of dormancy in aphid parasitoids: implications for enhancing their field effectiveness[END_REF]Legrand et al., 2004c). Dark mummies usually show a significant reduction in metabolic rate, which is typical of diapausing insects [START_REF] Tauber | Seasonal Adaptations of Insects[END_REF][START_REF] Hahn | Meeting the energetic demands of insect diapause: Nutrient storage and utilization[END_REF][START_REF] Colinet | Cold-induced expression of diapause in Praon volucre: fitness cost and morpho-physiological characterization[END_REF]. However, usually few developmental data are available to provide support for this statement, and the reliability of such cue may vary across species. Under temperature and short-day length, we observed higher percentages of dark brown mummies and a clear generational effect on mummy coloration and development timing of A. ervi. Our results revealed that at the most autumn-like regime (8°C, 8 : 16 h L : D), during 1st and 2nd generation, percentage of dark brown mummies was higher as compared to other photothermal regimes. The mummies which were dissected and in which parasitoids were found in diapause were all brown in color. According to Legrand et al. (2004a), change in mummy color could be an adaptation to overwintering, but not necessarily a direct consequence of diapause. In a related aphid parasitoid species, A. rhopalosiphi, dark mummies showed significantly higher water content, survival at low temperature and lower supercooling point values than pale mummies, but the proportion of colored mummies was not linked to diapause induction (Legrand et al., 2004b). Taken together, our results suggest that because the proportion of dark brown colored mummies increased after one generation at 8°C 8 : 16 h L : D, diapause levels could be high. However, as we only found a relatively small proportion of diapausing parasitoids using the dissection method, we cannot fully conclude on the reliability of the mummy coloration method.

The observed proportion of diapausing mummies, as well as development times, were higher for the 2nd generation at 8°C, 8 : 16 h L : D. In the same way, a generation effect was also observed for the coloration of the mummies with brown mummies more frequent in second generation. Such a potential maternal effect was already reported for instance for Cotesia vestalis [START_REF] Hao | Maternal effects of photoperiods on glycogen metabolism related to induction of diapause in Cotesia vestalis (Hymenoptera: Braconidae) from Jilin, China[END_REF] but also for Trichogramma parasitoids [START_REF] Voinovich | Maternal thermal effect on diapause in Trichogramma species (Hymenoptera: Trichogrammatidae)[END_REF] even on multiple generations [START_REF] Reznik | Multigenerational maternal inhibition of prepupal diapause in two Trichogramma species (Hymenoptera: Trichogrammatidae)[END_REF] and is usually observed for mosquitoes [START_REF] Denlinger | Mosquito diapause[END_REF][START_REF] Lacour | When mothers anticipate: Effects of the prediapause stage on embryo development time and of maternal photoperiod on eggs of a temperate and a tropical strains of Aedes albopictus (Diptera: Culicidae)[END_REF]. Adaptive role of TGP and mechanisms facilitating TGP are increasingly becoming elucidated [START_REF] Mousseau | Maternal effects in insect life histories[END_REF][START_REF] Sgrò | What can plasticity contribute to insect responses to climate change[END_REF][START_REF] Donelson | Transgenerational plasticity and climate change experiments: Where do we go from here?[END_REF]. Conditions perceived by females may influence contents of the egg which acts on the developmental plasticity of the embryo, or act on the next generation through changes in hormonal contents or epigenetic mechanisms [START_REF] Denlinger | Regulation of diapause[END_REF][START_REF] Uller | Developmental plasticity and the evolution of parental effects[END_REF][START_REF] Reynolds | Epigenetic Influences on Diapause[END_REF]. Such maternal effect would prepare offspring to better face incoming environment, which is known to happen in the natural environment [START_REF] Mousseau | Maternal effects in insect life histories[END_REF]. Although unlikely to happen under the non-lethal conditions we tested -because selection on diapause can only occur if parasitoids that do not enter diapause die (before reproduction) -non-random mortality at G1 is always possible so phenotypic changes observed within just one generation could either be due to TGP or to selection.

We found a male-biased sex-ratio for all conditions but more pronounced for the most stringent regime. However, it is possible that more females underwent diapause under our experimental conditions and so were part of the individuals that did not emerge rapidly and were dissected at the larval stage. Male biased sex ratios have been reported in some insects emerging from diapause, such as Lucilia caesar L. (Diptera: Calliphoridae) [START_REF] Ring | Variations in the photoperiodic reaction controlling diapause induction in Lucilia caesar L. (Diptera: Calliphoridae)[END_REF] and Asobora tabida Nees (Hymenoptera: Braconidae) [START_REF] Kraaijeveld | Variation in diapause and sex ratio in the parasitoid Asobara tabida[END_REF] and in Diprion pini L. (Hymenoptera: Diprionidae) [START_REF] Geri | Influence of consanguinity on the sex ratio and diapause of Diprion pini L. (Hym., Diprionidae) populations II.-Effect of inbreeding on diapause[END_REF]. Low temperatures are also known to bias sex-ratio in favor of males, which do not require as much resources for their development than females [START_REF] Langer | Oviposition, flight and walking capacity at low temperatures of four aphid parasitoid species (Hymenoptera: Aphidiinae)[END_REF][START_REF] Zamani | Hymenoptera: Braconidae), two[END_REF]. More recently in A. ervi it was found that females reared at 14 °C produced more males than females reared at 20 °C [START_REF] Tougeron | Trans-generational effects on diapause and life-history-traits of an aphid parasitoid[END_REF]. However, this does not explain why the progeny of A. ervi was male biased in our non-diapausing parasitoids. It has been suggested that high male proportion can be linked to laboratory conditions [START_REF] Heimpel | Sex ratios of commercially reared biological control agents[END_REF]. Time spent in diapause usually induces a higher mortality among diapausing pupae but also decrease egg load, fat reserves, and dry weight of the emerging adult females due to physiological trade-offs [START_REF] Ellers | Life history evolution in Asobara tabida: plasticity in allocation of fat reserves to survival and reproduction[END_REF]. Indeed, parasitoids have to deal with limited amount of resources provided by their host. Earlier researches showed such a tradeoff between the resources allocation to reproduction and survival [START_REF] Ellers | Life history evolution in Asobara tabida: plasticity in allocation of fat reserves to survival and reproduction[END_REF][START_REF] Ellers | The shape of the trade-off function between egg production and life span in the parasitoid Asobara tabida[END_REF]. In our case, there was a reduction in most of the fitness-related traits in parasitoids exposed to 8°C, 8 : 16 h L : D regime, especially at the second generation.

With the increase in development time, the emerging adults were stressed and tibia length was shorter. Although the link between diapause and fitness could only be indirectly assessed, because not all individuals underwent diapause at 8 °C, 8 : 16 h L : D, our results confirmed that exposure to diapause-inducing cues (photoperiod and temperature combination) has fitness consequences (Zhou et al., 1995;[START_REF] Ismail | Fitness consequences of low temperature storage of Aphidius ervi[END_REF].

We showed a reduction in size and fresh weight of adults under the 8°C, 8 : 16 h L : D regime for the second generation. Developmental temperature may modulate size of parasitic wasps [START_REF] Nealis | Temperature and development in host-parasite relationships[END_REF][START_REF] Bazzocchi | Effects of temperature and host on the pre-imaginal development of the parasitoid Diglyphus isaea (Hymenoptera: Eulophidae)[END_REF][START_REF] Ichiki | Effects of temperature on immature development of the parasitic fly Bessa parallela (Meigen)(Diptera: Tachinidae)[END_REF] and fitness indicators, such as longevity and fecundity. In addition, fat reserves have been shown to be positively correlated with size and mass [START_REF] Cloutier | Fecundity, longevity, and sex ratio of Aphidius nigripes (Hymenoptera: Aphidiidae) parasitizing different stages of its host, Macrosiphum euphorbiae (Homoptera: Aphididae)[END_REF][START_REF] Visser | The importance of being large: the relationship between size and fitness in females of the parasitoid Aphaereta minuta (Hymenoptera: Braconidae)[END_REF][START_REF] Ellers | Seasonal changes in female size and its relation to reproduction in the parasitoid Asobara tabida[END_REF][START_REF] Frère | Improvement in the cold storage of Aphidius ervi (Hymenoptera: Aphidiinae)[END_REF]. We have also shown that the longevity of adults was negatively affected by low photoperiod-temperature conditions. These findings can be compared with similar results in the literature [START_REF] Renault | Survival at low temperatures in insects: what is the ecological significance of the supercooling point?[END_REF][START_REF] Colinet | Water relations, fat reserves, survival, and longevity of a cold-exposed parasitic wasp Aphidius colemani (Hymenoptera: Aphidiinae)[END_REF][START_REF] Amice | Variability in responses to thermal stress in parasitoids[END_REF]. It is possible that females conserve most of their energy for their fecundity at low temperature, at the expense of their longevity, due to the trade-off longevity-fecundity [START_REF] Ellers | A trade-off between diapause duration and fitness in female parasitoids[END_REF][START_REF] Michaud | Reproductive diapause in Hippodamia convergens (Coleoptera: Coccinellidae) and its life history consequences[END_REF]. We suggest that females invested a part of their fat reserves in fecundity that could have been invested in longevity to resist the effects of cold storage, although egg-load was also negatively affected by low photothermal regimes, at both generations.

It can be predicted that change in phenology through modifications in diapause induction threshold can have dramatic consequences because if the appropriate environmental conditions are not reached, the proportion of individuals that may enter in diapause will be greatly reduced, threatening parasitoid survival in the field. How parasitoids will adapt their seasonal ecology to climate warming and climate unpredictability, and what will be the consequences for aphid-parasitoid interactions and biological control still remain challenging questions [START_REF] Tougeron | How climate change affects the seasonal ecology of insect parasitoids[END_REF]. Determining possible changes of threshold levels needed to induce diapause and their consequences on a set of life-history traits of emerging generation are of tremendous importance to better address these issues. authors are also thankful to the anonymous reviewers for their valuable comments that helped to improve the manuscript.

Disclosure

This paper is part of research publication from the PhD degree of Muhammad Mubashir Saeed. The author(s) declare that there is no conflict of interest. parasitoids of Aphis gossypii and Myzus persicae (Homoptera: Aphididae). Environmental Entomology, 36, 263-271. Zhou, X., Honek, A., Powell, W. and Carter, N. (1995) Tables Table 1 Emerging

  Variations in body length, weight, fat content and survival in Coccinella septempunctata at different hibernation sites. Entomologia Experimentalis et Applicata, 75, 99-107. Manuscript received January 14, 2020 Final version received April 16, 2020 Accepted April 24, 2020

  pattern of A. ervi adults during 1st and 2nd generation, at different photothermal regimes. Values in brackets are percentages. Total counts of mummies from different colors are displayed. The number of mummies from which nothing emerged after three weeks, the number of mummies from which parasitoids emerged when put back at 16°C and the number of dead and alive parasitoids in the remaining mummies are also indicated.

(

Figure captions Fig. 1 .

 captions1 Figure captions

Fig. 2

 2 Fig. 2 Percentage of mummies emerged during 1st and 2nd generation at different photo-

Fig. 3

 3 Fig. 3 Emerging pattern of 1st and 2nd generations A. ervi at different photo-thermal regimes.

Table 2

 2 Day-degrees necessary for egg-to-mummy and mummy-to-adult development at

	different photo-thermal regimes
	Photo-thermal regimes	Egg-Mummy G1 G2	Mummy-Adult G1 G2
	16°C, 16:8 h L:D	165.6	151.8	122.2	150.4
	16°C, 8:16 h L:D	179.4	193.2	141	197.4
	8°C, 16:8 h L:D	191.4	197.2	40.6	65.8
	8°C, 8:16 h L:D	232	243.6	92.4	121.8

Table 3

 3 Analysis of variance for life history traits for the different treatments. Note that a photothermal regime, as described in the method section, corresponds to a Photoperiod × Note regarding to-way interactions including the generation factor: parasitoids from G2 are necessarily coming from the same photoperiod x temperature regime as those of G1.

	Temperature interaction.
	Trait	Treatment				DF LR χ²	P (>χ²)
	Fresh weight	Photoperiod				1	0.17	0.68
		Temperature				1	10.45	<0.001***
		Potoperiod × Temperature		1	35.59	<0.001***
		Generation				1	8.87	<0.001***
		Generation × Photoperiod		1	0.69	0.40
		Generation × Temperature		1	0.08	0.78
		Generation Temperature	×	Photoperiod	×	1	7.69	<0.01 **
	Dry weight	Photoperiod				1	15.05	<0.001***
		Temperature				1	53.57	<0.001***
		Potoperiod × Temperature		1	37.47	<0.001***
		Generation				1	1.66	0.20
		Generation × Photoperiod		1	2.74	0.10
		Generation × Temperature		1	1.47	0.22
		Generation Temperature	×	Photoperiod	×	1	0.14	0.71
	Water contents	Photoperiod				1	24.21	<0.001***
		Temperature				1	39.46	<0.001***
		Potoperiod × Temperature		1	8.40	<0.01**
		Generation				1	1.44	0.23
		Generation × Photoperiod		1	0.46	0.50
		Generation × Temperature		1	1.51	0.22
		Generation Temperature	×	Photoperiod	×	1	5.76	<0.05 *
	Fat mass	Photoperiod				1	21.32	<0.001***

Table 4

 4 Mean ± S.E for life history traits parameters at different photo-thermal regimes.

	Photo-thermal	Generation	Fresh weight (mg)	Dry weight (mg)	Water contents	Fat (mg)	mass	Fat contents	Tibia size (mm)	Longevity (days)	Egg load
	regimes									Males	Females
	16°C, 16:8 h L:D	1	0.36 ± 0.11 ab	0.09 ± 0.004 a	4.00 ± 0.20 ab	0.030 ± 0.001 a	0.50 ± 0.03 b	0.46 ± 0.01 d	9.6 ± 0.65 cd	12.5 ± 0.48 cd	94.90 ± 8.11 b
	16°C, 8:16 h L:D	1	0.29 ± 0.01 c	0.06 ± 0.004 b	4.81 ± 0.26 a	0.027 ± 0.002 ab	0.71 ± 0.06 a	0.64 ± 0.01 a	11.3 ± 0.54 c	13.9 ± 0.43 c	83.65 ± 8.30 bc
	8°C, 16:8 h L:D	1	0.32 ± 0.01 bc	0.10 ± 0.003 a	3.07 ± 0.07 c	0.025 ± 0.001 abc	0.31 ± 0.02 cd	0.56 ± 0.01 bc	18.2 ± 0.29 a	25.5 ± 0.45 a	92.55 ± 11.21 b
	8°C, 8:16 h L:D	1	0.39 ± 0.01 a	0.11 ± 0.004 a	3.67 ± 0.12 bc	0.021 ± 0.002 bc	0.24 ± 0.02 de	0.64 ± 0.01 ab	17.9 ± 0.28 a	21.8 ± 0.59 b	43.70 ± 8.65 cd
	16°C, 16:8 h L:D	2	0.31 ± 0.08 bc	0.09 ± 0.003 a	3.49 ± 0.12 bc	0.029 ± 0.001 a	0.49 ± 0.02 b	0.58 ± 0.01 abc	7.6 ± 0.34 e	11.8 ± 0.33 d	145.9 ± 14.19 a
	16°C, 8:16 h L:D	2	0.29 ± 0.01 c	0.07 ± 0.003 b	4.69 ± 0.29 a	0.019 ± 0.001 c	0.43 ± 0.03 bc	0.58 ± 0.01 abc	8.5 ± 0.31 de	11.8 ± 0.36 d	146.6 ± 12.38 a
	8°C, 16:8 h L:D	2	0.31 ± 0.01 bc	0.09 ± 0.003 a	3.46 ± 0.13 bc	0.024 ± 0.002 abc	0.36 ± 0.03 bcd	0.54 ± 0.01 c	14.0 ± 0.39 b	20.7 ± 0.37 b	60.85 ± 12.88 bcd
	8°C, 8:16 h L:D	2	0.25 ± 0.01 d	0.10 ± 0.002 a	3.12 ± 0.10 c	0.018 ± 0.001 c	0.22 ± 0.02 e	0.49 ± 0.04 d	0.00 f	0.00 e	16.30 ± 8.6 d
	HSD		0.059	0.021	0.838	0.007		0.142	0.074	1.74	1.81	47.125
	Note: Means sharing similar letters are not statistically significant at p=0.05 among and
	within generations.

This article is protected by copyright. All rights reserved.

Acknowledgments

The authors are highly indebted to the Higher Education Commission of Pakistan for providing financial support to Muhammad Mubashir Saeed throughout the study period and the Biodiversity Research Center, Université catholique de Louvain, Belgium for providing the necessary facilities to carry out the research. KT was supported by the F.R.S.-FNRS. The


Baihan Lin

Xinxin Zhang
email: zhangx43@uw.edu

VoiceID on the fly: A Speaker Recognition System that Learns from Scratch

We proposed a novel AI framework to conduct real-time multi-speaker recognition without any prior registration or pretraining by learning the speaker identification on the fly. We considered the practical problem of online learning with episodically revealed rewards and introduced a solution based on semi-supervised and self-supervised learning methods in a webbased application at https://www.baihan.nyc/viz/VoiceID/.

Introduction

Speaker recognition involves two essential steps: registration and identification. In laboratory setting, the state-of-the-art approaches usually emphasize the registration step with deep networks trained on large-scale speaker profile dataset [START_REF] Nagrani | Voxceleb: a largescale speaker identification dataset[END_REF]. However, in real life, requiring all users to complete voiceprint registration before a multi-speaker teleconference is hardly a preferable way of system deployment. Dealing with this challenge, speaker diarization is the task to partition an audio stream into homogeneous segments according to the speaker identity. Similarly, a preferable AI engine for a realistic speaker recognition system should (1) not require user registrations, (2) allow new user to be registered into the system real-time, (3) transfer voiceprint information from old users to new ones, (4) be up running without pretraining on large amount of data in advance. While attractive, assumption (4) introduced an additional caveat that the labeling of the user profiles happens purely on the fly, trading off models pre-trained on big data with the user directly interacting with the system by correcting the agent as labels. To tackle all these challenges, we formulated this problem into an interactive online learning problem with coldstart arms and episodically revealed rewards (the user can either reveal no feedback, approving the agent by not intervening, or correcting the agent). We built upon LinUCB [START_REF] Li | A contextualbandit approach to personalized news article recommendation[END_REF] and proposed a semi-supervised learning variant to account for the fact that the rewards are entirely missing in many episodes. For each episodes without feedbacks, we applied a self-supervision process to assign a pseudo-action upon which the reward mapping is updated. Finally, we generated new arms by transferring the learned arm parameters for similar profiles given the user feedbacks. Unlike others, our VoiceID system is interactive, register-free, real-time and completely web-based (Figure 1).

Problem Setting

Algorithm 1 presents at a high-level our problem setting, where c(t) ∈ R d is a vector describing the context at time t, ra(t) ∈ [0, 1] is the reward of action a at time t, and r(t) ∈ [0, 1] K denotes a vector of rewards for all arms at time t. Pc,r denotes a joint probability distribution over (c, r), and π : C → A denotes a policy. Unlike traditional setting, in step 5 we have the rewards revealed in an episodic fashion (i.e. sometimes there are feedbacks of rewards being 0 or 1, sometimes there are no feedbacks of any kind). We consider our setting an online semisupervised learning problem [START_REF] Yver | Online semi-supervised learning: Application to dynamic learning from radar data[END_REF][START_REF] Ororbia | Online semisupervised learning with deep hybrid boltzmann machines and denoising autoencoders[END_REF], where the agent learns from both labeled and unlabeled data in online setting from scratch.

Algorithm 1 Online Learning with Episodic Rewards Player chooses an action i = πt(c(t)) Feedbacks ra(t) for all arms are episodically revealed 6:

1: for t = 1,2,3,• • • , T do 2: (c(t), r(t)) is drawn according to Pc,
Player updates its policy πt 7: end for 3. Proposed Method

Episodically Rewarded LinUCB

We proposed Background Episodically Rewarded LinUCB (BerlinUCB) [START_REF] Lin | Online semi-supervised learning in contextual bandits with episodic reward[END_REF], a semi-supervised and self-supervised online contextual bandit which updates the context representations and reward mapping separately given the state of the feedbacks being present or missing (Algorithm 2). We assume that (1) when there are feedbacks available, the feedbacks are genuine, assigned by the oracle, and (2) when the feedbacks are missing (not revealed by the background), it is either due to the fact that the action is preferred (no intervention required by the oracle, i.e. with an implied default rewards), or that the oracle didn't have a chance to respond or intervene (i.e. with unknown rewards). Especially in the Step 15, when there is no feedbacks, we assign the context xt to a class a (an action arm) with the self-supervision given the previous labelled context history. Since we don't have the actual label for this context, we only update the reward mapping parameter b a and leave the covariance matrix A a untouched. This additional usage of unlabelled data (or unrevealed feedback) is especially important in our model.

The Self-Supervision and Semi-Supervision Modules

We construct our self-supervision modules given the cluster assumption of the semi-supervision problem: the points within the same cluster are more likely to share a label. We chose three popular clustering algorithms in modern speaker diarization task as the self-supervision modules: Gaussian Mixture Models (GMM), Kmeans and K-nearest neighbors (KNN). for all a ∈ At do 5:

θa ← A -1 a ba 6: pt,a ← θ a xt + ct x t A -1 a xt 7:
end for 8:

Choose arm at = arg max a∈A t pt,a 9:

if the background revealed the feedbacks then 10:

Observe feedback ra t ,t 11: end if 21: end for

Aa t ← Aa t +

The AI Engine for Online Speaker Diarization

To adapt our BerlinUCB algorithm to the specific application of speaker recognition, we first define our actions. There are three major classes of actions: an arm "New" to denote that a new speaker is detected, an arm "No Speaker" to denote that no one is speaking, and N different arms "User n" to denote that user n is speaking. Table 1 presents the reward assignment given four types of feedbacks. Note that we assume that when the agent correctly identifies the speaker (or no speaker), the user (as the feedback dispenser) should send no feedbacks to the system by doing nothing. In another word, in an ideal scenario when the agent does a perfect job by correctly identifying the speaker all the time, we are not necessary to be around to correct it anymore (i.e. truly feedback free). As we pointed out earlier, this could be a challenge earlier on, because other than implicitly approving the agent's choice, receiving no feedbacks could also mean the feedbacks are not revealed properly (e.g. the human oracle took a break). Furthermore, we note that when "No Speaker" and "User n" arms are correctly identified, there is no feedback from us the human oracle (meaning that these arms would never have learned from a single positive reward if we don't use the "None" feedback iterations at all!). The semi-supervision by self-supervision step is exactly tailored for a scenario like this, where the lack of revealed positive reward for "No Speaker" and "User n" arms is compensated by the additional training of the reward mapping ba t if context xt is assigned to the right arm. To tackle the cold start problem, the agent grows it arms in the following fashion: the agent starts with two arms, "No Speaker" and "New"; if it is actually a new speaker speaking, we have the following three conditions: (1) if "New" is chosen, the user approves this arm by giving it a positive reward (i.e. clicking on it) and the agent initializes a new arm called "User N " and update N = N + 1 (where N is the number of registered speakers at the moment); (2) if "No Speaker" is chosen, the user disapproves this arm by giving it a zero reward and clicking on the "New" instead), while the agent initializes a new arm; (3) if one of the user arms is chosen (e.g. "User 5" is chosen while in fact a new person is speaking), the agent copies the wrong user arm's parameters to initialize the new arm, since the voiceprint of the mistaken one might be beneficial for the new profile. 1: Routes given either no feedbacks, or a feedback telling the agent that the correct label is a * . (+,+) means that the agent guessed it right by choosing the right arm; (+,-) means that the agent chose this arm incorrectly, since the correct one is another arm; (-,+) means that the agent didn't choose this arm, while it turned out to be the correct one. "-" means NA.

Interactive System: VoiceID on the fly

As shown in Figure 1, our system "VoiceID on the fly" is an interactive continual learning web-based system. We computed the Mel Frequency Cepstral Coefficients (MFCC) in a sliding window fashion given the real-time audio input from microphone, with the MFCC bands color coded in the page. Figure 2 evaluated the system's performance in the MiniVox benchmark [START_REF] Lin | Speaker diarization as a fully online learning problem in minivox[END_REF] where the labels are only revealed 1% of the time (p=0.01). We observe that although not pretrained on any dataset, our algorithms effectively learns "who speaks when", better than the baselines (GMM, Kmeans and KNN). This suggests a smooth deployment and a good user experience in the demonstration.

At the start, there are only two buttons available: "No Speaker" and "New Speaker". The agent chooses an arm by setting it be highlighted. If it is correct, we do not have to change it (unless it's "New Speaker", where we need to click on it to confirm creating a new arm). The feature band θa of each arm is also color coded real-time to visualize how the agent learns across trials. If it is incorrect, we click on the right arm to give the system a feedback. This demonstration provides an intriguing example of how an AI agent can learn to recognize speaker identity (1) entirely escaping the necessity of registering user voiceprint beforehands, (2) effortlessly incorporating new users under an optimal exploration-exploitation trade-off, (3) effectively transferring representation of registered user features to new users, and (4) continually learning despite minimal involvement of human corrections (i.e. sparse and episodic feedbacks).

Figure 1 :

 1 Figure 1: Flowchart of the demo system: VoiceID on the fly.

  r

Algorithm 2 1 :

 21 BerlinUCB Initialize ct ∈ R+, Aa ← I d , ba ← 0 d×1 ∀a ∈ At 2: for t = 1,2,3,• • • , T do 3: Observe features xt ∈ R d 4:

Feedback

Figure 2 :

 2 Figure 2: Rewards in MiniVox benchmark. (a) C5-MFCC, p=0.01; (b) C10-MFCC, p=0.01; (c) C20-MFCC, p=0.01.

  Aa t ← Aa t + xtx t

	14:	if use self-supervision feedback
	15:
	17:	elif	% ignore self-supervision signals
	18:
	19:	end if
	20:

xtx t 12: ba t ← ba t + ra t ,txt 13: elif the background revealed NO feedbacks then r = [at == predict(xt)] % clustering modules 16: ba t ← ba t + r xt