Y Audrey Hay

Nicolas Deperrois

Tanja Fuchsberger

Thomas Matthew Quarrell

Anna-Lucia Koerling

Ole Paulsen

Thalamus mediates neocortical Down state transition via GABA B receptor-targeting interneurons Running title: Thalamic control of Up state termination

Slow-wave sleep is characterized by near-synchronous alternation of active Up states and quiescent Down states in the neocortex. Whilst the cortex itself can maintain these oscillations, the full expression of Up-Down states requires intact thalamocortical circuits. Sensory thalamic input can drive the cortex into an Up state. Here we show that midline thalamic neurons terminate Up states synchronously across cortical areas. Combining local field potential, single unit, and patch-clamp recordings in conjunction with optogenetic stimulation and silencing in mice in vivo, we report that thalamic input mediates Down transition via activation of layer 1 neurogliaform inhibitory neurons acting on GABAB receptors.

These results strengthen the evidence that thalamocortical interactions are essential for the full expression of slow-wave sleep, show that Down transition is an active process mediated by cortical GABAB receptors, and demonstrate that thalamus synchronizes Down transitions across cortical areas during natural slow-wave sleep.

INTRODUCTION

Brain activity during slow-wave sleep (SWS) participates in memory consolidation by synchronizing neuronal activity across brain regions [START_REF] Klinzing | Mechanisms of systems memory consolidation during sleep[END_REF]. During SWS, cortical network activity alternates between quiescent 'Down' states and active 'Up' states (Steriade et al., 1993a). Up states are initiated locally in the cortex [START_REF] Sanchez-Vives | Cellular and network mechanisms of rhythmic recurrent activity in neocortex[END_REF] or by thalamocortical inputs (Steriade et al., 1993b;[START_REF] Poulet | Thalamic control of cortical states[END_REF][START_REF] David | Essential thalamic contribution to slow waves of natural sleep[END_REF] and propagate wave-like over the cortical surface [START_REF] Massimini | The sleep slow oscillation as a traveling wave[END_REF]. Up state termination is often assumed to be passive, due to build-up of K + conductances, synaptic depression, and local inhibitory cells, including somatostatin interneurons (SST INs;[START_REF] Sanchez-Vives | Cellular and network mechanisms of rhythmic recurrent activity in neocortex[END_REF][START_REF] Shu | Barrages of synaptic activity control the gain and sensitivity of cortical neurons[END_REF][START_REF] Funk | Role of somatostatin-positive cortical interneurons in the generation of sleep slow waves[END_REF][START_REF] Niethard | Cortical circuit activity underlying sleep slow oscillations and spindles[END_REF][START_REF] De Filippo | Somatostatin interneurons activated by 5-HT2A receptor suppress slow oscillations in medial entorhinal cortex[END_REF]; for a recent review, see Tukker et al., 2020). However, Down transition is strikingly synchronous across cortical areas [START_REF] Massimini | The sleep slow oscillation as a traveling wave[END_REF][START_REF] Volgushev | Precise long-range synchronization of activity and silence in neocortical neurons during slow-wave sleep[END_REF], suggesting an active, coordinating mechanism. Interconnecting all cortical areas the thalamus has a privileged position for such a coordinating role. Thalamic nuclei separate into sensory nuclei that relay sensory inputs to layer (L)4 of primary sensory cortices and non-specific nuclei that primarily target L1 of both sensory and association cortical areas [START_REF] Halassa | Thalamocortical circuit motifs: a general framework[END_REF].

Here we demonstrate a contribution of midline non-specific thalamic neurons to synchronizing Down transition across cortical areas. We show that (1) midline thalamic neurons fire in bursts towards the end of cortical Up states, (2) activating midline thalamic neurons terminates Up states, and (3) silencing these neurons prolongs Up states. These thalamic neurons target L1 neurogliaform cells (NGFCs), whose optogenetic stimulation and silencing shortens and prolongs Up states, respectively. This mechanism can reset all cortical areas prior to a next Up state wave. !

RESULTS

Necab1 thalamic neurons control cortical Down transition

We implanted tetrodes in midline thalamic nuclei (MTN) for single unit recording and dual wire probes in prefrontal (PFC), primary motor (M1), and primary somatosensory (S1) cortices to simultaneously record local field potentials (LFPs) during natural sleep (STAR Methods; n = 34 neurons from 3 mice; Figures 1A andS1A). All neurons emitted spikes primarily during Up states. Thalamic neurons show two firing modes, tonic and burst firing, with different effects on cortical targets (Swadlow and Gusev, 2001). We observed a peak of firing at Up state initiation comprising both tonic and burst activity and a second peak at the end of Up states primarily due to burst activity (n = 1463 spikes; Rayleigh distribution test, p = 0.01; Figure 1Aii, iii), the latter suggesting causal involvement of thalamic neurons in Up state termination. To target MTN optogenetically, we used Necab1-Cre mice, whose Cre-expressing thalamic neurons are restricted to non-specific thalamic nuclei, including MTN [START_REF] Phillips | A repeated molecular architecture across thalamic pathways[END_REF]Figure 1B). Injection of a Cre-dependent adeno-associated virus (AAV) yielded transduction in 94.5 ± 3.5% of thalamic Necab1 neurons (n = 448 neurons from 3 mice; Figure S1B), which projected primarily to cortical L1 (168 ± 41% of averaged signal intensity; Figure 1B). We transduced Necab1 thalamic neurons with the neuronal silencer archaerhodopsin-T (ArchT) and silenced MTN using green light pulses (530 nm, 15 mW, 300 ms) delivered every 10 s through an implanted optic fiber. During natural sleep, light pulses prolonged Up states in Necab1-ArchT mice (ArchT-Ctrl: 925 ± 27 ms;

ArchT-Light: 1095 ± 28 ms; n = 6 mice; paired one-way Mann-Whitney (M-W) test, p = 0.03) but not in control Necab1-GFP mice (GFP-Ctrl: 907 ± 29 ms; GFP-Light: 927 ± 27 ms; n = 6; paired M-W test, p = 1; Figure 1C andS1D). The increased Up state duration in ArchT-expressing mice was due to a delay of Down transition following light delivery (GFP-Light: 436 ± 20 ms; ArchT-Light: 612 ± 20 ms; Kolmogorov-Smirnov (K-S) test, p = 1.2 . 10 -11 ; Figure 1C andS1D). Conversely, following transduction of the neuronal activator channelrhodopsin-2 (ChR2), a burst of 3 blue light pulses (473 nm, ≤15 mW, 5 ms, 100 Hz) led to a reduction of Up state duration (ChR2-Ctrl: 909 ± 23 ms; ChR2-Stim: 690 ± 18 ms; n = 5; one-way paired M-W test, p = 0.03), an effect not observed in Necab1-GFP mice (GFP-Ctrl: 926 ± 30 ms; GFP-Stim: 917 ± 26 ms; n = 6; paired M-W test, p = 1; Fig- ure 1D andS1E). The reduced Up state duration was due to an advancement of Down transition (ChR2-Ctrl: 491 ± 19 ms; ChR2-Stim: 231 ± 11 ms; K-S test, p < 2.2 . 10 -16 ; Figure 1D andS1E). Thalamic neurons project to L1 and electrical stimulation in L1 induces Down transition via GABABRs [START_REF] Mann | Distinct roles of GABAA and GABAB receptors in balancing and terminating persistent cortical activity[END_REF]. Intraperitoneal injection of the GABABR antagonist CGP55,845 (5 mg/kg) reduced the probability of light-induced Down transition (ART ANOVA: GFP vs ChR2 vs ChR2-CGP, p < 7.11 . 10 -8 ; Figure 1Div-v) and prevented the reduction of Up state duration (ChR2-CGP-Ctrl: 974 ± 27 ms; ChR2-CGP-Stim: 928 ± 24 ms; n = 5; paired M-W test, p = 0.63; Figure 1Diii), showing that thalamic stimulation induces Down transition via GABABRs.

Neocortical GABAB receptors mediate the thalamus-induced Down transition

To identify the location of GABABRs, we recorded from urethane anesthetized mice, which show SWS-like activity and allow local delivery of light and receptor blockers (Figure S2A and S2B). As in natural sleep, midline thalamic bursts were skewed towards the second half of Up states (465 out of 825 spikes; z proportion test, p = 2.6 . 10 -4 ; Figure 2A). Intraperitoneal injection of CGP55,845 or its local application (1 µM) at the cortical surface prolonged Up states by 29.5 ± 5% and 24.5 ± 8%, respectively (Figure 2Aiv), and local application delayed Up state termination relative to the peak of thalamic burst activity by 111 ms pooled over all mice (n = 8; Figure 2Aiii and S2Civ). In Necab1-ArchT mice, cortical silencing of thalamic fibers (500 ms) during Up states increased Up state duration (Control, 0.89 ± 0.07 s; ArchT, 1.06 ± 0.10 s; n = 8; paired M-W test, p = 0.02; Figure 2B), an effect occluded by local CGP55,845 (Control, 1.12 ± 0.16 s; ArchT, 1.14 ± 0.18 s; n = 5; paired M-W test, p = 0.63; Figure 2B). In Necab1-ChR2 mice, a burst of light pulses in the thalamus or at the cortical surface induced Down transition within 300 ms in 83 ± 3.5% (n = 6) and 79 ± 7.3% (n = 6), respectively, compared to 41 ± 5.5% and 42 ± 7.9% following sham light delivery (paired M-W test, both p = 0.03) with no difference between thalamic and cortical stimulation (M-W test, p = 1; Figure 2C

Layer 1 neurogliaform cells (NGFCs) mediate thalamus-induced Down transition

Necab1 thalamic neurons project to L1 where they target mainly neurogliaform cells (NGFCs) [START_REF] Anastasiades | Mediodorsal and ventromedial thalamus engage distinct L1 circuits in the prefrontal cortex[END_REF] and evoke a disynaptic inhibitory postsynaptic current (IPSC) with a GABABR component in pyramidal cells (PCs; Figure S3A). To target NGFCs optogenetically, we used a neuron-derived neurotrophic factor (NDNF)-Cre mouse line (Figure 3A and S3B;[START_REF] Abs | Learning-related plasticity in dendrite-targeting layer 1 interneurons[END_REF]

Thalamic neurons synchronize Down transition across cortical areas

Given the widespread distribution of their axons, midline thalamic neurons could play a role in synchronizing slow waves across the cortex. We implanted Necab1-Cre mice with 3-4 dual wire electrodes in PFC, M1, S1 and secondary visual (V2) cortices and an optic fiber above midline thalamus (Figure 4A). In Necab1-ChR2 mice, light bursts induced Down transition within 300 ms in ≥1, ≥2, or ≥3 cortical area with a probability of 91 ± 5%, 76 ± 6%, and 52 ± 10%, respectively (Figure 4A). Intraperitoneal CGP55,845 reduced transition probability in ≥2 and "3 cortices to 55 ± 5% and 30 ± 4%, respectively (ART ANOVA Base vs CGP, p = 0.0013; n = 4; Figure 4A). Thus, activation of midline thalamic neurons triggers synchronized Down transition across cortical areas through GABABRs.

Silencing Necab1 thalamic neurons (30 s long pulse; Figure 4B) reduced slow oscillation (<2 Hz) power during SWS in Necab1-ArchT but not Necab1-GFP mice (Figure S4A). The intra-individual dispersion (standard deviation, SD) of the delay of Up transition in M1, S1 and V2 compared to the transition in PFC did not change significantly upon silencing (Ctrl: M1 = 37 ± 3 ms, S1 = 50 ± 6 ms, V2 = 52 ± 3 ms; ArchT: M1 = 45 ± 6 ms, S1 = 52 ± 8 ms, V2 = 61 ± 4 ms; ART ANOVA Ctrl vs ArchT, p = 0.13; Figure 4Biii). However, thalamus silencing increased the dispersion of Down transition (SD of delay, Ctrl: M1 = 30 ± 3 ms, S1 = 38 ± 2 ms, V2 = 41 ± 3 ms; ArchT: M1 = 44 ± 5 ms, S1 = 50 ± 6 ms, V2 = 49 ± 3 ms; ART ANOVA Ctrl vs ArchT, p = 6.4 . 10 -4 ; Figure 4Biv), an effect not observed in Necab1-GFP mice (Ctrl: M1 = 39 ± 5 ms, S1 = 45 ± 5 ms, V2 = 53 ± 8 ms; ArchT: M1 = 34 ± 4 ms, S1 = 44 ± 3 ms, V2 = 53 ± 8 ms; n = 5; ART ANOVA Ctrl vs ArchT, p = 0.64; Figure 4B).

Finally, we tested GABABR involvement in SWS synchronization (Figure 4C). On average, Up state initiation was observed first in PFC with a delay to M1, S1 and finally V2 propagating as a traveling wave in an antero-posterior direction (Delay relative to PFC: M1 = 13 ± 5 ms, S1 = 18 ± 5 ms, V2 = 34 ± 4 ms; n = 8; Figure 4Ci). CGP55,845 reduced Up state initiation delay and directionality (Delay relative to PFC transition: M1 = 3 ± 6 ms, S1 = 6 ± 8 ms, V2 = 3 ± 5 ms; ART ANOVA Base vs CGP, p = 0.04). The intra-individual dispersion (SD) of the delay of transition in M1, S1 and V2 relative to PFC transition increased in the presence of CGP55,845 (Ctrl: M1 = 36 ± 2 ms, S1 = 42 ± 3 ms, V2 = 44 ± 4 ms; CGP: M1 = 50 ± 7 ms, S1 = 54 ± 6 ms, V2 = 61 ± 6 ms; ART ANOVA Base vs CGP, p = 0.006; Fig- ure 4Cii), suggesting that blocking GABABRs affects Up state wave propagation. Down transition was more synchronous than Up state initiation across recording sites (Delay relative to PFC transition: M1 = 6 ± 13 ms, S1 = 4 ± 10 ms, V2 = 7 ± 13 ms; Figure 4Ciii) and the dispersion was low (SD: M1 = 33 ± 3 ms, S1 = 35 ± 3 ms, V2 = 42 ± 4 ms; ART ANO-VA Base vs CGP, p = 0.006; Figure 4Civ). Blocking GABABRs had little effect on the averaged delay (M1 = 11 ± 9 ms, S1 = 15 ± 14 ms, V2 = 8 ± 14 ms; ART ANOVA Base vs CGP, p = 0.11), but greatly increased the dispersion (SD: M1 = 55 ± 7 ms, S1 = 59 ± 8 ms, V2 = 62 ± 7 ms; ART ANOVA Base vs CGP, p = 6.3 . 10 -5 ). Thus, Down transition is highly syn-chronous and blocking GABABRs reduces this synchrony, consistent with the synchronization by MTN via GABABRs.

DISCUSSION

Our study shows that burst activity in midline thalamic neurons induces cortical Down transition in naturally sleeping mice. These projection neurons fire in bursts towards the end of Up states; their optogenetic stimulation terminates Up states, and their silencing prolongs Up states in both anesthetized and naturally sleeping mice. The thalamus-induced Down transition is mediated via L1 NGFCs acting on cortical GABABRs and it synchronizes slow waves across cortical areas during natural sleep.

Non-specific thalamic nuclei control cortical Down transition

The involvement of the thalamocortical loop in SWS oscillations is well established (Steriade 1993a,b;Crunelli et al., 2015) but most emphasis was placed on sensory thalamic nuclei, which can trigger Up transition [START_REF] Poulet | Thalamic control of cortical states[END_REF]. Preferential firing of thalamic neurons was reported at Up state initiation [START_REF] Slézia | Phase advancement and nucleus-specific timing of thalamocortical activity during slow cortical oscillation[END_REF][START_REF] Varela | mPFC spindle cycles organize sparse thalamic activation and recently active CA1 cells during non-REM sleep[END_REF] and just before their termination [START_REF] Sheroziya | Global intracellular slow-wave dynamics of the thalamocortical system[END_REF]. We found burst firing in midline neurons at the end of Up states in both urethane-anesthetized and naturally-sleeping mice, with firing activity also at Up state initiation (Figure 1 and2). It was reported that optogenetic stimulation of midline thalamic neurons by strong illumination induces cortical Up states [START_REF] Gent | Thalamic dual control of sleep and wakefulness[END_REF], which might be expected from an excitatory input. However, our results show that, despite being glutamatergic, the Necab1 midline thalamic neurons when activated during Up states have a mainly inhibitory effect and during natural SWS these neurons fire bursts at a time consistent with a role in terminating Up states.

Non-specific thalamic neurons target L1 NGFCs

To target midline thalamic neurons optogenetically, we used the Necab1-Cre mouse line.

Necab1 is expressed in non-specific thalamic nuclei but not in sensory nuclei [START_REF] Phillips | A repeated molecular architecture across thalamic pathways[END_REF]. Due to diffusion of viral particles, we could not restrict transduction to midline neurons. Although midline nuclei were targeted by viral injections from the left and optic fiber from the right (Figure 1 and 2), local stimulation of thalamic fibers in the cortex could recruit fibers from other non-specific thalamic nuclei which also target L1 INs (Figure 2; [START_REF] Rubio-Garrido | Thalamic input to distal apical dendrites in neocortical layer 1 is massive and highly convergent[END_REF]Clascà et al., 2012;Cruickshank et al., 2012;[START_REF] Collins | Reciprocal circuits linking the prefrontal cortex with dorsal and ventral thalamic nuclei[END_REF][START_REF] Hay | Target interneuron preference in thalamocortical pathways determines the temporal structure of cortical responses[END_REF]. Necab1 fibers targeted both NGFCs and MPCs (Figure 3), consistent with results by [START_REF] Anastasiades | Mediodorsal and ventromedial thalamus engage distinct L1 circuits in the prefrontal cortex[END_REF].

The role of GABAergic interneurons in cortical Down transition

Most cortical INs are preferentially active during Up states (Steriade et al., 2001;[START_REF] Gentet | Membrane potential dynamics of GABAergic neurons in the barrel cortex of behaving mice[END_REF][START_REF] Zucca | An inhibitory gate for state transition in cortex[END_REF], but a population of Down state-active deep-layer NGFCs was recently reported [START_REF] Valero | Sleep down state-active ID2/Nkx2.1 interneurons in the neocortex[END_REF]. We identified L1 NGFCs as cortical relay for thalamus-induced Down transition, acting through GABABRs (Figure 3; [START_REF] Oláh | Regulation of cortical microcircuits by unitary GABA-mediated volume transmission[END_REF].

L1-projecting SST INs also activate GABABRs [START_REF] Urban-Ciecko | Neocortical somatostatin neurons reversibly silence excitatory transmission via GABAb receptors[END_REF] and SST INs are targeted by MTN [START_REF] Hay | Target interneuron preference in thalamocortical pathways determines the temporal structure of cortical responses[END_REF]. Thus it is possible that SST INs also contribute to Down transition in naturally sleeping mice [START_REF] Funk | Role of somatostatin-positive cortical interneurons in the generation of sleep slow waves[END_REF][START_REF] Niethard | Cortical circuit activity underlying sleep slow oscillations and spindles[END_REF] through thalamic activation, which we might have missed because SST IN firing is suppressed by urethane (Urban-Ciecko & Barth, 2016). In contrast, stimulating PV interneurons failed to induce Down transition despite producing a strong GABAAR-mediated current (Figure S3), consistent with [START_REF] Funk | Role of somatostatin-positive cortical interneurons in the generation of sleep slow waves[END_REF], but in contrast to [START_REF] Zucca | An inhibitory gate for state transition in cortex[END_REF].

Synchronization of Down transition by thalamic input

Many factors contribute to terminating Up states in cortical slice preparations, including mechanisms intrinsic to L5 neurons, synaptic depression, and GABAergic inhibition [START_REF] Sanchez-Vives | Cellular and network mechanisms of rhythmic recurrent activity in neocortex[END_REF][START_REF] Mann | Distinct roles of GABAA and GABAB receptors in balancing and terminating persistent cortical activity[END_REF][START_REF] Perez-Zabalza | Modulation of cortical slow oscillatory rhythm by GABAB receptors: an in vitro experimental and computational study[END_REF]. Our findings that midline thalamic neurons take part in controlling Down transition and that blocking GABABRs prolongs Up states in naturally sleeping mice (Figure S1) indicate an active mechanism governing Up state termination, which may help explain the remarkable synchrony of Down transition across cortical areas [START_REF] Volgushev | Precise long-range synchronization of activity and silence in neocortical neurons during slow-wave sleep[END_REF][START_REF] Sheroziya | Global intracellular slow-wave dynamics of the thalamocortical system[END_REF]. Thalamus has an ideal position to promote synchronization, but the claustrum has also been implicated in synchronizing cortical slow waves [START_REF] Narikiyo | The claustrum coordinates cortical slow-wave activity[END_REF]. Claustrum and MTN project widely throughout the cortex [START_REF] Vertes | Efferent projections of reuniens and rhomboid nuclei of the thalamus in the rat[END_REF] and individual thalamic neurons can send collaterals to several cortical areas [START_REF] Clascá | Unveiling the diversity of thalamocortical neuron subtypes[END_REF]. Because Down state is a prominent feature of SWS, it is likely that several subcortical structures participate in their synchronization, including other non-specific thalamic nuclei that also project to L1 [START_REF] Cruikshank | Thalamic control of layer 1 circuits in prefrontal cortex[END_REF][START_REF] Anastasiades | Mediodorsal and ventromedial thalamus engage distinct L1 circuits in the prefrontal cortex[END_REF]. Likewise, it is likely that besides GABABR-targeting L1 INs, other cortical INs are also involved.

An active mechanism of globally synchronized Down transition is conceptually important, as it enables the anterior areas with the highest intrinsic excitability to initiate new Up states that then propagate as traveling waves [START_REF] Massimini | The sleep slow oscillation as a traveling wave[END_REF]. This antero-posterior wave propagation was disrupted by a GABABR antagonist (Figure 4), and it will be of interest to investigate the behavioral consequences. Synchronization of slow oscillations allows coordinated activation of brain areas, a feature thought to be important for memory consolidation [START_REF] Huber | Local sleep and learning[END_REF][START_REF] Major | Persistent neural activity: prevalence and mechanisms[END_REF][START_REF] Compte | Computational and in vitro studies of persistent activity: edging towards cellular and synaptic mechanisms of working memory[END_REF]. Accordingly, cortical spindles and hippocampal sharp wave ripples are synchronized with Up and Down states (Sirota et al., 2003;Molle et al., 2011). We suggest that MTN play an important role in synchronizing Down transition across cortical areas, enabling faithful initiation and propagation of Up states during SWS.! (Aii) Rose plots of all spikes, bursts and tonic and inter-spike interval plots for two units.

STAR METHODS

RESOURCE AVAILABILITY

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Audrey Hay (ah831@cam.ac.uk).

Materials availability

This study did not generate new unique reagents. # chased from Harlan (Bicester, UK) and imported to the animal facility as litters with their mother at P6 or directly bred in the facility; transgenic lines were bred in the facility. #021188) lines were purchased from The Jackson Laboratory. PV-Cre, Ai32 and Ai40D lines were kept homozygous while NDNF-Cre, Necab1-Cre, GAD67-Cre and Sert-Cre lines were kept heterozygous. For ex vivo and anesthetized recordings, both males and females were used and results from both sexes were pooled. We did not observe any influence or association of sex on the experimental outcomes. For naturally sleeping recordings, although we do not expect sex differences in the basic mechanisms of sleep, we aimed to reduce variance and the number of animals required for statistical analyses by using only male mice for recordings during natural sleep.

METHOD DETAILS

Neonatal viral injection surgeries

Viral injections were performed on pups between postnatal day 1 (P1) and P2. The whole litter was removed from the parent cage and placed within a heated recovery chamber (32-34 °C) in a clean cage containing part of the nest. Pups were then taken one by one to the surgery room where they were deeply anesthetized by means of hypothermia. Upon loss of movement, pups were transferred onto an adapted mould that held them in a fixed posi-tion without compressing them. The mould was placed on ice to maintain the hypothermia.

Injections were performed using a pulled capillary fixed on a Hamilton syringe and filled with mineral oil (Sigma, Saint Louis, USA). 0.3 µL of virus (AAV5-EF1a-dio-ChR2(H134R)-EYFP-WPRE; titers ranging 1.2-13 . 10 12 vg/mL; UNC vector Core; USA) was loaded by the tip of the capillary, which was then placed half way between bregma and lambda and ML = 1.2 mm. The capillary was quickly lowered through the skin and skull to the brain then slowly approached to the target area (DV = 1.8 mm for VB). Two minutes after viral injection, the capillary was slowly pulled out, the pup was removed from the mould and warmed up manually until recovery of regular breathing and pink complexion. They were then placed back inside the recovery chamber with their littermates. Ten minutes after the last pup had been placed back in the recovery chamber, the whole litter was transferred back to the parents.

Viral injections and implant surgeries in adult mice

All surgical procedures were performed according to Home Office minimal standard for aseptic surgery. Mice were given Meloxicam analgesics (2 mg/kg in saline) 30 minutes before induction of anesthesia using 5% isoflurane. Upon placement on the stereotaxic frame, isoflurane level was lowered to 1.8-2.2%, and the surgical field was aseptically prepared. The skull was exposed, bregma-lambda aligned and holes were drilled above sites of injection or implant. Injections in the non-specific midline thalamus of Necab1-Cre mice was done at coordinates AP: -1.58, ML: +1.1 and DV: 3.4-3.65 at an angle of 15-20º. Viruses, 0.3-0.5 µL of AAV5/9-EF1a-dio-ChR2(H134R)-EYFP-WPRE, AAV5/9-EF1a-dio-EGFP-WPRE or AAV5-EF1a-dio-eArch3.0-EYFP-WPRE (titers ranging 1.2-13 . 10 12 vg/mL; UNC vector Core; USA) were delivered through a metal cannula fixed to a 5 µL Hamilton syringe. Seven wild type and seventeen Necab1-Cre mice were implanted in the prefrontal (PFC; AP: +1.96 and ML: -0.4), primary motor (M1; AP: +0.5 and ML: -1.5), primary soma-tosensory (S1; AP: -1.3 and ML: -2.8) and secondary visual (V2; AP: -3 and ML: -2) cortices with staggered wire electrodes ('duodes'), consisting of two twisted 75 µm tefloncoated silver electrodes (AGT0510, World Precision Instruments, Hitchin, UK) with tips spaced 400-600 µm, the upper tip being implanted in layer 1 and the lower tip reaching infragranular layers. An optical fiber (300 µm diameter, 0.29 NA, Thorlabs) was also implanted above the MTN at coordinates: AP: -1.58, ML: 1.2, DV: 3.2-3.6 at an angle of 20°.

To minimize the risk of manipulating the activity of thalamic neurons located outside the MTN, viral particles were injected in the midline thalamus through the left hemisphere while the optic fiber was implanted above the midline through the right hemisphere. Three additional mice were implanted with both duodes and tetrodes. Four or five tetrodes made of four strands of 17 µm twisted tungsten wire (HM-L coated 90% platinum/10% iridium 17 µm diameter wire) mounted on a micro-drive (Cambridge NeuroTech, Cambridge, UK) were implanted in the MTN at an angle of 10º at coordinates: AP: -1.58, ML: 0.9-1.1 and DV: 3.4 lowered to 3.8 by 50 µm step after each recording day. Ground and reference silver wires were connected to a stainless microscrew implanted over the cerebellum: AP:

-5.3 ML: +/-1.5. Lastly, a 75 µm Teflon-coated silver electrode was implanted in the neck muscles to record electromyographic activity. All wires were connected to a 32 pins Omnetics connector (Genalog, Cranbrook, UK). The exposed brain was covered with a protective dura gel (Cambridge NeuroTech, UK) to prevent damage upon cementing of the electrodes. Duodes were individually glued to the skull using UV-cured glue (Loctite 303389; Rapid Electronics, UK) and the implant was secured to the skull using dental cement (Super-Bond C & B; Prestige Dental, Bradford, UK). Mice were allowed to recover in a heated recovery chamber before returning to their home cage. Mice were given Meloxicam (2 mg/kg) and were weighed daily for 5 days following the surgery to ensure a good recovery.

Recordings in naturally-sleeping mice

After a minimum of 5 days following surgery, the mice were handled daily for 3-5 consecutive days and then habituated to the sleeping cage for another 2-7 days before recordings started. On the recording days, to reduce stress, mice were placed in the recording chamber for 2-3 hours before the recordings commenced. Then, a Whisper 32-channels preamplifier was plugged to the Omnetics connector and plugged to the Whisper amplifier (Neural Circuits, LLC, VA, USA). Data were acquired with a 16-bit National Instruments data acquisition card and recorded using SpikeGLX software at 25 kHz (https://billkarsh.github.io/SpikeGLX/). Analysis was performed offline. Recordings were taken within 3 hours after the mouse was connected.

Recordings in anesthetized mice

Mice, aged 3 weeks to 6 months, were deeply anesthetized with intraperitoneal (i.p.) injections of urethane (1.5 to 1.8 g/kg injected in several doses until loss of reflexes as assessed by paw pinching). Mice were placed in a stereotaxic apparatus (David Kopf instruments) for extracellular recordings or their skull was cemented to a home-made device for patch-clamp experiments (see detailed procedure in González-Rueda et al., 2018). Their temperature was monitored and maintained at 34-36 °C using a heating blanket (Basi, West Lafayelle, IN, USA). A small burr hole was drilled at coordinates AP: -1.5, ML: 1.5-2.5 mm for S1 and a burr or a craniotomy was performed at coordinates AP: -1.58 and ML: 1-2.8 mm for recordings from the thalamus. Cortical whole cell, silicon probe recordings, and thalamic single unit recordings were performed after craniotomy and gentle removal of the dura. For optogenetic experiments, an optic fiber (300 µm diameter, 0.29 NA, Thorlabs) was lowered above the thalamic nuclei at coordinates AP: -1.58, ML: 1.2, DV: 3.2-3.6 at an angle of 10° or was set above the surface of the barrel cortex. Light delivery was performed using a blue laser (473 nm, power adjusted to 4-8 mW for fiber stimulation and up to 15 mW for stimulation of somata during natural sleep; Ciel, Laser Quantum, Cheshire, UK) or green laser (532 nm, power 5-15 mW) or a LED-light source (473 nm, power 3-7 mW; Thorlabs, Ely, UK). In order to be physiologically relevant, blue light intensity was reduced to terminate a maximum of 90% of Up states. Light pulses were applied at regular intervals, therefore not targeting a specific phase of Up or Down state. LFP recordings were carried out using an extracellular parylene-C-insulated tungsten microelectrode (127 µm diameter, 1 MΩ; A-M Systems), single-unit recordings were carried out using a glasscoated tungsten electrode (5-10 µm long tip, 7-8 MΩ, microelectrodes.com), silicon probes recordings were done using 4 channels Q-probes (100 µm diameter recording sites spaced 200 µm apart on a single shank, with the upper electrode placed in layer 1; Neuronexus), and whole-cell recordings were performed using electrodes pulled from borosilicate glass capillaries mounted on a MultiClamp headstage (Molecular Devices) that was controlled using a digital arm (Scientifica). Silicon probes signals were recorded in a way equivalent to that described above in naturally sleeping animals. Extracellular recordings were amplified with a two-channel amplifier (Microelectrode AC Amplifier Model 1800, A-M systems) and acquired at 20 kHz using an ITC18 data acquisition board (HEKA Instruments, Lambrecht/Pfalz, Germany). Acquisition and stimulation were controlled using custom made procedures in Igor Pro (WaveMetrics, Oregon, USA). For in vivo whole-cell recordings, equipment and procedures equivalent to those described for slice whole-cell recordings were used (see specific section for details).

Slice preparation and slice electrophysiology

Postnatal day 13-22 (P13-P22) or 1-4 months old mice from both sexes were used for slice electrophysiology. P13-P22 mice were used for experiments requiring spontaneous or induced Up states. One-to-four months old mice were used for measurements of synaptic currents in mice expressing ChR2 either in GABAergic interneurons or after stereotaxic injection in MTN. Mice were deeply anesthetized with isoflurane (4%) and decapitated.

Brains were removed and immersed in chilled artificial cerebrospinal fluid (aCSF) bubbled with carbogen gas (95% O2/ 5% CO2) and containing: 126 mM NaCl, 3-3.5 mM KCl, 1.25 mM NaH2PO4, 1-2 mM MgSO4, 1.2-2 mM CaCl2, 26.4 mM NaHCO3 and 10 mM glucose (pH 7.2). Unless otherwise stated, all slice recordings throughout the manuscript were performed from neurons of the somatosensory cortex. For P13-P22 recordings, thalamocortical slices (400 µm) preserving connections between the thalamus and the barrel cortex were prepared as originally described by [START_REF] Agmon | Thalamocortical responses of mouse somatosensory (barrel) cortex in vitro[END_REF]. For slices from 1-4 months old mice, 350-400 µm coronal slices were prepared. Slices were incubated for at least 1 h at room temperature (20-25 °C) in an oxygenated interface chamber or in a conventional submerged incubation chamber. For Up state recordings, the recording chamber was perfused at 6-10 mL/min with heated (30-32 °C) aCSF saturated with carbogen gas and containing 3.5 mM KCl. For all other recordings, the recording chamber was perfused at 2-4 mL/min with aCSF saturated with carbogen gas and containing 3 mM KCl at room temperature (20-22 °C). Each slice was maintained for a maximum of 3 h in the recording chamber and all the recordings were performed within 8 h following the slice preparation.

Whole-cell patch-clamp recordings were performed with glass pipettes (3-6 MOhm; borosilicate glass; Harvard Apparatus LTD, Kent, UK) containing (in mM): 110 mM K-gluconate, 40 mM HEPES, 2 mM ATP-Mg, 0.3 mM GTP, 4 mM NaCl, 2 mg/mL biocytin (275-285 mOsm/L, pH 7.2-7.3). Whole-cell recordings were made with a MultiClamp 700B amplifier in current-clamp mode for recording Up-Down states or in voltage-clamp mode for recording excitatory and inhibitory synaptic currents.

To investigate cortical targets of thalamic inputs, monosynaptic connections were isolated as previously described [START_REF] Petreanu | The subcellular organization of neocortical excitatory connections[END_REF][START_REF] Hay | Target interneuron preference in thalamocortical pathways determines the temporal structure of cortical responses[END_REF] Excitatory postsynaptic currents (EPSCs), recorded in the presence of 1 µM tetrodotoxin (TTX) and 1 mM 4aminopyridine (4-AP), were evoked by light pulses.

For both in vivo and ex vivo pharmacological experiments, aCSF was superfused at 2-10 mL/min and the GABABR blocker CGP55,845 (1 µM) was bath applied for an incubation period of a minimum of 15 minutes before resuming the recording.

Drugs and chemicals were obtained from Sigma (Saint Louis, MO, USA) and Fisher Scientific UK Ltd (Loughborough, UK), except CGP55,845 hydrochloride (Bio-techne, Abingdon, UK) and tetrodotoxin citrate (Cambridge Bioscience Ltd, Cambridge, UK).

Immunofluorescence

After electrophysiological recordings, brains or slices were collected and fixed overnight at 4 °C in 4% paraformaldehyde/ 0.1 M phosphate buffer (PB), pH 7.4. Brains were cryoprotected using 30% sucrose in phosphate buffered saline (PBS). Brains were sliced to 60-80 µm thickness using a freezing microtome and kept in PBS at 4 °C. Slices were then rinsed and kept in PBS. Residual aldehyde was inactivated by incubation with 50 mM NH4Cl/PBS solution for 10 min at room temperature followed by several washes in PBS. Slices from optogenetics experiments were blocked for 2 h in PBS/0.25% Triton X-100/0.2% gelatine solution (PBS-GT) and then incubated with chicken anti-GFP (1:1000-2000, Abcam:

#ab13970) and for some slices with rabbit anti-Necab1 (1:1000, Cambridge Bioscience LTD: #HPA023629) for 1-3 days at 4 °C. After several washes in PBS-GT, slices were incubated overnight at 4 °C with goat anti-chicken Alexafluor-488 (1:400, Abcam: #ab150169) and Alexafluor 555-conjugated streptavidin (1:400, Invitrogen: #S32355) or donkey anti-rabbit Alexafluor 568 (1:400, Abcam: #ab175470) diluted in PBS-GT. After extensive washes in PBS, slices were mounted in Fluoromount-G (Fisher Scientific, UK).

Fluorescence images were acquired with a scanning confocal microscope (SP8, Leica) using 488-and 561-nm Argon lasers and analyzed using ImageJ.

QUANTIFICATION AND STATISTICAL ANALYSIS

In vivo recording analysis: spike detection Spike sorting was done using Klusta program in Python [START_REF] Rossant | Spike sorting for large, dense electrode arrays[END_REF] and each cluster was individually checked to ensure consistent spike waveform and spike timing auto-correlation and cross-correlation. To ensure clear differentiation between tonic and burst firing, only single units with a high signal-to-noise ratio were analyzed. Thus, the amplitude of the first spike of a burst had to exceed 5 SD of the baseline. Inter-spike interval plots were made for each individual cluster to assess the spikes belonging to a single neuron (Figure 1). Bursts and tonic action potentials were also extracted from this plot, with action potentials emitted within 5-20 ms of one another being considered to belong to the same burst and action potentials emitted at >30 ms of any other spike being classified as a tonic action potential. Any spike falling between 20 and 30 ms was manually assessed based on their amplitude with a smaller spike assessed as the last action potential of a burst while an action potential larger than the previous one was defined as a tonic spike or the first spike in a new burst. Association of spikes timing to the Up and Down state cycle was performed after detection of Up and Down state boundaries (see below). The duration of each Up and Down state was linearized and spike occurrence was plotted on rose plot where Up and Down states each take up half the cycle.

In vivo recording analysis: Up and Down states detection LFP was analyzed using custom-made procedures in Igor Pro and spike timing was aligned to the LFP in Igor Pro. The LFP signal was down-sampled to 1 kHz and low-pass filtered (120 Hz).

For LFP recordings in anesthetized animals, Up states were identified based on the power spectrum intensity in the gamma frequency range (30-80 Hz). Power spectrum density was extracted from Fourier transform analysis and Up states were automatically detected and subsequently manually assessed as events lasting >100 ms and whose gamma in-tensity was 3-5 SD above mean gamma power. Events <100 ms separated from another event by <15 ms were considered as a single continuous Up state, while isolated events <100 ms were discarded.

Determination of Up and Down states for in vivo whole-cell experiments was performed as for ex vivo whole-cell experiments (see below).

For recordings in naturally-sleeping mice, Up states were detected using the following procedure, adapted from [START_REF] Sheroziya | Global intracellular slow-wave dynamics of the thalamocortical system[END_REF] and [START_REF] Zucca | An inhibitory gate for state transition in cortex[END_REF]: First, to isolate the local field potential (LFP) the signal from the superficial (L1) electrode was subtracted from the signal from the deep (L5/6) electrode to reduce any confounding effects of volume conduction. Then, the DC component was removed from the LFP signal. A Morlet wavelet analysis was performed for frequencies ranging 10-50 Hz. The intensity of the spectrum at each time point for each frequency (f) was multiplied by the corresponding frequency to compensate for the 1/f attenuation factor and a vector was created which summed up at each time point the intensity * f signal. Lastly, for each time point, we multiplied the latest vector with the inverted LFP signal to create a time series incorporating information on the waveform of the LFP signal and on the intensity of the oscillations. From this time series Up and Down state boundaries were detected when the voltage trace crossed 0. In order to be classified as Down states and Up states, events needed to have a duration longer than 80 ms and 100 ms, respectively. Thus, Down events <80 ms were incorporated into flanking Up states as a long single continuous Up state; Up events <100 ms were incorporated into flanking Down states as a long single continuous Down state.

Up and Down states automatically detected were then visually confirmed.

The accuracy of the Up and Down state detection method was assessed by comparing it with two other commonly used methods (see Sanchez-Vives and McCormick, 2001;Saleem et al., 2010;[START_REF] Slézia | Phase advancement and nucleus-specific timing of thalamocortical activity during slow cortical oscillation[END_REF][START_REF] Gent | Thalamic dual control of sleep and wakefulness[END_REF][START_REF] Narikiyo | The claustrum coordinates cortical slow-wave activity[END_REF]. First, recordings were down-sampled to 10 kHz and band-pass filtered (200-1000 Hz) to isolate the multi-unit activity from the LFP (n = 6 recordings from 6 mice). Up states were detected for SD > 1.5-1.8 depending on the quality of the recovered multi-unit signal (Figure S1Civ).

On the same set of recordings, we also used the slow oscillation phase detection (see [START_REF] Lemieux | Neocortical inhibitory activities and longrange afferents contribute to the synchronous onset of silent states of the neocortical slow oscillation[END_REF]. LFP signal was low-pass filtered (<4 Hz) and state transition was defined at the half-amplitude of the sigmoid fitting of the transition. In our sample this detection method caused the detection of many false positive Down states as judged visually due to the difference of duration of Up and Down states. To counterbalance this, we adjusted the method described by Lemieux and collaborators by filtering separately frequencies <2 Hz and frequencies 2-4 Hz. Then Down state transitions were obtained when both filtered signals were positive. This method reduced the false positive Down states, while limiting false negatives to < 5% compared to Down state detected with the phase and gamma power method.

Last, we compared Down states induced by the light burst in Necab1-ChR2 experiments and detected by our method with grand average of optogenetic stimulation trials occurring during SWS. Peak amplitude was determined within 300 ms following the initiation of the light burst. Down state duration was measured as the time crossing 0 mV.

In vivo recording analysis: Detection of down transitions

For all optogenetic experiments, light-induced Down transitions were determined as the probability of transition within 50-300 ms following the onset of the stimulation, as reported throughout the manuscript. For silencing experiments (300 ms light pulse), a window of 300 ms was chosen, corresponding to the period of light delivery. For stimulation experiments, a transition window of 200 ms was chosen for local stimulation and 300 ms for thalamic stimulation. Shorter windows were used to compare the probability of transition with different windows. The probability of spontaneous transition was determined as the probability of transition within the same time window from a fixed time point midway be-tween light pulses (referred to as 'sham' light delivery). Up states occurring during optogenetic stimulation were compared with these control events, and with events occurring during light stimulation in GFP-only control experiments. Lastly, for ChR2 experiments in anesthetized animals, to ensure physiological relevance, the light intensity was adjusted to terminate ~90% Up states. For natural sleep experiments, the light intensity was 12 mW for all animals.

Ex vivo data acquisition and analysis

Neurons were visualized under infrared videomicroscopy, which allows spatial and morphological identification of L2/3 and L5 pyramidal neurons and L1 interneurons. Optogenetic stimulation was performed using either a mercury lamp controlled by a Uniblitz shutter system (Vincent Associations, Rochester, NY, USA) or a DPL-473 laser controlled by a UGA-40 point laser system (Rapp OptoElectronic, Wedel, Germany). For Up state experiments, the light intensity and shutter aperture duration were adjusted to terminate Up states in ~90% of the stimulations. Recordings were performed using a patch-clamp amplifier (Multiclamp 700B, Molecular Devices, Sunnyvale, CA, USA) connected to an ITC18 interface board (HEKA Instrument, Belmore, NY, USA). Data were digitized at 20 kHz and collected using custom-made procedures in Igor Pro (Wavemetrics, Lake Oswego, Oregon, USA). All offline analysis was done using custom-made procedures in Igor Pro. Automated detection of Up states was performed as previously described (Craig et al., 2013) and visually verified. Briefly, the root mean square (RMS) noise of the membrane potential was computed in 4 successive non-overlapping windows (noise_win_a, noise_win_b, noise_win_c, noise_win_d, 6 ms each). Then the noise_win_a/noise_win_b ratio was compared to the noise_win_c/noise_win_d ratio and a difference of 10% or more was considered as a state transition.

Inhibitory postsynaptic currents (IPSCs) were recorded at a holding membrane potential of -50 mV. The amplitude, charge transfer and decay time of IPSCs were determined averaged from 10 repeats. A baseline current was determined by averaging the current over 200 ms preceding the light pulse. The decay time was measured as the delay to return to within baseline noise level from the peak amplitude, and the charge transfer was defined as the area of the IPSCs. The GABABR blocker CGP55,845 (1 µM) was bath applied for a minimum of 15 minutes before resuming the recordings.

Statistical analysis

Results are given in the text as mean ± standard error of mean (SEM) and individual values are presented in the figures. All analyses are based on at least 200 spikes and 50 state transitions in each animal or preparation, using the number of mice as the statistical n. Between-group comparisons were performed using paired Mann-Whitney (M-W) nonparametric test. The Kolmogorov-Smirnov (K-S) test was used to test experimental distribution against uniform distribution or between two experimental distributions. Rayleigh distribution test was used to analyze the uniformity of the spike distribution during the Up and Down state cycle. In addition to the Rayleigh distribution, two-proportions z-test was used to test if spikes were associated to the first or second part of the Up states. Two-proportions z-test use was restricted to samples >100 data points when the two proportions tested had similar sample sizes. One-way ANOVA on ranks was done using Kruskal-Wallis (K-W) test, and two-way ANOVA on ranks (ART ANOVA) was done using standard ANOVA after data alignment and ranking using the ARTool package in R.

  ). Similarly, thalamic or cortical stimulation shortened Up states (Thal-Ctrl 864 ± 43 ms, Thal-Stim 572 ± 22 ms; n = 6; Local-Ctrl 807 ± 33 ms, Local-Stim 583 ± 23 ms; n = 6; paired M-W test, both p = 0.03) and delayed Down transition (Thal-Ctrl 417 ± 22 ms, Thal-Stim 240 ± 15 ms; n = 6; paired M-W test, both p = 0.03) to a similar extent (ART ANOVA: p = 0.49; Figure 2Cii). In contrast, stimulation of sensory thalamic fibers induced Up states in S1 cortical slices but did not terminate them (Figure S2E). Local application of CGP55,845 reduced Down transition probability within 300 ms in Necab1-ChR2 mice to control level (CGP-Ctrl: 43 ± 7.0%; CGP-Stim: 50 ± 7.4%, n = 3 Thalamus and n = 2 Local mice; paired M-W test, p = 1; Figure 2Ciii-iv).

  . Optogenetic stimulation of ChR2-expressing NDNF INs induced a biphasic IPSC in PCs, a slow component of which was blocked by CG-P55,845 (Figure 3A). Stimulation (one 20-ms-long pulse) during Up states in urethaneanesthetized NDNF-ChR2 mice terminated 89 ± 5% of Up states within 200 ms of stimulation onset (n = 7), compared to 35 ± 19% with sham stimulation (M-W test, p = 0.02) and 31 ± 6% in control NDNF-GFP mice (M-W test, p = 0.01), an effect abolished by local application of CGP55,845 (47 ± 9.8%; n = 3; M-W test, p = 0.04; Figure 3B). In contrast, stimulation of parvalbumin (PV) INs terminated only 28 ± 4% of Up states (n = 6). Light pulses reduced Up state duration in NDNF-ChR2 mice (ChR2-Ctrl: 737 ± 37 ms; ChR2-Stim: 466 ± 26 ms; n = 7; paired M-W test, p = 0.02) but not after application of CG-P55,845 (ChR2-Ctrl-CGP: 788 ± 50 ms; ChR2-Stim-CGP: 724 ± 55 ms; n = 3; paired M-W test, p = 1; ART ANOVA baseline vs CGP, p = 0.04; Figure 3Biii), and not in control NDNF-GFP mice (GFP-Ctrl: 891 ± 22 ms; GFP-Stim: 872 ± 22 ms; n = 4; paired M-W test, p = 0.13). In NDNF*ArchT mice, a 200 ms light pulse during Up states increased Up state duration (Base-Ctrl: 847 ± 10 ms; Base-ArchT: 1068 ± 28 ms; n = 7; paired M-W test, p = 0.02; Figure 3C), which was occluded by local CGP55,845 (CGP-Ctrl: 1033 ± 20 ms; CGP-ArchT: 1100 ± 42 ms; n = 4; paired M-W test, p = 0.25; Figure 2Cii,iv), with no effect in NDNF-GFP mice (GFP-Ctrl: 917 ± 28 ms; GFP-Light: 966 ± 28 ms; n = 4; paired M-W test, p = 0.63). Thus, L1 NDNF INs are targets of Necab1 thalamic neurons and they locally terminate Up states through GABABRs.

#Figure 1 .

 1 Figure 1. Necab1 thalamic neurons control cortical Down transition. (A) Thalamic

(

  Aiii) Rose plots of all spikes, bursts and tonic for all 34 units. (B) GFP immunohistochemistry following Cre-dependent ChR2-eYFP viral transduction of Necab1-Cre neurons. Coronal section of the thalamus (Bi) and S1 cortex (Bii) and fluorescence intensity across cortical layers. (C) Optogenetic silencing of midline thalamic Necab1 neurons. (Ci-ii) M1 LFP showing prolonged Up states during light in Necab1-ArchT but not Necab1-GFP. (Ciii) Up state duration for Necab1-ArchT (n = 6 mice) and Necab1-GFP (n = 6) during light delivery and control. Black lines represent mean duration per mouse. (Civ) Distribution of Up state termination delay from start of 300-ms light pulse in Necab1-ArchT and Necab1-GFP. (Cv) Probability of Down transition following light pulse in Necab1-ArchT (green) and Necab1-GFP (black). (D) Optogenetic stimulation of midline thalamic Necab1 neurons. (Di-ii) M1 LFP showing Down transition induced by burst of 3 light pulses (5 ms at 100 Hz) in Necab1-ChR2 but not Necab1-GFP and block by GABABR antagonist CGP55,845 (I.P.). (Diii) Up state duration for Necab1-ChR2 (n = 6) and Necab1-GFP (n = 6) during light delivery and control. Black lines, mean duration per mouse. (Div) Delay distribution from light burst initiation to Up state termination in Necab1-ChR2 (blue) and after CGP55,845 (red) compared to no-light control (black). (Dv) Probability of Down transition at different times after light delivery in Necab1-ChR2 (blue), and after CGP55,845 (red), compared to control without light (black). See also Figure S1.

Figure 2 .

 2 Figure 2. Neocortical GABAB receptors mediate thalamus-induced Down transition.

Figure 3 .

 3 Figure 3. Thalamus-induced Down transition is mediated via L1 NGFCs. (A) NDNF

Figure 4 :

 4 Figure 4: Thalamus synchronizes Down states across cortical areas via GABABRs in

Necab1-

  Cre line was generated in the Janelia Transgenic Mouse Core (https://www.janelia.org/support-team/gene-targeting-and-transgenics) and provided pre-publication by Drs Hantman and Dudman (Janelia Research Campus, USA, Phillips et al., 2019; Necab1: N-EF-hand calcium-binding protein 1; Sugita et al., 2002). The Sert-Cre line (Slc6a4-Cre) was kindly provided by Prof Patricia Gaspar (Institut du Fer à Moulin, France, Narboux-Nême et al., 2008) and the Gad67-Cre line was kindly provided by Prof. Hannah Monyer (University of Heidelberg, Tolu et al., 2013). The NDNF-Cre (Stock number: #028536), PV-Cre (Stock number: #008069), Ai32 (Stock number: #024109) and Ai40D (Stock number:

  p values < 0.05 were considered significant. Unless otherwise stated, all traces in the figures are single-trial events.

ACKNOWLEDGEMENTS

We are grateful to Drs Hantman and Dudman for pre-publication access to the Necab1-Cre mouse line. This work was supported by a BBSRC grant (BB/S015922/1) to Y.A.H. and O.P. and an Herchel Smith Fellowship to Y.A.H. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data and code availability

The datasets generated during this study are available at Mendeley Data repository (doi: 10.17632/87k2cg82rx.1 and doi:10.17632/kb3j4m6wvp.1).

AUTHOR CONTRIBUTIONS

DECLARATION OF INTERESTS

The authors declare no competing interests.

EXPERIMENTAL MODEL AND SUBJECT DETAILS #

Mice

All experiments were performed in accordance with United Kingdom Home Office regulations (Animals (Scientific Procedures) Act 1986 Amendment Regulations 2012) following ethical review by the University of Cambridge Animal Welfare and Ethical Review Body (AWERB). All animal procedures were authorized under Personal and Project licences held by the authors.

Mice were group housed in conventional, open cages with food and water ad libitum and maintained on a 12h-12h light-dark cycle, with temperature maintained at 22-24 °C and relative humidity kept at 50-55%. Experiments were carried out using wild type C57/BL6J mice and transgenic lines of mice crossed on C57 background. Wild type mice were pur- Steriade, M., Timofeev, I., Grenier, F. (2001). Natural waking and sleep states: a view from inside neocortical neurons. J. Neurophysiol. 85, 1969-1985. Sugita, S., Ho, A., Südhof, T.C. (2002). NECABs: a family of neuronal Ca 2+ -binding proteins with an unusual domain structure and a restricted expression pattern. Neuroscience 112, 51-63. Sun, Q., Li, X., Ren, M., Zhao, M., Zhong, Q., Ren, Y., Luo, P., Ni, H., Zhang, X., Zhang, C., et al. (2019). A whole-brain map of long-range inputs to GABAergic interneurons in the mouse medial prefrontal cortex. Nat. Neurosci. 22, 1357-1370. Swadlow, H.A., Gusev, A.G. (2001). The impact of 'bursting' thalamic impulses at a neocortical synapse. Nat. Neurosci. 4, 402-408. Tremblay, R., Lee, S., Rudy, B. (2016). GABAergic interneurons in the neocortex: From cellular properties to circuits. Neuron 91, 260-292. Tukker, J.J., Beed, P., Schmitz, D., Larkum, M.E., Sachdev, R.N.S. (2020). Up and Down states and memory consolidation across somatosensory, entorhinal, and hippocampal cortices. Front. Syst. Neurosci. 14, 22. Urbain, N., Fourcaud-Trocmé, N., Laheux, S., Salin, P.A., Gentet, L.J. (2019). Brain-statedependent modulation of neuronal firing and membrane potential dynamics in the somatosensory thalamus during natural sleep. Cell Rep. 26, 1443Rep. 26, -1457.e5.e5. This method is compared with a multi-unit activity (MUA) approach (Civ) and a phase of slow (<4 Hz) oscillation approach (Cv). All three methods yield consistent detection of Up and Down states. (Cvi) Multi-unit activity detection method detects 95.3 ± 3.3% (n = 6 mice) of Up states detected with the phase and gamma power method, while the phase of slow oscillation method detected 100% of the Up and Down states as detected with the phase and gamma method. However, this last method detects ~ 20% false positive Down state. This could be addressed by combining a phase detection of slow (<2 Hz) and delta (2-4 Hz) oscillations, which abolishes false positives but reduces the detection success (94 ± 4%). (Cvii) Down states are longer using the phase and gamma method compared with the multi-unit activity method due to the sliding window used for MUA detection (Phase- [START_REF] Petreanu | The subcellular organization of neocortical excitatory connections[END_REF]. (Biv) EPSC peak amplitude was bigger in NGFCs than in MPCs for the same light intensity (NGFC,68 ± 16.3 pA,n = 11;MPC,26 ± 7.8

SUPPLEMENTARY INFORMATION


Alexandre Alcoforado
email: alexandre.alcoforado@usp.br

Thomas Palmeira Ferraz

Rodrigo Gerber
email: rodrigo.gerber@usp.br

André Enzo Bustos
email: enzobustos@usp.br

Seidel Oliveira

Bruno Miguel Veloso
email: bruno.m.veloso@inesctec.pt

Fabio Levy Siqueira
email: levy.siqueira@usp.br

Anna Helena Reali Costa

André Seidel Oliveira

ZeroBERTo: Leveraging Zero-Shot Text Classification by Topic Modeling

Keywords: Low-Resource NLP, Unlabeled data, Zero-Shot Learning, Topic Modeling, Transformers

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The current success of supervised learning techniques in real-world Natural Language Processing (NLP) applications is undeniable. While these techniques require a good set of labeled data, large corpora of annotated texts are difficult to obtain, as people (sometimes experts) are needed to create manual annotations or revise and correct predefined labels. This problem is even more critical in languages other than English: statistics1 show that English is used by 63.1 % of the population on the internet, while Portuguese, for instance, is only used by 0.7 %. This scenario has contributed to the rise of Low-Resource NLP, which aims to develop techniques to deal with low data availability in a specific language or application domain [START_REF] Michael A Hedderich | A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios[END_REF].

Recently, the concept of zero-shot learning emerged in NLP: a semi-supervised approach in which models can present results equivalent to those of supervised tasks, such as classification in the absence of labeled data. Current approaches to the zero-shot text classification task (0SHOT-TC) make use of the good performance that Transformers have demonstrated in text entailment tasks [START_REF] Yin | Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach[END_REF]. In order to be able to process text in a way that is not uniquely suited to any specific task or data-set, these Transformers are first pre-trained in large general databases (usually taken from Wikipedia) and then fine-tuned into a small mainstream data-set for the natural language inference task (such as GLUE [START_REF] Wang | GLUE: A multi-task benchmark and analysis platform for natural language understanding[END_REF] and XNLI [START_REF] Conneau | XNLI: Evaluating Cross-lingual Sentence Representations[END_REF]). However, the use of models based entirely on Transformers falls into two critical problems: (i) limitation of the maximum size of the input text, and (ii) long run-time for large volumes of data. While there are transformer-based solutions to these problems individually [START_REF] Beltagy | Longformer: The long-document transformer[END_REF][START_REF] Sanh | Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter[END_REF][START_REF] Zaheer | Big bird: Transformers for longer sequences[END_REF], to the best of our knowledge, there is no solution that addresses both, nor even in the context of 0SHOT-TC.

In this paper, we propose a new hybrid model that merges Transformers with unsupervised learning, called ZeroBERTo -Zero-shot BERT based on Topic Modeling -, which is able to classify texts by learning only from unlabeled data. Our contribution not only handles long inputs -not limiting the input size and considering every input token to encode the data -but also offers a faster execution time. We propose an experimental setup with unlabeled data, simulating low-resource scenarios where real-life NLP researchers may find themselves. Then, we compare ZeroBERTo to a fully-Transformer-based zero-shot on a categorization dataset in Portuguese, FolhaUOL2 . Our results show that our model outperforms the previous one, in the best scenario, with about 12 % better label aware weighted F1-score and around 13 times faster total time.

The paper is structured as follows: Sect. 2 presents a background of how it is possible to move from data scarcity to zero-shot learning, as well as the related work on getting the best model for the 0SHOT-TC task. Sect. 3 formalizes the ZeroBERTo task and describe its training and inference procedures. Then, Sect. 4 describes the experimental setup that makes it possible to simulate low-resource scenarios to evaluate the proposed model. Finally, the discussion of the results of the experiments along with our final remarks is in Sect. 5.

Background and Related Work

The first approach to overcome the shortage of labeled data for classification suggests adopting data augmentation strategies [START_REF] Paul | Joining statistics with NLP for text categorization[END_REF], relying on methods to generalize from small sets of already annotated data. Still, the problem persists when there is an extreme lack of data. An alternative approach is to treat the task as a topic modeling problem. Topic modeling is an unsupervised learning technique capable of scanning a set of documents, detecting patterns of words and phrases within them, and automatically clustering groups of similar words and expressions that best characterize a set of documents [START_REF] Chen | Short text classification based on LDA topic model[END_REF]. There is usually a later labeling step for these clusters, which can be a problem as their interpretation is sometimes challenging, and a labeling error can affect the entire process. An automatic method for this is in our interest.

The context presented helps explain the growing interest in the field of Low-Resource NLP [START_REF] Chang | Importance of Semantic Representation: Dataless Classification[END_REF][START_REF] Michael A Hedderich | A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios[END_REF], which addresses traditional NLP tasks with the assumption of scarcity of available data. Some approaches to this family of tasks propose semi-supervised methods, such as adding large quantities of unlabeled data to a small labelled dataset [START_REF] Nigam | Text classification from labeled and unlabeled documents using EM[END_REF], or applying cross-lingual annotation transfer learning [START_REF] Bentivogli | Evaluating Cross-Language Annotation Transfer in the MultiSemCor Corpus[END_REF] to leverage annotated data available in languages other than the desired one. Other approaches try to eliminate the need for annotated data for training, relying, for example, on pre-trained task-agnostic neural language models [START_REF] Meng | Text Classification Using Label Names Only: A Language Model Self-Training Approach[END_REF], which may be used as language information sources, as well as representation learning models [START_REF] Ji | Representation Learning for Text-level Discourse Parsing[END_REF] for word, sentence or document classification tasks.

Recent breakthroughs in pre-trained neural models have expanded the limits of what can be done with data shortage. The Transformer model [START_REF] Vaswani | Attention is all you need[END_REF], which relies solely on attention mechanisms for learning, followed by BERT [START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF]] -a pre-trained Transformer encoder capable of deep bidirectional encoding -offered the possibility of using general-purpose models with previous language understanding. With little to no fine-tuning, BERT-like models have been successfully applied to most natural language comprehension tasks [START_REF] Logeswaran | Few-shot sequence learning with transformers[END_REF], and also show a significant reduction in the need for training data [START_REF] Tom B Brown | Language models are few-shot learners[END_REF]. Such models tend to work better for the 0SHOT-TC task, as they carry information about the context and semantic attributes within their pre-trained parameters. On the downside, pre-trained Transformers are complex, with millions of trainable parameters and slow processing of large quantities of data, and due to memory issues, most pre-trained Transformers cannot process inputs larger than 512 tokens at a time.

Also, attention models have another problem related to input size: attention cannot keep track of all information present in a large text, worsening the performance of the models.

In this context, zero-shot learning approaches stand out [START_REF] Socher | Zero-Shot Learning Through Cross-Modal Transfer[END_REF]. A simple way to explain zero-shot is to compare its paradigm with humans' ability to recognize classes of objects by having a high-level description of them without seeing an example of the object previously. [START_REF] Yin | Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach[END_REF] defines that 0SHOT-TC aims to learn a classifier f : X → Y , whereas classifier f (.), however, does not have access to data X specifically labeled with classes from Y . We can use the knowledge that the model already has to learn an intermediate layer of semantic attributes, which is then applied at inference time to recognize unseen classes during the training stages [START_REF] Zhang | Integrating Semantic Knowledge to Tackle Zero-shot Text Classification[END_REF].

Several works that seek to improve the performance of zero-shot learning inspired ours. [START_REF] Li | Semi-Supervised Zero-Shot Classification with Label Representation Learning[END_REF] worked in the image domain, seeking to develop a two-stage model that first learns to extract relevant attributes from images automatically and then learns to assign these attributes to labels. Our proposal performs the same two steps for the text classification problem but does not use any specific knowledge of external data or require any labelled data.

In the text domain, [START_REF] Mekala | Contextualized weak supervision for text classification[END_REF] defines weak-supervised learning similar to our definition of zero-shot learning. With unlabeled data and a list of classes as inputs, it applies seed word lists to guide an interactive clustering preview. [START_REF] Meng | Text Classification Using Label Names Only: A Language Model Self-Training Approach[END_REF] uses topic mining to find out which words have the same semantic meaning as the proposed labels, and with that makes a fine-tuning of the language model assuming the implicit category as the presence of these words in the text. Unlike these approaches, our model does not require the user to have any seed word for the labels, and instead of automatically learning them from the labels themselves, ZeroBERTo discovers them from the input data through topic modeling and then assigns them to the labels based on the language model used.

Proposed Method

In this section, we introduce ZeroBERTo which leverages Topic Modeling and pre-trained Language Models (LMs) for the task of zero-shot multi-class text classification (0SHOT-TC).

0SHOT-TC Task Formalization

Given a set of unlabeled documents D = {d 1 , d 2 , . . . , d n } and a set of m semantically disjoint and relevant label names L = {l 1 , l 2 , . . . , l m }, 0SHOT-TC aims to learn f : D×L → Θ, |Θ| = |L| and Θ defines a probability θ i j ∈ [0, 1] for each label l j being the label for d i [START_REF] Yin | Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach[END_REF]. A single-label classification of a document d i may then be carried out as l j ∈ L | j = argmax (j) (θ i 1 , θ i 2 , . . . , θ i m )as a notation simplification, for now on, we mention this as argmax (l∈L) (Θ i ).

Standard approaches to the 0SHOT-TC task treat it as a Recognizing Textual Entailment (RTE) problem: given two documents d 1 , d 2 , we say "d 1 entails d 2 " (d 1 ⇒ d 2 ) if a human reading d 1 would be justified in inferring the proposition expressed by d 2 (named hypothesis) from the proposition expressed by d 1 [START_REF] Daniel Z Korman | Defining textual entailment[END_REF]. In the case of 0SHOT-TC, d 2 is the hypothesis H(l j ), which is simply a sentence that expresses an association to l j . For example, in a news categorization problem, a label could be "sports" and a hypothesis for it could be "This news is about sports". Creating the hypothesis is essential to make it understandable by a Language Model, and allows us to discover the probability P (l j |d i ) = P (d i ⇒ H(l j )), as P (d i ⇒ H(l j )) can easily be inferred by a LM, using d i and H(l j ) as inputs. For the zero-shot text classification task, it calculates the textual entailment probability of each possible label. This inference, however, is quite demanding computationally.

ZeroBERTo

ZeroBERTo works differently: instead of processing the entire document in the LM, it learns a compressed data representation in an unsupervised way and only processes this representation in the LM. Thus, it is possible to obtain better performance with large inputs and shorter total time than the standard model, even considering the training time added by the unsupervised step.

To learn this representation, ZeroBERTo uses a statistical model, named Topic Modeling (TM), which examines documents and discovers, from the statistics of the words that occur in them, which abstract "topics" are covered, discovering hidden semantic structures in a text body. Given a set of unlabeled documents D, TM aims at learning a set of topics T . A topic t ∈ T is a list of q words or n-grams that are characteristic of a cluster but not of the entire documents set. Then, TM also learns how to represent any document d i ∈ D as a composition of topics expressed by

Ω T M (d i ) = (ω i 1 , ω i 2 , . . . , ω i k )
, such that ω i k denotes the probability of a document d i belonging to a topic t k .

With this in place, instead of analyzing the relation between document d i and label l j , we determine the entailment between the learned topic representation Ω T M (d i ) of each document and each label l j . Topics found are given as input to the LM, as a text list of words/expressions that represent the topic, in order to infer entailment probabilities. If the topic representation was learnt properly, then we can assume independence between l j and d i given a topic t k , therefore P (l j |t k , d i ) = P (l j |t k ) = P (t k ⇒ H(l j )). We then solve the 0SHOT-TC task by calculating the compound conditional probability

θ j i = P (l j |d i ) = t k ∈T P (l j |t k ) * P (t k |d i ) = t k ∈T P (t k ⇒ H(l j )) * Ω k T M (d i ) (1)
for each label l j to determine

Θ i = (θ 1 i , θ 2 i , . . . , θ m i ).
Classification is then carried out by selecting argmax (l∈L) (Θ i ).

Algorithm 1:

Given a set of documents D, a set of labels L, a hypotesis template H, a topic model T M and a Language model LM as input, ZeroBERTo-training (see Alg. 1) returns a trained model Z. For that, it trains T M on D using T M .FIT (line 2), that learns the topic representation of those documents. Then, it applies LM.PREDICT for all topics learned in T M (lines 4 to 7). This function, given a topic t k , returns the set of probabilities P (t k ⇒ H(l j )) for all l j ∈ L. In the end, the model Z gathers all information learned from D.

Algorithm 2: ZeroBERTo-prediction leverages a trained model Z and a specific document d i to return the predicted label l ∈ L (see Alg. 2). For this, it uses Z.T M.TOPICENCODER (line 1), that returns the topic representation Ω T M (d i ) of d i . This was learned by Z.T M in Alg. 1. Then, it calculates the equation ( 1) for all candidate labels (lines 2 to 8), returning the one with maximum probability. θ i j ← 0 5:

Algorithm 1 ZeroBERTo-training

for each t k ∈ Z.T M.topics do 6:

θ i j ← θ i j + (P(t k ) * Ω i T M (t k )) 7:
end for 8:

Θi ← Θi ∪ {θ i j } 9: end for 10: return argmax (l∈L) (Θi)

Experiments

In this section, we present the experiments performed to validate the effectiveness of ZeroBERTo. Considering that it would be difficult to evaluate our model in a real low-resource scenario, we propose an experimental setup to simulate low-resource situations in labeled datasets. We compare ZeroBERTo with the XLM-R Transformer, fine-tuned only on the textual entailment task. To perform the unsupervised training and evaluation, we use FolhaUOL dataset3 .

Dataset

The FolhaUOL dataset is from the Brazilian newspaper "Folha de São Paulo" and consists of 167,053 news items labeled into journal sections (categories) from January 2015 to September 2017. Categories too broad, that do not have a semantic meaning associated with a specific context (as the case of "editorial" and "opinion"), were removed from the dataset keeping only the categories presented in Table 1. For each news article, we take the concatenation of its title and content as input.

Table 1 presents the data distribution by category.

Models

We compare our proposal to the XLM-R model.

XLM-R is the short name for XLM-RoBERTa-large-XNLI, available on Hugging Face4 , which is state of the art in Multilingual 0SHOT-TC. It is built from XLM-RoBERTa [START_REF] Conneau | Unsupervised Cross-lingual Representation Learning at Scale[END_REF] pre-trained in 100 different languages (Portuguese among them), and then fine-tuned in the XNLI [START_REF] Conneau | XNLI: Evaluating Cross-lingual Sentence Representations[END_REF] and MNLI [START_REF] Williams | A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference[END_REF] datasets (which do not include the Portuguese language). It is already in the zero-shot learning configuration described by [START_REF] Yin | Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach[END_REF] with template hypothesis as input. The template hypothesis used was "O tema principal desta notícia é {}" and texts larger than the maximum size of XLM-R (512 tokens) are truncated.

ZeroBERTo The implementation of our model here makes use of BERTopic [START_REF] Grootendorst | BERTopic: Leveraging BERT and c-TF-IDF to create easily interpretable topics[END_REF] with M-BERT-large (Multilingual BERT) [START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF] as topic modeling step, and the same XLM-R described above as the Transformer for associating the topic representation of each document to labels. Repeating the use of XLM-R seeks to make the comparison fair. BERTopic's hyperparameters are: interval n for n-grams to be considered in the topic representation (n_grams_range ∈ {1, 2, 3}); number of representative words/ngrams per topic (top_n_words = 20); and minimum amount of data in each valid topic (min_topic_size = 10). The XLM-R template hypothesis used is "O tema principal desta lista de palavras é {}".

Evaluation

To simulate real-world scenarios, we propose a variation of stratified k-fold cross-validation [START_REF] Refaeilzadeh | Cross-validation[END_REF]. First, we split the data into k disjoint stratified folds, i.e. the data were evenly distributed in such a way as to make the distribution of the classes in the k folds follow the distribution in the entire dataset. Next, we use these k-folds to perform the following 4 experiment setups:

Exp. 1 -Labeling a dataset: Simulates a situation where one needs to obtain the first labeling of a dataset. ZeroBERTo is trained in (k -1) folds and has the performance compared to XLM-R in the same (k -1) folds, in order to assess its ability to label data already seen. Since this is unsupervised learning, evaluating the model's labeling ability in the training data makes sense as it was not exposed to the ground truth labels.

Exp. 2 -Building a model for additional inferences: Simulates a situation where the researcher wants to create a current model in a real-life application without having data labeled for it.

ZeroBERTo is trained in (k -1) folds and can infer new data compared to XLM-R on the remaining fold.

-R ZeroBERTo XLM-R ZeroBERTo XLM-R ZeroBERTo XLM-R ZeroBERTo P
0.47 ± 0.00 0.66 ± 0.01 0.46 ± 0.01 0.16 ± 0.08 0.46 ± 0.01 0.64 ± 0.01 0.47 ± 0.00 0.29 ± 0.17 R 0.43 ± 0.00 0.54 ± 0.01 0.43 ± 0.00 0.21 ± 0.05 0.43 ± 0.00 0.56 ± 0.02 0.43 ± 0.00 0.31 ± 0.12 F1 0.43 ± 0.00 0.54 ± 0.01 0.42 ± 0.01 0.15 ± 0.07 0.42 ± 0.01 0.52 ± 0.02 0.43 ± 0.00 0.19 ± 0.17 We evaluated the performance of both models for each experiment with the following label-aware metrics: weighted-average Precision (P), weighted-average Recall (R), and weighted-average F1score (F1). For the k-fold CV, we use k = 5. Experiments were run on an Intel Xeon E5-2686 v4 2.3GHz 61 GiB CPU and an NVIDIA Tesla K80 12 GiB GPU using the PyTorch framework. To run XLM-R, we use batches sized 20 to prevent GPU memory overflow.

Results

Table 2 shows the results of the proposed experiments. Time for ZeroBERTo considers unsupervised training time and inference time. Further, as no training is required, only a single run of XLM-R was done on all data. Thus, the times for XLM-R are estimated. Nevertheless, in all experiments, the total time (training + execution) of ZeroBERTo was much lower than the execution time of XLM-R. Our model surpassed XLM-R in all metrics in the experiments in which the evaluation was performed on the data used in the unsupervised training (Exp. 1 and 3). Figure 1 presents a visualization for the entailment mechanism between topics and labels represented by term P (l j |t k ) = P (t k ⇒ H(l j )) in equation ( 1). The darker the green, the greater the conditional odds.

Discussion and Future Work

The experiments simulated low-resource scenarios where a zero-shot text classifier can be useful.

The results showed that it is possible to obtain a better performance in the 0SHOT-TC task with the addition of an unsupervised learning step that allows a simplified representation of the data, as proposed by ZeroBERTo. Moreover, the proposed model presents itself as an excellent tool to help researchers deal with low-resource scenarios, such as the need to label an entire dataset without any previously labeled training data. Another interesting feature is that the model showed evidence of robustness for smaller amounts of data. In experiment 3, it was trained with 25 % of the data from experiment 1 and got similar performance metrics in lower time, refuting our concern that little data could be a bottleneck in the model.

However, for configurations where ZeroBERTo was tested simulating real-life applications (Exp. 2 and 4), being exposed to new data, the performance was worse than XLM-R. The results suggest it occurs due to the inability of the embedded topic model to adequately represent new data as a composite of previously learned topics, overfitting training data. This is clear from observing the high variance of the metrics among the k-folds. It allows us to conclude that, for now, the scenarios presented in experiments 1 and 3 are more suitable for using the model.

We have 0.54 of F1-score in the best scenario regarding the metrics obtained. Despite being a positive result considering that it is a multi-class classification, there is much room for improvement. The main reason to be pointed out for low performances is the use of multilingual models that were not fine-tuned in the Portuguese language, which is quite impressive.

A critical remark to be made is concerning the memory and time trade-off. For example, ZeroBERTo was more than 10x faster than XLM-R in Exp. 3. However, the topic model used by ZeroBERTo bases its clustering on the HDBSCAN method, which reduces time taken for data processing but increases the need for memory [START_REF] Mcinnes | Accelerated hierarchical density based clustering[END_REF], which XLM-R does not do. As the size of input data grows, processing may become unfeasible. XLM-R, on the other hand, does not use any interaction between data and can be processed in parallel and distributed without any negative effect on the final result. It should be noted, however, that ZeroBERTo does not depend on BERTopic and can use other Topic Modeling techniques that address this issue more adequately in other scenarios.

A significant difficulty of this work was that, as far as the authors are aware of, there are no large benchmark datasets for multi-class text classification in Portuguese, nor general use datasets with semantically meaningful labels. In this sense, some future work directions involve the production of benchmark datasets for Portuguese text classification (and 0SHOT-TC). It would also be interesting to produce Natural Language Inference datasets in Portuguese, which could, in addition to the existing ones [START_REF] Fonseca | ASSIN: Avaliacao de similaridade semantica e inferencia textual[END_REF][START_REF] Real | The ASSIN 2 Shared Task: a quick overview[END_REF], enable fine-tuning of Transformers 100 % in Portuguese. Then, it would be possible to compare the performance of the models using BERTimbau (BERT-Portuguese) [START_REF] Souza | BERTimbau: pretrained BERT models for Brazilian Portuguese[END_REF] both in clustering and classifying. It would also be worthwhile to test the proposed model in other domains: to name one, legislative data present similar challenges [START_REF] Palmeira Ferraz | DEBACER: a method for slicing moderated debates[END_REF]. Another interesting future work would be to enable ZeroBERTo to deal with multi-label classification, where each document can have none, one or several labels.

  4: for each t k ∈ T M.topics do 5: p k ← LM.predict(t k , H(L)) 6: P ← P ∪ {p k } 7: end for 8: Z.T M ← T M 9: Z.P, Z.L ← P, L 10: return Z Algorithm 2 ZeroBERTo-prediction Require: Z, di Ensure: l 1: Ω i T M ← Z.T M.TOPICENCODER(di) 2: Θi ← {} 3: for each lj ∈ Z.L do 4:

Figure 1 :

 1 Figure 1: Figure shows text entailment results between topics (X-axis) and labels (Y-axis) for the first 50 Topics (sorted by size) in fold 0 from Experiment 3. In total, 213 topics were generated in this experiment.

Table 1 :

 1 Number of articles by news category within FolhaUOL dataset after cleaning and organizing the data.

	Category	# of articles	Category	# of articles
	Poder e Política	22022	Educação	2118
	Mercado	20970	Turismo	1903
	Esporte	19730	Ciência	1335
	Notícias dos Países	17130	Equilíbrio e Saúde	1312
	Tecnologia	2260	Comida	828
	TV, Televisão e Entretenimento	2123	Meio Ambiente	491

Table 2 :

 2 Table shows the results of the experiments for the FolhaUOL dataset. P is weighted-average Precision, R is weighted-average Recall, and F1 is weighted-average F1-score.

	Exp. 1	Exp. 2	Exp. 3	Exp. 4
	XLM

Statistics available at: https://w3techs.com/technologies/overview/content_language. Accepted at PROPOR

2022: 15th International Conference on Computational Processing of Portuguese

Available at: https://www.kaggle.com/marlesson/news-of-the-site-folhauol.

Available at: https://www.kaggle.com/marlesson/news-of-the-site-folhauol.

Available at: https://huggingface.co/joeddav/xlm-roberta-large-xnli

Acknowledgments

This research was supported in part by Itaú Unibanco S.A., with the scholarship program of Programa de Bolsas Itaú (PBI), and by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Finance Code 001, CNPQ (grant 310085/2020-9), and USP-IBM-FAPESP Center for Artificial Intelligence (FAPESP grant 2019/07665-4), Brazil. Any opinions, findings, and conclusions expressed in this manuscript are those of the authors and do not necessarily reflect the views, official policy, or position of the financiers.