Andrea Stefani

Alexandra Bogdan

Flavia Pop

Francesco Tassinari

Luca Pasquali

Claudio Fontanesi
email: claudio.fontanesi@unimore.it

Narcis Avarvari
email: narcis.avarvari@univ-angers.fr

Spin dependent electrochemistry and electrochemical enantioselective recognition with chiral methylated bis(ethylenedithio)-tetrathiafulvalenes

I. INTRODUCTION

One of the most exciting properties of chiral molecules and materials, which currently triggers a massive interest in chemistry, physics and biology, is the electron spin selective transmission referred to as chirality induced spin selectivity (CISS) effect. 1,2 Accordingly, in a simplified model, an electron of up spin traverses a homochiral medium more easily than the down spin, the situation being reversed for the opposite handedness of the molecule. 3 The CISS effect stems from the experimental observation that charge transmission through enantiopure chiral interfaces is spin-selective. As a result, the current flowing through an enantiopure system is spin-polarized. It has been shown that this phenomenon can have huge interest, among others, in quantum technologies, 4 enantioselective chemistry, 5 enantiomer separation 6 or water splitting. 7,8,9 Since the pioneering results of Naaman et al. on the exceptional high spin polarization values reached with double-stranded DNA selfassembled monolayers on gold in low-energy photoelectron transmission experiments, 10 several CISS active systems have been described, 11,12,13 including small chiral molecules such as helicenes deposited on metallic surfaces 9,14 or on highly oriented pyrolytic graphite (HOPG). 15 One of the manifestation of the CISS effect is through the spin dependent electrochemistry (SDE) paradigm, where the charge transmission is determined via electrochemical methods. Typically, cyclic voltammetry (CV) measurements as a function of spin-injection from a ferromagnetic chirally functionalized electrode show current intensity depending on the magnetization of the electrode relative to the chirality of the adsorbed compound. 16,17,18 Thus, several chiral substrates such as the transmembrane protein bacteriorhodopsin (bR), [START_REF] Mishra | Proc. Natl. Acad. Sci[END_REF] helical oligopeptides [START_REF] Kettner | [END_REF] or double-stranded DNA 21 in the presence of the K4[Fe(CN)6]/K3[Fe(CN)6] bulk redox couple, chiral conductive polymers in the presence of Fc/Fc + (Fc = ferrocene), 22 or L-or D-cysteine molecule (either L or D) covalently functionalized by the redox probe toluidine blue O (TBO) 23 have been deposited on Ni or Ni/Au surfaces which served as working electrodes. Values of the spin polarization (SP), defined as SP = I↑ -I↓/I↑ + I↓, where I↑ and I↓ represent the current when the nickel electrode is magnetized up or down by an external permanent magnet, generally ranging between 5 and 20% have been measured. Note that in the case of the helical conducting polymers SP reached up to 34 -50% at short times after the voltage pulse (2 -3 s) and drastically decreased at longer times, possibly as a consequence of changes in the polymer conformation under the influence of the applied electric field. 22 These polymers were actually polythiophenes and they were very likely adsorbed on the Ni/Au electrode thanks to the favorable Au•••S interactions. It is therefore interesting to use sulfur containing chiral compounds to coat the surface of this type of magnetic electrodes.

Besides the CISS effect through the spin-dependent electrochemistry, another direct application of chirally functionalized electrodes is the possibility to electrochemically address the enantiomer discrimination of a chiral analyte, a field of investigation which is continuously increasing as well. 24,25 Indeed, electron transfer processes taking place at enantiopure chiral interphases translate into different potentials for the two enantiomers of a chiral probe in voltammetric measurements. 26 Moreover, the enantiomer concentration and even an estimation of the enantiomeric excess can be determined from the relation between concentration and current intensity.

Certainly, the most important family of sulfur rich electroactive molecular precursors for chiral crystalline conductors is that of chiral tetrathiafulvalenes (TTF), 27 which considerably developed within the last decade, 28 very much motivated by the observation of direct or indirect chirality triggered effects in the electrical conductivity, such as the electrical magnetochiral anisotropy (eMChA) effect 29 for the former or the modulation of the crystalline packing for the latter. 30,31,32 Some of the most popular chiral TTFs are the methylated BEDT-TTF [BEDT = bis(ethylenedithio)] derivatives, such as DM-BEDT-TTF (DM = dimethyl) 1 33 and TM-BEDT-TTF (TM = tetramethyl) 2, 34 , 35 which provided important series of chiral conductors with peculiar properties and structures, 36,37,38,39,40 not encountered in the achiral homologue BEDT-TTF (Fig. 1). However, neither CISS investigations nor electrochemical enantio-discrimination have been reported so far with chiral TTF precursors, despite of the propensity of TTF to self-assemble on Au surfaces. 41 We report herein spin-dependent electrochemistry (SDE), as a consequence of the CISS effect, and chiral electrochemical recognition with the chiral BEDT-TTF derivatives 1 and 2.

II. EXPERIMENTAL

A. Materials

Sodium nitrate NaNO3 ≥ 99.0%, KCl ≥ 99.0%, ferrocyanide K4[Fe(CN)6], ferricyanide K3[Fe(CN)6] are from Sigma Aldrich and were used without further purification. Ethanol C2H6O ≥ 99.5% from Merck was used without further purification. 3,4-Dihydroxy-L-phenylalanine (referred as L-DOPA in the text), (S)-(-)-N,N-dimethyl-1-ferrocenylethylamine 98% (referred as S-FC in the text), (R)-(+)-N,N-dimethyl-1-ferrocenylethylamine 97% (referred as R-FC in the text) are from Alfa Aesar and were used without further purification. A ferrocyanide 5 mM / ferricyanide 5 mM in a KCl 0.1 M aqueous solution is referred as the Fe(III)|Fe(II) redox couple in the text. DM-BEDT-TTF 1 and TM-BEDT-TTF 2 were prepared according to the published protocols. 33,34 A.1 Gold functionalization Au(111)-oriented surfaces were obtained by e-beam evaporation of 100 nm of gold, on a silicon substrate (Siltronix), with thermally grown 300 nm SiO2 (100), > 400 Ω cm, on an 8-nm-thick Cr adhesion layer, followed by gentle flame annealing just prior to use (in the following Au stays for Au(111)-oriented). Gold functionalization is obtained by 24 hours incubation of the Au surface in a 5 mM TTF-derivative solution in ethanol.

A.2 Silver nanoparticles preparation

AgNPs are obtained exploiting the electrochemical based method of Starowicz et al., 42 obtaining AgNPs with dimension ranging between 20 and 50 nm. 43 A.3 TTF-derivative capped AgNPs preparation [(S)-1@AgNPs, (R)-1@AgNPs, (S)-2@AgNPs and (R)-2@AgNPs)]

The electrochemically fabricated AgNPs (vide supra, section A.2) were added to a 5 mM TTFderivative ethanolic solution in order to obtain TTF capped-silver nanoparticles ((S)-2@AgNPs for example). The solution was left at rest under dark for 24 hours at room temperature. Then, the resulting pale-yellow solution (free of visible precipitate to the naked eye) was subjected to centrifugation at 6000 rpm for 30 minutes, followed by decantation of the supernatant and replacement with fresh solvent. This cycle is repeated twice to purify the AgNPs@TTF-derivatives from excess, non-adsorbed molecules.

A.4 Nickel functionalization

Ni surfaces were obtained by e-beam evaporation of 100 nm of nickel, on a silicon substrate (Siltronix) with thermally grown 300 nm SiO2 (100), > 400 Ω cm, on an 8-nm-thick Ti adhesion layer.

A one-step Ni-functionalized preparation strategy was followed. Ni functionalized surfaces were obtained by immersion in a suspension of TTF-derivative capped Ag nanoparticles (yielding a hybrid Ni|(S)-1@AgNPs chiral interface and so on). During the functionalization process the Ni surface was kept at a negative (reducing) potential of -0.3 V, to avoid nickel oxidation.

B. Electrochemical set-up

Electrochemical measurements were performed using an Autolab PGSTAT 128N and CH-Instruments CHI660A potentiostats. A typical three-electrode cell is employed: a Pt sheet and a silver, silver chloride, KCl saturated solution (Ag/AgCl/KClsat) electrodes were the counter and reference electrodes, respectively. Evaporated gold substrates served as the working electrode (WE): 100 nm gold (Lesker) was evaporated on top of a 10 nm Ti layer on SiO2 (100) wafer (Siltronix) and then followed by gentle flame annealing to obtain Au(quasi-111). 0.1 M KCl and K2SO4 aqueous solution were used as base electrolyte. All the potentials reported in this work are referred to the Ag/AgCl/KClsat reference electrode.

C. UHV surface characterization

X-ray photoemission spectroscopy measurements (XPS) were carried out employing a MgKα source.

The electron analyzer was an Omicron EA125, installed at LFMS lab, Department of Engineering Enzo Ferrari, University of Modena, Italy. Photoemission measurements were carried out at normal emission with a resolution of 0.5 eV. The S 2p XPS spectra have been fitted, after a Shirley background subtraction, using Voigt doublet functions accounting for the spin-orbit splitting of 1.2 eV between S 2p3/2 and S 2p1/2 components. The measurements were performed at room temperature.

Binding energies are referred to the Fermi energy of a gold substrate. on a bare and freshly cleaned gold surface, the red line curve is the CV recorded on the Au functionalized with (S)-2; please note the substantial decrease in the current and the absence of neat peaks, replaced by two "shoulders" featuring a potential difference of about 300 mV. Fig. 2(b) shows the electro-desorption CVs for the Au/(S)-2 interface, which acts as the WE: the first two scans are carried out in the -0.1 to 0.5 V potential window (red line) then the potential scan window is widened to -1.6 V (black line). In the first scan in the negative potential range two peaks (reduction current, negative in sign) are evident in the -1.0 to -1.4 V potential range (labelled as Edes1); in the following potential scans the current is definitively lower showing a much weaker peak at -1.0 V (labelled Edes2).

III. RESULTS AND DISCUSSION

A. Au electro-desorption experiments

Figs. 2(c) and 2(d) show exactly the same type of experiment carried out for the Au functionalized with (S)-1. The blue curves in Figs. 2(c) and 2(d) refer to the functionalized surface, for which a similar behavior is observed compared to Figs. 2(a) and 2(b), excepting the Edes1 pattern. Indeed, the latter is much more neat and defined with respect to the result observed for (S)-2. Thus, the SAM formed by (S)-1 is very likely a little more ordered with respect to the SAM formed by (S)-2. In addition, the charge calculated by integration of the Edes1 peaks yields a surface area of 0.55 and 0.80 nm 2 for the SAMs of 1 and 2, respectively, suggesting an orientation of 1 a little more tilted with respect to the electrode surface, while 2 should be adsorbed almost "flat" on the surface. Indeed, the latter values of surface coverage match reasonably well with the theoretical projected area of (S)-1 and (S)-2 disposed almost flat on the electrode surface, the calculations of the surface having been performed by using the Van der Waals radii (see the SI). 1 V s -1 potential scan rate: the red line is the Au|(S)-2 interface acting as the WE, the black line is the same Au|(S)-2 WE after the electro-desorption procedure; (b) 0.5 V s -1 potential scan rate: the red line is the CV forward 2 scans recorded before electro-desorption, the black curves feature a main peak (labelled as Edes1 in Fig. 2b) in the first reductive scan and a small peak (Edes2) in the second scan (due to the electro-desorption of (S)-2): the first CV forward scan starts from OCP to 0.5 V, then the potential is scanned from 0.5 to -1.7 V; the successive 6 CV scans are carried out in the -1.7 to 0.5 V potential range; (c) 0.1 V s -1 potential scan rate: the blue line is the Au|(S)-1 interface acting as the WE, the black line is the same Au|(S)-1 WE after the electro-desorption procedure; (d) 0.5 V s -1 potential scan rate: the blue line is the CV forward 2 scans recorded before electro-desorption, the black curves feature a main peak (labelled as Edes1 in Fig. 4d) in the first reductive scan and a small peak (Edes2) in the second scan (due to the electrodesorption of (S)-1): the first CV forward scan starts from OCP to 0.5 V, then the potential is scanned from 0.5 to -1.7 V; the successive 6 CV scans are carried out in the -1.7 to 0.5 V potential range. 44,45,46,47,48,49 ). The absence of any signal for energy values larger than 166.5 eV further supports the effective gold surface functionalization (signals at energies larger than 166.5 eV are typical of oxidized sulfur). 50,51 All in all, XPS results concerning the Au|(R)-1 and Au|(R)-2 interfaces suggest bonding between the metal and the chiral molecules.

B. XPS analysis

C. Enantio-recognition with the chiral gold electrodes

Electrochemical enantiomer discrimination in both analytical and preparative contexts is of great fundamental and applicative interest. 24,25 This technique most commonly involves the occurrence of a diastereomeric relation between a chirally modified electrode surface, serving as working electrode (WE), and a chiral analyte. Accordingly, chiral modifiers such as tartaric acid, 52 chiral PEDOT (poly-3,4-ethylenedioxythiophene) derivatives, 53 oligomers of the axially chiral 2,2'-bis[2-(5,2-bithienyl)]-3,3'-bi-thianaphthene (BT2T4), 54 Cu(II) cysteine complexes, 55 thiahelicenes, 56 have been deposited

on Ni, Au or glassy carbon electrodes for the enantio-recognition of a large variety of analytes including the commercially available N,N-dimethyl-1-ferrocenylethylamine (S-FC and R-FC hereafter for the (S) and (R) enantiomers), provided with planar chirality thanks to the electroactive ferrocene unit, or the L-dopamine (L-Dopa hereafter). 56,57 In the present work we have evaluated the enantio-discrimination ability of our Au/1 and Au/2 chiral interfaces as working electrodes (WE) towards N,N-dimethyl-1-ferrocenylethylamine and Ldopamine. As can be observed from Figs. 4(a), 4(b), 4(e) and 4(f) excellent enantio-recognition is obtained with S-FC and R-FC for both platforms, as evidenced by the large differences in the peak potential values, i.e. average values of 150 mV for the system Au/1//FC (compare Fig. 4(a) and 4(b))

and of 90 mV for the system Au/2//FC (compare Fig. 4(e) and 4(f)). It can be thus hypothesized that the interaction between the chiral TTFs 1-2 and FC involves the chiral group of the ferrocene unit, possibly through hydrogen bonding interactions that can establish between the NMe2 group attached to the stereogenic center of FC and H atoms of the ethylene groups. 58 Note that for both systems the oxidation of FC is perfectly reversible.

For comparison sake we report in Table 1 the peak potential values reported for closely related systems, i.e. Au, Ni or glassy carbon (GC) electrodes and various chiral analytes, including N,Ndimethyl-1-ferrocenylethylamine (FC) and L-Dopa as in the present study. The enantio-discrimination of our platforms with L-Dopa is comparatively less efficient, but still not negligible, as highlighted in Figs. 4(c) and 4(d). As usually observed for this analyte, its oxidation is irreversible, 56 therefore only the oxidation peak potentials are considered in order to determine the diastereomeric potential differences, which are just larger than 50 mV for Au/1/L-Dopa (compare Fig. 4(c), indeed referred to the onset potential, peaks are rather broad) and 120 mV for Au/2/L-Dopa (compare Fig. 4(d)) with an oxidation at lower potentials for the (R) enantiomers of 1 and 2. Different competing intermolecular interactions could take place between L-Dopa and the TTFs 1 and 2, such as: i) hydrogen bonding involving the H atoms of the ethylene bridges of 1 and 2, as mentioned above, and the ortho-dihydroxy group of L-Dopa, which would not provide an efficient discrimination as this group is remote from the stereogenic center, ii) - interactions between the two  frameworks, which would probably afford a better chiral recognition for 2 which possesses stereogenic centers on both sides of the  system, and, finally, iii) hydrogen bonding involving the chiral -amino-acid group of L-Dopa and the H atoms of the TTF ethylene groups. All in all, the enantio-discrimination towards L-Dopa is more efficient for 2, containing four stereogenic centers, than for 1, with two stereogenic centers.

D. Spin dependent electrochemistry (SDE)

The ultimate goal of our investigations with the chiral TTF precursors 1 and 2 was the demonstration of the CISS effect through spin dependent electrochemistry experiments. To reach this goal, we have prepared ferromagnetic nickel electrodes coated with enantiopure 1 and 2, according to previous reports on such chiral working electrodes. [START_REF] Mishra | Proc. Natl. Acad. Sci[END_REF] Functionalized Ni/TTF@AgNPs surfaces were obtained by overnight incubation of the compounds 1 and 2 capped with AgNPs (see the section II.A.4).

Although our chiral precursors are redox active themselves, their first oxidation potentials occur at 0.50-0.53 V vs SCE (saturated calomel electrode). Unfortunately, these oxidation values are more positive than the oxidation potential of the nickel surface. We have therefore decided to use as redox mediator AgNPs, 43 spin-dependent electrochemistry. [16][17][18] However, the present system is very easy to implement since the chiral mediator consists in the simplest chiral TTF precursors for which the synthesis is straightforward. Going into details: the SDE results for 2 compared to 1 appear a bit more robust (this at variance of enantio-recognition results, please compare Figure 4 CVs and the relevant discussion in section C). This could be most likely related to the different Ni-TTF interaction: spin-injection efficiency depends also on the reciprocal spatial dispositiongeometry of adsorption on the electrode of the chiral molecule on the electrode surface. Indeed, precursor 1 contains a more accessible side, without methyl substituents, which is prone to interact stronger with the surface (although possibly leading to a twisted adsorbed disposition). In addition, 2 possesses stereogenic centers on both sides of the TTF framework, thus having a chiral space more pronounced than in 1. These results point out the importance of the proper selection of the chiral mediators and their interaction with the electrode in order to maximize the chiral induction.

IV. CONCLUSIONS

We have demonstrated the efficiency of two chiral methylated TTF derivatives for electrochemical chiral enantio-discrimination and for chirality induced spin selectivity through spin dependent electrochemistry measurements. The chiral precursors are suitable for the functionalization of gold and nickel surfaces which show very good stability in a wide range of potential. Excellent enantiorecognition was achieved with both Au/TTF platforms towards N,N-dimethyl-1ferrocenylethylamine, with potential differences between the two enantiomers of 90 -150 mV, while the recognition of L-Dopa is more effective with compound 2. The CISS effect is clearly more robust for the latter as well, with average spin polarization values of ±15%, very likely thanks to the more chiral environment created at the interface between the chiral TTF and the nickel surface and thus correlating with the higher ratio of bound vs unbound sulfurs as observed by XPS in compound 2 with respect to 1. These results are a clear proof-of-concept concerning the use of chiral TTFs for electrochemical enantio-discrimination and for the spin polarization and open up numerous perspectives. The electrochemical recognition of redox inactive substrates can be envisaged when taking into account that TTFs are redox active all the more since their redox potentials can be finely tuned by changing the substitution scheme. To comply with the relatively low oxidation potential of metallic Ni, chiral metal bis(dithiolene) complexes can be used as spin polarizers in SDE experiments as they possess much lower oxidation potentials and have accessible reduction stable states. 59,60

SUPPLEMENTARY MATERIAL

Bulk electrochemistry measurements for 1 and 2, XPS details and surface area estimation for 1 and 2.

FIG. 1 .

 1 FIG. 1. Chiral TTF derivatives 1 and 2 investigated in this work for SDE and their achiral homologue BEDT-TTF.

Figure 2 shows

 2 Figure 2 shows CVs relevant to the electro-desorption experiments of enantiopure (S)-1 and (S)-2 SAMs chemisorbed on Au, after overnight incubation in ethanol solution. CVs were recorded in a 5 mM Fe(III)|Fe(II), 0.1 M KCl aqueous solution, Fig. 2(a) black line shows the reference CV recorded

FIG. 2 .

 2 FIG. 2. CV curves of a 5 mM Fe(III)|Fe(II), 0.1 M KCl aqueous solution, recorded at the Au|(S)-2 (a and b panels) and

Fig. 3 1 and 2

 312 Fig. 3 shows the S2p XPS spectra relevant to the Au, Ni and Ag surfaces functionalized with the 1 and 2 SAMs: red dots are the experimental data, blue thick line the overall fit of the experimental curve, thin lines represent single Voigt doublets. Au and Ni functionalized surfaces have been produced by incubation of clean surfaces in solution (details of the functionalization procedure are reported above in section IIA), whilst functionalized Ag nanoparticles have been directly dropcasted on a clean glassy carbon flat surface.

Figs. 3 FIG. 3 .

 33 Figs.3(c) and3(d) show XPS S 2p spectra for the Ni|(R)-1 and Ni|(R)-2 interfaces, a rather neat difference being observed between them. Spectra in Figs.3(c) and 3(d) feature a prominent peak at 164 eV which has the typical line-shape of thiophene-based SAMs. Both interfaces present the two components related to bound and free thia-heterocycles, with Ni|(R)-2 showing a slightly larger chemisorbed/unbound component ratio with respect to Ni|(R)-1, suggesting a stronger bonding for Ni|(R)-2. Also, a prominent component is found for Ni|(R)-1 at binding energies larger 166.5 eV, indicating a not negligible amount of oxidized sulfur, whilst for the Ni|(R)-2 interface this peak is significantly smaller.

FIG. 4 .

 4 FIG. 4. Voltammetric measurement of the Au|(R)-1 & Au|(S)-1 interfaces (panels a, b, c) and of the Au|(S)-2 & Au|(R)-2 interfaces (panels d, e, f), acting as the WE. CVs were recorded with a 0.05 V s -1 potential scan rate and the support electrolyte is 0.1 M tetrabutylammonium tetrafluoroborate (TBATFB) in ACN solution. (a) CV curves in 5 mM S-FC (black curve) or R-FC (red curve); the WE is the Au|(R)-1 interface; (b) CV curves in 5 mM S-FC (black curve) or R-FC (red curve); the WE is the Au|(S)-1 interface; (c) CVs curves in 2 mM L-Dopa recorded at the Au|(R)-1 (red curve) and at the Au|(S)-1 interface (black curve); (d) CVs curves in 2 mM L-Dopa recorded at the Au|(S)-2 (blue curve) and at the Au|(R)-2 interface (orange curve); (e) CV curves in 5 mM R-FC (orange curve) or S-FC (blue curve); the WE is the Au|(R)-2 interface; (f) CV curves in 5 mM S-FC (orange curve) or R-FC (blue curve); the WE is the Au|(S)-2.

FIG. 5 .

 5 FIG. 5. Spin-dependent electrochemistry CVs: a Pt sheet and Ag/AgCl/KClsat are used as the CE and RE, respectively. CVs were recorded with a 0.05 V s -1 potential scan rate in a 0.1 M KCl aqueous solution. (a) Ni|(R)-1|AgNPs interface acts as WE; (b) Ni|(S)-1|AgNPs interface acts as WE; (c) Ni|(R)-2|AgNPs interface acts as WE; (d) Ni|(S)-2|AgNPs interface acts as WE. Black solid curve magnet DOWN orientation. Red solid curve magnet UP orientation. The main panels display the data after background subtraction (removal of the double layer contribution), while the insets show the raw data from the corresponding measurements.The calculated spin polarization values are 6% for (R)-1 and 13% for (S)-1, spin polarization being defined as:

TABLE I .

 I Average peak potential difference for the electrochemical enantio-recognition.

	Reference	Working electrode (WE) Chiral analyte	E (mV)
	[52]	Ni/tartaric acid	glucose	120
	[53]	GC/PEDOT	Dopa	50
	[54]	GC/oligo-(BT2T4)	ferrocenes	20 -120
	[56]	GC/[7]thiahelicene	FC	250
	[56]	GC/[7]thiahelicene	Dopa	350
	this work	Au/1 and Au/2	FC	150 and 90
	this work	Au/1 and Au/2	Dopa	50 and 120

ACKNOWLEDGMENTS C.F. gratefully thanks financial support from Dipartimento di Ingegneria "Enzo Ferrari" (DIEF), UniMORE, FARD 2021 -linea di azione di tipo 3: "Materiali chirali per batterie al litio e celle a combustibile" and from Consorzio Interuniversitario Nazionale per la Scienza e Tecnologia dei Materiali (INSTM), fondi triennali: "INSTM21MOFONTANESI". This work was supported in France by the CNRS, the University of Angers and the French National Agency for Research (ANR) project SECRETS (ANR PRC 20-CE06-0023-01). Financial support from the French Ministry of Foreign Affairs and University of Angers through an Eiffel scholarship for A.B. is also acknowledged. The authors warmly acknowledge Prof. Ron Naaman (Weizmann Institute of Science, Israel) for fruitful discussions.

DATA AVAILABILITY

The data that support the findings of this study are available within the article and its supplementary material.

AUTHOR DECLARATIONS

Conflict of Interest

The authors have no conflicts to disclose.

Author contributions

Andrea


Aimé Lachal

Keywords:

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.


Manel Ben-Romdhane

Chiheb Rebai

Adel Ghazel

Patricia Desgreys
email: patricia.desgreys@enst.fr

Patrick Loumeau

Pseudorandom Clock Signal Generation for Data Conversion in a Multistandard Receiver

Keywords: Radio Receivers, Signal sampling, Antialiasing, Digital Control Unit

In previous work, a Non-Uniform Sampling (NUS) technique to control Analog-to-Digital Conversion (ADC) in a multistandard Radio receiver was proposed. In this context of wide band radio signals, the NUS-based ADC offers the advantages of relaxing antialiasing filter (AAF) constraints, decreasing the sampling frequency average and reducing ADC dynamic power consumption. In this paper, we focus on generating non-uniform clock for the sample and hold block of the ADC. The proposed non-uniform clock is based on quantified clock timing. Non-uniform clock is generated from Digital Control Unit (DCU). The DCU is composed of a Gray counter, a Linear Feedback Shift Register (LFSR) and a multiplexer to generate specified signals and to select the required clock phase. Simulation results show that performances are similar for conventional uniform sampling (US), Additive Random Sampling (ARS) and Time Quantized Additive Random Sampling (TQ-RS) for different quantizing factors.

I. INTRODUCTION

Today, the new commercial challenge to enhance mobile technologies business, need the development of multi-standard wireless mobile equipments offering to end users multi-mode and multi-services facilities by using a single low-cost, lowpower and highly integrated devices. Software Defined Radio (SDR) is the ultimate approach to reach this goal.

In previous work, a novel concept of Non-Uniform Sampling technique for controlling Analog-to-Digital Conversion was proposed to relax constraints of receiver circuits supporting multistandard broadband WiMax/WiFi processing [START_REF] Ben-Romdhane | Non-Uniform Sampled Signal Reconstruction for Multistandard WiMax/WiFi Receiver[END_REF]. This NUS technique is used to eliminate spectral replicas at integer multiples of sampling frequency produced by conventional uniform sampling technique [START_REF] Shapiro | Alias-Free Sampling of Random Noise[END_REF][START_REF] Ben-Romdhane | Non-Uniform Sampling Schemes for IF Sampling Radio Receiver[END_REF][START_REF] Artyukh | Wideband RF Signal Digitizing for High Purity Spectral Analysis[END_REF]. NUS-based ADC will deliver non-uniform samples data stream that need to be converted into uniform samples adapted to DSP stage data stream. A Reconstruction Algorithm (RA) is required at the output of the ADC to achieve this operation. To achieve the required processing, a Digital Control Unit (DCU) is proposed to provide non-uniform clock to the ADC. The main contribution presented in this paper is DCU development and non-uniform timing signal generation to control the ADC clocking. Signal delivered by DCU will be used in the DSP stage to define sampling occurrences for the RA.

The paper is organized as following. Section 2 presents the multistandard receiver architecture for WiMAX (IEEE802.16 [START_REF]Part 16: Air Interface for Fixed Broadband Wireless Access Systems[END_REF]) and WiFi (IEEE802.11a[6]). NUS technique formulation used in this work is given in section 3. Building blocks of proposed DCU are detailed in section 4. Section 5 is dedicated to simulation results and performances analysis in terms of SNR for proposed Time quantized random sampling processing.

II. MULTISTANDARD NUS-BASED HOMODYNE RECEIVER

It is proposed for the WiMAX and WiFi receiver design to use a homodyne topology as illustrated in figure 1. This topology has the advantage of shifting all radio signals processing at base band. It is the most adapted, among analog down conversion topologies, to SDR and multistandard processing. It also reduces the number of discrete components and improves flexibility [START_REF] Kilpatrick | New SDR architecture enables ubiquitous data connectivity[END_REF]. Image problem is solved by zero-IF quadrature downconversion. Digital algorithms can be implemented in the DSP stage to compensate I/Q mismatch [START_REF] Sohn | Data-aided approach to I/Q mismatch and DC offset compensation in communication receivers[END_REF] and the DC-offset [START_REF] Tsurumi | Broadband RF Stage Architecture For Software-defined Radio in Handheld Terminal Applications[END_REF] produced in the homodyne receiver. Designed receiver is composed of one RF filter because WiMAX and WiFi have the same band. This filter is followed by a wide-band LNA, two mixers and two ADCs. Conventional uniform sampling on the ADC requires selective AAF to suppress alias spectral bins (in case of using Nyquist converter) or relaxed AAF (in case of using over-sampling converter). A trade-off could be achieved between relaxing AAF selectivity and decreasing sampling frequency rate thanks to non-uniform sampling [START_REF] Ben-Romdhane | Non-Uniform Sampled Signal Reconstruction for Multistandard WiMax/WiFi Receiver[END_REF]. Such trade-off implies reconstruction algorithm filter in digital domain to recover uniform sampled signal. This digital filter could be easily performed by Digital Signal Processing since the average of samples arrival times is equal to the Nyquist rate relative to the channel bandwidth. This architecture offers advantages of relaxing AAF constraints and reducing ADC dynamic consumption.

III. NUS PROCESSING FORMULATION

Non-uniform sampling process converts a continuous analog bandpass signal x(t) into its discrete representation x s (t) as indicated in equation ( 1).

() () ( ) +∞ -∞ = - = k k s t t t x t x δ (1)
The sampling instant sequence {t k } is defined as t k <t k+1 with T s =1/f s is the mean of the sampling period.

Non-uniform sampling theory is a set of techniques characterized by irregular sampling sequences randomly distributed. Additive Random Sampling (ARS) sequence is the first defined random sampling process known as alias-free processing [START_REF] Shapiro | Alias-Free Sampling of Random Noise[END_REF]. The sampling is:

,... 2 , 1 , 0 , 1 = + = - k t t k k k τ (2)
With {τ k } is a set of independent identically distributed random variables with a probability density function (pdf) p k (τ), a mean T s and a variance σ 2 . The pdf of the k th sampling instant is given by equation (3).

For any distribution with ARS sequence, the process is alias-free because sampling point density satisfies stationary condition [START_REF] Bilinskis | Randomized Signal Processing[END_REF]:

() () s k k T p p 1 1 = = +∞ = τ τ (4) ARS sequence is characterized by σ /T s ∈[0,0.3] to guarantee t k in [(k-0.5) T s , (k+0.5) T s ].
Pseudorandom generators are used for hardware implementation [START_REF] Artyukh | Wideband RF Signal Digitizing for High Purity Spectral Analysis[END_REF]. In [START_REF] Wojtiuk | Randomized Sampling for Radio Design[END_REF], author proposed quantizing axis time by using the Time Quantized Random Sampling TQ-RS. Each sampling interval is quantized and represented by [START_REF]Part 16: Air Interface for Fixed Broadband Wireless Access Systems[END_REF] according to (6). This schema is illustrated in figure 2.

Δ = n q τ with T s q T = Δ (5) Δ + ≤ < Δ - 2 1 2 1 n n τ (6) n=0 n=1 n=3 n=2 n=1 n=0
Random Sampling Time-Quantized Random Sampling n Occurence of the quantized time in the samplig period Based on simulation results, we note that TQ-RS allows to digitize analog bandwidth equal to q T .f s /2 where q T is quantization time factor and f s the mean sampling frequency. This property increases the AAF's transition band by the quantization time factor and consequently decreases AAF order.

IV. DIGITAL CONTROL UNIT (DCU)

To sample non-uniformly, we need to have sevral phases of sampling instead of conventional uniform sampling. For instance, to sample non-uniformly using TQ-RS schema with a factor q T equal to 8, eight clock phases are required. The delay between two successive clock phases is the resolution Δ of TQ-RS defined as T s /q T and the duty cycle of all phases {φ 0 ,… , φ 6 , φ 7 }. The main drawback is the phases' overlap between {φ 0 , φ 1 , φ 2 , φ 3 } and {φ 4 , φ 5 , φ 6 , φ 7 }.

A. DCU architecture

The proposed architecture of the Digital Control Unit (DCU) is used to generate the control signal to activate nonuniformly the sample and hold bloc of the ADC and overcome the overlap of sampling phases. The generation of the control signal is implemented, as illustrated in figure 3, using the following blocks :

• Gray counter, • Combining function to generate different sampling phases of sampling with different duty cycles,

• Linear Feedback Shift Register (LFSR) to generate a pseudorandom number, and finally, • Multiplexer which acts as a selective combiner between different phases and LFSR result to select the signal corresponding to the pseudorandom number generated each period of the principal clock.

The clock controlling the DCU has to be as fast as the resolution needed for the TQ-RS schema.

B. Gray counter and combiner

To overcome overlap, we propose to use a gray counter outputs combined to deliver the needed phases for the nonuniform sampling. The sampling time is the rise edge of the available phases. The Gray counter is controlled by CLK and is based on a shift register with log 2 (q T ) flip-flops.

Non-uniform sampling using TQ-RS schema with a factor q T equal to 8 requires 3 flip-flops, 7 gates 'NAND', 4 gates 'NOR' and one gate 'XOR'. In this case, the resulting generated phases for non-uniform sampling are presented in figure 4.

C. LFSR

Phases generated, from the Gray counter and combiner, are selected according to a random number. A Linear Feedback Shift Register (LFSR) with n length characteristic polynomial G[x] generates a binary pseudorandom sequence each period p=2 n -1 when initial state is nonzero. LFSR synthesis requires at least n flip-flops. Here we propose to generate pseudorandom numbers using an LFSR composed of log 2 (q T ) flip-flops at CLK/q T rate. In our previous example, we need an LFSR as represented in figure 5. The characteristic polynomial is given by (7).

1 ] [ x x x G + + = (7) 0V 0V 0V S D CP R Q _ Q S D CP R Q _ Q S D CP R Q _ Q Init CLK x0 x1 x2

D. Selective combiner

Having the different sampling phases and the pseudorandom number, we have to introduce a selective combiner to select the phase according to the delay iΔ where i is the pseudorandom number generated by the LFSR being in the set {0,1,…, q T -1}. The selective combiner is a multiplexer q T inputs to one output controlled by log 2 (q T ) signals.

E. General results

In figure 6, phases generated by the Gray counter and combiner (Φ 0 ,Φ 1 ,...,Φ 7 ), pseudorandom numbers "L 2 L 1 L 0 " and the result pseudorandom clock CLK NUS are given.

The mean sampling frequency corresponds to the frequency of

T CLK s q f f / =
where CLK f is the reference clock frequency.

Sampling frequency takes values between

CLK T T f q q
)) 1 2 /( ( and CLK f . ADC should be able to convert at all these rates to finally deliver data at mean sampling frequency and so diminush his dynamic power consumption. V. PERFORMANCE RESULTS WiMAX and WiFi have channel bandwidth (BW) equal to 15,75 MHz and 16.6 MHz respectively and channel spacing of 20 MHz. When signal test x(t) is sinusoidal and has a frequency f in in the range [DC,BW/2], the mean sampling frequency f s is 16.6 MHz. In this case of non-uniform sampling, antialiasing requires a 3 rd order filter. However, Nyquist sampling needs an 25 th AAF order [START_REF] Ben-Romdhane | Non-Uniform Sampled Signal Reconstruction for Multistandard WiMax/WiFi Receiver[END_REF]. The NUS sequence used is TQ-RS for a time quantization Δ=1/(q T .f s ), where q T the quantization time factor is an integer.

Simulation results are performed for cubic spline interpolation. Simulation cases are conventional uniform sampling US, additive random sampling ARS and time quantized random sampling TQ-RS for different values of quantification factor (q T =8, 16, 32) using the pseudorandom clock generated from a reference clock frequency 132.8MHz. Figure 7 gives simulation results for WiMAX/WiFi signals processed for 12-bit signal quantization. for different casesof qT [START_REF] Sohn | Data-aided approach to I/Q mismatch and DC offset compensation in communication receivers[END_REF]16,32) According to results of figure 7, performances in terms of SNR are simular for all cases. All of simulation cases return an SNR reconstruction between 70 dB and 77 dB obtained for different σ /T s values. Therefore, we can satisfy the dynamic ranges required by the WiMAX and WiFi standards using the ADC controlled by the pseudorandom clock defined in the present work.

VI. CONCLUSION

In previous works, a system analysis of a baseband multistandard wide-band receiver architecture time-controlled by Non-Uniform Sampling technique was proposed. The main motivation of proposed radio design is to reduce ADC and AAF circuit complexities observed for SDR and multistandard radio design comparing with conventional uniform sampling technique. In this paper, authors proposed a pseudorandom clock signal generation to be applied to digitizing circuit. The proposed clock deliver quantized timing and is generated from digital control unit requiring gray counter, LFSR, decoder and some combiner blocs. Quantizing resolution corresponds to the reference clock period. Obtained simulations show that random sampling and time quantized random sampling for some quantization time factors return similar SNR performances when applying cubic spline interpolation for reconstruction algorithm. Conventional sampling returns the same SNR however the required AAF order is widely higher.

Figure 1 .

 1 Figure 1. Multistandard NUS-based Homodyne Receiver

Figure 2 .

 2 Figure 2. TQ-RS description for qT=4

Figure 3 .

 3 Figure 3. Description of the DCU (example qT=8)

Figure 4 .

 4 Figure 4. Modified clock phases for non-uniform sampling

3

 3

Figure 5 .

 5 Figure 5. LFSR Architecture

Figure 6 .

 6 Figure 6. Non-uniform clock signal (DCU output)

Figure 7 .

 7 Figure 7. Performances versus sm=σ /Ts : Comparison between conventional uniform sampling (US), Additive Random sampling (ARS) and Time Quantized Additive Random Sampling (TQ-RS)for different casesof qT[START_REF] Sohn | Data-aided approach to I/Q mismatch and DC offset compensation in communication receivers[END_REF] 16, 32)

ACKNOWLEDGMENT

The authors would like to thank the CMCU organization (Comité Mixte de Coopération Universitaire) to financially support this project.


Bilal Faye
email: faye@lipn.univ-paris13.fr

Hanane Azzag
email: azzag@univ-paris13.fr

Mohamed-Djallel Dilmi
email: dilmi@lipn.univ-paris13.fr

Context Normalization Layer with Applications 1 st

Deep neural networks (DNNs) have gained prominence in many areas such as computer vision (CV), natural language processing (NLP), robotics, and bioinformatics. While their deep and complex structure enables powerful representation and hierarchical learning, it poses serious challenges (e.g., internal covariate shift, vanishing/exploding gradients, overtting, and computational complexity), during their training phase. Neuron activity normalization is an effective strategy that lives up to these challenges. This procedure consists in promoting stability, creating a balanced learning, improving performance generalization and gradient ow efciency. Traditional normalization methods often overlook inherent dataset relationships. For example, batch normalization (BN) estimates mean and standard deviation from randomly constructed mini-batches (composed of unrelated samples), leading to performance dependence solely on the size of mini-batches, without accounting for data correlation within these batches. Conventional techniques such as Layer Normalization, Instance Normalization, and Group Normalization estimate normalization parameters per instance, addressing mini-batch size issues. Mixture Normalization (MN) utilizes a two-step process: (i) training a Gaussian mixture model (GMM) to determine components parameters, and (ii) normalizing activations accordingly. MN outperforms BN but incurs computational overhead due to GMM usage. To overcome these limitations, we propose a novel methodology that we named "Context Normalization" (CN). Our approach assumes that the data distribution can be represented as a mixture of Gaussian components. However, unlike MN that assumes a-priori that data are partitioned with respect to a set of Gaussian distributions, CN introduces the notion of concept that accounts for data relationship via a neural network classication scheme. Samples that are gathered within a cluster dene a context. The estimation of the Gaussian components parameters is conducted through a supervised neural network-based concept classication. CN is more precise when clusters are thick and not sparse. Extensive comparative experiments conducted on various datasets demonstrates the superiority of CN over BN and MN in terms of convergence speed and performance generalization. In fact, CN outperforms BN and MN with a convergence speed margin of 5% and a performance margin of 10%. These results reveal the importance and the need of capturing inherent data context to learn the Gaussian component parameters. Our proposed approach harnesses data relationships, and therefore enhances deep learning models in various applications.

I. INTRODUCTION

Normalization is a general process that transforms data to possess specic statistical properties. Various methods exist for data normalization, with some of the most common techniques including centering, scaling, standardizing, decorrelating, and whitening [START_REF] Kessy | Optimal whitening and decorrelation[END_REF]. Input normalization can be used in deep neural networks (DNNs) training to remove the magnitude difference between features, speeding convergence during linear model optimization [START_REF] Lecun | Efcient backprop[END_REF]. In layered neural networks, as the input is only directly connected to the rst weight matrix, it is not clear how the input impacts the optimization landscape with respect to other weight matrices. The initial weights are typically not normalized, and this can indeed have a signicant impact on the gradient optimization procedure. To overcome this, some methods employ weight-initializing techniques to obtain equal variances for layer input/output gradients across different layers [START_REF] He | Delving deep into rectiers: Surpassing human-level performance on imagenet classication[END_REF]. However, due to the updating of the weight matrices during training, the equal variance property across layers may be broken. From this perspective, it is important to normalize activations during training, to obtain similar benets of normalizing inputs. Different normalization techniques, including activation normalization, weight normalization, and gradient normalization, are employed to enhance the training performance of DNNs. To normalize activations, the most common technique is Batch Normalization (BN) [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF]. BN has been proposed to solve the problem caused by the changing distribution of the inputs of each layer during training, called internal covariate shift. In fact, the distribution change problem often requires lower learning rates and careful parameter initialization, which slows down the training process and makes it difcult to train models with saturating nonlinearity. As a layer of the neural network, BN methods perform the normalization on each training minibatch with its respective mean and variance. This mini-batchwise approach allows for a more appropriate representation of the data and, therefore, faster processing, thus increasing the performance of the neural network in terms of convergence.

Despite the good performance, the effect of BN depends on the mini-batch size, and it is not obvious how to apply it to some DNNs architectures. To address this, several variants and alternative methods have been proposed [START_REF] Huang | Normalization techniques in training dnns: Methodology, analysis and application[END_REF]. BN and its few extensions can be studied from the viewpoint of Fisher kernels that arise from generative probability models. Kalayeh and al. [START_REF] Kalayeh | Training faster by separating modes of variation in batch-normalized models[END_REF] show that assuming samples within a minibatch are from the same probability density function, then BN is identical to the Fisher vector of a Gaussian distribution. More specically, each instance in the mini-batch is assigned to a component of the Gaussian Mixture Model (GMM), where the GMM approximates the probability density function of the input activations. Instead of computing one set of statistical measures from the entire population (of instances in the minibatch) as BN does, Mixture Normalization (MN) [START_REF] Kalayeh | Training faster by separating modes of variation in batch-normalized models[END_REF] proposes a normalization on sub-populations which can be identied by disentangling modes of the distribution, estimated via GMM. In addition to speeding up training, MN allows better accuracy results to be achieved.

In this perspective, we propose a novel normalization technique called context normalization (CN). In fact, assuming that the data are well modeled by a mixture of several components, each sample in the mini-batch is normalized using the mean and variance of the associated component. Indeed, the capability of GMM to approximate any continuous distribution with arbitrary precision has been demonstrated by [START_REF] Titterington | statistical analysis ofnite mixture distributions[END_REF]. Building upon this foundation, our paper follows a similar track but introduces a novel method. In particular, we dene a context that can come from various sources that describe the structure of the dataset. A context can be conceptualized as a coherent cluster of samples that share common characteristics and can be effectively grouped together. Each context can be viewed as a component of the Gaussian mixture with its own probability density function. By normalizing samples from the same context with the parameters learned during backpropagation, CN allows an estimation of the mean and variance of each mixture component thus improving the discrimination power of the data representation according to the target task.

In summary, the main contributions of this work are as follows:

• We propose Context Normalization (CN), a novel approach that utilizes dened contexts to capture underlying distribution variations. In CN, each sample in a mini-batch is normalized using the mean and standard deviation specic to its context. By treating contexts as components of a Gaussian mixture, we learn their parameters during model training, eliminating the need for the EM algorithm. This leads to improved efciency and simplied implementation of CN.

• Through a comprehensive set of experiments, we demonstrate that CN not only accelerates model convergence, but also achieves superior nal test accuracy. These results highlight the effectiveness of our proposed method in improving the overall performance of models.

For consistency, we use the variable notation proposed in

: m ∈ [1, N ]×[1, H]×[1, W ]}
with attened x across all but the channel axis by:

xi = x i -µ B  σ 2 B + ϵ , (1)
where

µ B = 1 m  m i x i , σ 2 B = 1 m  m i (x i -µ B ) 2
are the mean and variance respectively, and ϵ > 0 a small number to prevent numerical instability. Due to the constraints introduced by standardization, additional learnable parameters γ and β are introduced to eliminate the linear regime of nonlinearity of some activations:

xi = γ xi + β (2)
During inference, population statistics are needed for deterministic inference. They are usually computed by running the average over the training iterations, as follows:

 μ = (1 -λ)μ + λµ B σ2 = (1 -λ)σ 2 + λσ 2 B ( 3
)
If the samples within the mini-batch are from the same distribution, the transformation in Equation ( 1) generates a zero mean and unit variance distribution. This zero-mean and unit-variance constraint allows stabilizing the distribution of the activations and thus benets training. This mini-batch-wise approach makes it possible to have a more suitable representation of the data and, therefore, faster processing, thus increasing the performance of the neural network in terms of convergence. Despite the good performance, the effect of BN is dependent on the mini-batch size, and the discrepancy between training and inference limits its usage in complex networks (e.g. recurrent neural networks).

To tackle these challenges and address parameter estimation on unrelated samples, numerous variants, and alternative methods have been proposed.

B. Variants of Batch Normalization

Some extensions to batch normalization (BN) have been proposed, in particular, Layer Normalization (LN), Instance Normalization (IN), Group Normalization (GN) and Mixture Normalization (MN) [START_REF] Huang | Normalization techniques in training dnns: Methodology, analysis and application[END_REF], [START_REF] Kalayeh | Training faster by separating modes of variation in batch-normalized models[END_REF]. The general transformation x → x according to a mini-batch, on the attened spatial domain (L = H × W ) , can be written as follows

v i = x i -E Bi (x), xi = v i  E Bi (v 2 ) + ϵ , (4)
given The batch normalization (BN) [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF] hypothesis that a Gaussian distribution can model the generative process of mini-batch samples is less valid. To address this, Mixture Normalization (MN) [START_REF] Kalayeh | Training faster by separating modes of variation in batch-normalized models[END_REF] investigates BN from the viewpoint of Fisher kernels, which arise from generative probability models. Rather than using a mean and standard deviation calculated over entire instances within a mini-batch, MN uses a Gaussian Mixture Model (GMM) to affect each instance in the mini-batch to a component and then normalizes with respect to multiple means and standard deviations associated with different modes of variation in the underlying data distribution. MN normalizes each sample in the mini-batch using the mean and standard deviation of the mixture component to which the sample belongs to. The probability density function p θ that describes the data can be parameterized as a Gaussian Mixture Model (GMM). Let

B i = {j : j N ∈ [1, N ], j C ∈ [i C ], j L ∈ [1, L]}, where i = (i N , i C , i L ) a
x in R D , if θ = {λ k , µ k , Σ k : k = 1, ..., K}, p(x) = K  k=1 λ k p(x|k), s.t. ∀ k : λ k ≥ 0, K  k=1 λ k = 1, (5)
where

p(x|k) = 1 (2π) D/2 |Σ k | 1/2 exp  - (x -µ k ) T Σ -1 k (x -µ k ) 2  ,
is the k th Gaussian in the mixture model p(x), µ k the mean vector and Σ k is the covariance matrix. The probability that x has been generated by the k th Gaussian component in the mixture model can be dened as:

τ k (x) = p(k|x) = λ k p(x|k)  K j=1 λ j p(x|j) ,
Based on these assumptions and the general transform in Equation ( 4), the Mixture Normalizing Transform for a given x i is dened as

xi = K  k=1 τ k (x i ) √ λ k xk i , (6)
given

v k i = x i -E Bi [τ k (x).x], xk i = v k i  E Bi [τ k (x).(v k ) 2 ] + ϵ , (7)
where

τk (x i ) = τ k (x i )  j∈Bi τ k (x j )
, is the normalized contribution of x i with respect to the minibatch B i in the estimation of the statistical measures of the k th Gaussian component. With this approach, Mixture Normalization can be done in two stages:

• estimation of mixture model's parameters θ = {λ k , µ k , Σ k : k = 1, ..., K} by Expectation-Maximization (EM) [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] algorithm.

• normalization of each sample with respect to the estimated parameters (Equation ( 7)) and aggregation using posterior probabilities (Equation ( 6)). On convolutional neural networks, this method allows Mixture Normalization to achieve better results than batch normalization in terms of convergence and accuracy in supervised learning tasks.

III. PROPOSED METHOD: CONTEXT NORMALIZATION

A. Method description

Based on the Mixture Normalization (MN) hypothesis proposed by [START_REF] Kalayeh | Training faster by separating modes of variation in batch-normalized models[END_REF] (ref. to Figure 1), our Context Normalization (CN) approach operates under a similar assumption that data can be effectively represented by a mixture of multiple components, as opposed to batch normalization (BN) [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF]). In the Context Normalization (CN) approach, a fundamental concept is introduced, namely, the notion of context, which represents a cluster of samples sharing common characteristics that can be efciently grouped together. Unlike the Expectation-Maximization (EM) algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] typically employed for parameters estimation in each component, CN utilizes a deep neural network to learn these parameters through contextbased normalization. In our approach, we assign a unique identier to each context and utilize it for normalization during training. Samples within the same context share the same identier, allowing for alignment in a shared space that aligns with the target task. This approach not only facilitates the normalization of samples within the same context but also enables the estimation of optimal parameters for all contexts, promoting the convergence of the model. By leveraging these context identiers, our approach enhances the alignment and adaptability of the model to different contexts, leading to improved performance.

B. Parameters learning

For a given x i in B i , the mean (µ r ) and standard deviation (σ r ) are estimated based on the context r associate to x i . This estimation process is outlined in Algorithm 1 and visually depicted in Figure 2. Subsequently, these estimated parameters are used to normalize x i according to Equation [START_REF] Dumoulin | A learned representation for artistic style[END_REF]. The CN transform can be added to a deep neural network to provide a better representation of the input data. In Algorithm 1, xi = CN (x i , r) indicates that the parameters µ r and σ r are to be learned depending on each training example according to the context r. Algorithm 2 summarizes the procedure for training context-normalized networks.

xi ← x i -µ r  σ 2 r + ϵ (8)
In algorithm 1, we use an afne transformation followed by element-wise linearity:

α r ← Embedder r (onehot(r)) : W r onehot(r) + b r , µ r ← Embedder µ (α r ) : W µ α r + b µ , σ 2 r ← Embedder σ (α r ) : W σ α r + b σ
where W and b are learned parameters of the Embedder model and r is the context associated to x i . The onehot(.) is the function that performs one-hot encoding on the categorical

B i = {x i } m i=1 ; context r associated to x i Output: { xi = CN(x i , r)} α r ← Embedder r (onehot(r)) // identier embedding µ r ← Embedder µ (α r ) //
Θ = Θ ∪ {µ r , σ r } T r=1
variable representing the context r, which is embedded in the continuous space α r . During the training process, it is necessary to backpropagate the gradient of the loss function, denoted as ℓ, through the transformation. Additionally, the gradients with respect to the parameters of the CN transform need to be computed. This is achieved using the chain rule, as shown in the following expression (before simplication):

∂ℓ ∂µ r = ∂ℓ ∂ xi . ∂ xi ∂µ r = - ∂ℓ ∂ xi .(σ 2 r + ϵ) -1/2 ∂ℓ ∂σ 2 r = ∂ℓ ∂ xi . ∂ xi ∂σ 2 r = µ r + x i 2(σ 2 r + ϵ) 3/2
CN transform is a differentiable operation in deep neural networks that normalizes input data. By applying CN, the model can continuously learn from input distributions and adapt its representations to the target task, leading to improved performance. This normalization helps mitigate the inuence of variations in input distributions, allowing the model to focus on relevant patterns and features. The differentiability of CN enables efcient gradient ow during training, facilitating parameter updates and learning from the normalized data while preserving differentiation through the normalization process. Overall, CN plays a vital role in enhancing model performance by promoting effective learning and adaptability through data normalization. It demonstrates higher exibility compared to MN due to its ability to establish consistent data grouping based on provided contexts, without the need for additional algorithms. This is advantageous over MN since the Expectation-Maximization (EM) algorithm employed in MN can exhibit slower convergence. In the specic case of classifying dog images, where data scarcity is a challenge, the method addresses this issue by partitioning the dog class into subclasses. This approach enables the acquisition of specic features applicable to all dogs, facilitating the normalization of images within the dog superclass and creating a more coherent and easily learnable feature space. Importantly, the context identier used for learning the normalization parameters is unrelated to the images themselves. Instead, it can be viewed as noise, contributing to the regularization of the deep neural network during training, similar to techniques like dropout, thereby enhancing the generalization performance of the model [START_REF] Noh | Regularizing deep neural networks by noise: Its interpretation and optimization[END_REF].

Context Normalization Layer

Forward pass

Backward pass Fig. 2: Context Normalization Layer applied to a given activation x i . The context identier (r) is encoded by a neural network, the output of which is then used as input to two different neural networks to generate a mean (µ r ) and a standard deviation (σ r ), respectively, for normalizing x i .

After the model has converged, we enter a scenario similar to mixture normalization, where the parameters of each context are known and dependent on the target task. During the inference step, we have the option to normalize the data directly using CN or consider all the contexts collectively. This enhanced approach is referred to as CN+ (ref. Algorithm 3). For a given x i in B i and using the Equation 6, CN+ can be formulated as

xi = √ T T  r=1 τ r (x i )x r i , (9)
given

v r i = x i -E Bi [τ r (x).x], xr i = v r i  E Bi [τ r (x).(v r ) 2 ] + ϵ , where τr (x i ) = τ r (x i )  j∈Bi τ r (x j )
, under the assumption that prior probabilities (λ r = 1 T , r = 1, ..., T ) are constant;

IV. EXPERIMENTS

In the proposed experiments, we will assess the performance of the context normalization method and compare it to BN and MN. Detailed descriptions of the methods under consideration are provided in Table II.

CN-Patches can be used as layer on Transformer-based architecture [START_REF] Vaswani | Attention is all you need[END_REF] like Vision Transformer (ViT) [START_REF] Dosovitskiy | An image is worth 16x16 words: Transformers for image recognition at scale[END_REF]. ViT model takes as input a grid of non-overlapping contiguous image patches. CN-Patches consist of normalizing image patches with a vector µ and σ that are learned from the context identier embedding (ref. Figure CN-Channels is designed to be applied directly to images. The parameters µ and σ are vectors of size the number of channels. They are learned independently according to the context by using context identiers. CN-Channels incorporates the context identier into the image normalization process. In this case, the context identier embedding is used to directly adjust the image representation per channel. This allows the model to adapt the image representation to the context, which improves the overall performance of the model.

A. Datasets

The experiments in this study utilize several benchmark datasets commonly used in the classication community:

• CIFAR-10: A dataset with 50,000 training images and 10,000 test images, each of size 32×32 pixels, distributed across 10 classes [START_REF] Krizhevsky | CIFAR-10 (canadian institute for advanced research)[END_REF].

• CIFAR-100: Derived from the Tiny Images dataset, it consists of 50,000 training images and 10,000 test images of size 32 × 32, divided into 100 classes grouped into 20 superclasses [START_REF]CIFAR-100 (canadian institute for advanced research)[END_REF].

• MNIST digits: Contains 70,000 grayscale images of size 28 × 28 representing the 10 digits, with around 6,000 training images and 1,000 testing images per class [START_REF] Lecun | MNIST handwritten digit database[END_REF].

• VERI-Wild: Comprises 416,314 vehicle images with 40,671 identities, captured from different angles and at different times [START_REF] Lou | VERI-wild: A large dataset and a new method for vehicle re-identication in the wild[END_REF].

• SVHN: A challenging dataset with over 600,000 digit images, focusing on recognizing digits and numbers in natural scene images [START_REF] Sermanet | Convolutional neural networks applied to house numbers digit classication[END_REF]. Table III illustrates the mapping of each dataset to the corresponding experiments in which it is employed.

B. Context Normalization vs Mixture Normalization on CIFAR

In this experiment, we use a shallow convolutional neural network architecture as described in Table IV.

The mixture normalization approach replaces the BN layer in conv3 with mixture normalization layer, and the context normalization experiment uses CN-Channels as the rst CI-FAR ConvNet layer, incorporating context information directly into the base image. According to [START_REF] Kalayeh | Training faster by separating modes of variation in batch-normalized models[END_REF], we experiment with two different learning rates, one 5 times larger than the other. The same is done with weight decay. The mini-batch is set to 256, and all models are trained for 100 epochs using RMSprop [START_REF] Tieleman | Divide the gradient by a running average of its recent magnitude. coursera: Neural networks for machine learning[END_REF] with 0.9 momentum. During training on CIFAR-10 and CIFAR-100, we t a Gaussian mixture model by Maximum Likelihood Estimation (MLE). We use K-menas++ [START_REF] Arthur | k-means++: The advantages of careful seeding[END_REF] to initialize the centers of the mixture component and Expectation-Maximization (EM) [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] In the following experiments, we aim to demonstrate that context normalization can be implemented in a single step in specic scenarios, unlike mixture normalization. This approach would lead to a decrease in time complexity.

C. Context Normalization using superclass as context

As described in Section IV-A, the CIFAR-100 dataset incorporates a superclass structure in addition to the class partition. Our proposed context normalization leverages this superclass information as "prior knowledge" for classication. Each superclass corresponds to a context, identied by a onehot encoded vector of size 20 (the number of superclasses). We integrated the context normalization technique into the Vision Transformer (ViT) architecture, using Context Normalization on Patches (CN-Patches) and Context Normalization on Channels (CN-Channels). Training employed early stopping based on validation performance, and images were pre-processed by normalizing them with respect to the dataset's mean and standard deviation. Data augmentation techniques such as horizontal ipping and random cropping were applied to enhance the dataset. The AdamW optimizer with a learning rate of 10 -3 and weight decay of 10 -4 was chosen to prevent overtting and optimize model parameters [START_REF] Loshchilov | Decoupled weight decay regularization[END_REF], [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF].

Table VII demonstrates the signicant performance improvement of context normalization over batch normalization (BN) when using the ViT architecture trained from scratch on CIFAR-100. Both CN-Patches and CN-Channels approaches outperform BN by approximately 10% and 18% in terms of accuracy and top-5 accuracy. The train and test loss comparison in Figure 4 further supports this observation, showing that

D. Image classication on blended dataset

As deep neural networks require a certain amount of labeled data for effective training, it is well known that the lack of a large enough corpus of accurately labeled high-quality data can produce disappointing results. Data augmentation [START_REF] Perez | The effectiveness of data augmentation in image classication using deep learning[END_REF] is one way to overcome this problem. However, current approaches generate the data with a distribution that may be quite different from the original one, and this data bias will often lead to suboptimal performance [START_REF] Xu | Wemix: How to better utilize data augmentation[END_REF]. Inspired by the transfer learning [START_REF] Torrey | Handbook of research on machine learning applications and trends: algorithms, methods, and techniques[END_REF], we study in this subsection a new approach using the proposed context normalization. The goal of this approach is to improve the performance of the model on a target dataset by training the model on the combined dataset and then applying the trained model to the target dataset. In this framework, the proposed normalization technique allows to obtain a domain adaptation.

Specically, we train ViT models with the same settings as in Section IV-C with context normalization approaches (CN-Channels and CN-Patches) on the combined dataset CIFAR-100 and MNIST digits. We target two contexts r ∈ {1, 2}, corresponding to the datasets and the context identier is encoded by onehot(r = 1) = (1, 0) for images in CIFAR-100 and onehot(r = 2) = (0, 1) for images in MNIST digits.

The parameters µ and σ of each Gaussian distribution are then learned using these vectors after embedding. The trained models (on the blended dataset) are then applied to the CIFAR-100 test dataset.

It is important to mention that the baseline models (ViT with standard preprocessing and ViT with batch normalization) collapsed in this blended dataset as the two datasets have different structures, and simple normalization does not allow a suitable representation of the data. Context normalization, on the other hand, gives an adaptive representation per dataset (according to the contexts), which makes training possible. As shown in Table VIII, models with context normalization technique achieve good results on the blended dataset. It is also interesting to notice that this performance in terms of accuracy is biased by the MNIST digits dataset, which is less difcult to learn than CIFAR-100. More precisely, the model with CN-Patches achieves 55.04% accuracy and 81.83% top-5 accuracy, which exceeds the results of all baseline models (ref. The Latent space normalization (CN-Patches) has a better performance than the CN-Channels in this experiment. The drop in performance for the CN-Channels method can be explained by the fact that the normalization is applied directly to images located in a space with several different data characteristics.

E. Self-supervised learning for image similarity estimation

This experiment is motivated by the interpretability of context normalization parameters of each context obtained after training. To illustrate this, we construct the American night with context normalization on the VERI-Wild dataset. American night [START_REF] Haro | Visual acuity in day for night[END_REF] is a set of cinematic techniques used to simulate a night scene while lming in daylight. To accomplish this, we make a partition of two contexts on the dataset: r ∈ {Day, N ight}. A context vector onehot(r = Day) = (1, 0) is assigned to day images and onehot(r = N ight) = (0, 1) to night images. We label pairs of images containing the same object with 1, and otherwise with 0.

We use context normalization on the backbone of a siamese network (ViT architecture) with contrastive loss [START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF] to estimate the similarity between images. The aim of this experiment is to reveal the behavior of the parameters learned by the model and to understand how context information has an inuence on the normalization process. The use of a Siamese network makes it possible to measure the similarity between the images and to evaluate the efciency of the normalization in preserving the relevant information.

After training, we normalize day images with the appropriate parameters (µ Day and σ Day ), then denormalize (scale and shift) with the night parameters (µ N ight and σ N ight ), to construct night images, as shown in Figure 5. The images become darker, giving the night effect known as the American night (day for night) used for cinematic production in Hollywood, where a lter is applied to emphasize the light in the blue channel. A reverse process can be applied to night images to obtain day images, as shown in Figure 6, with brighter images reecting the daylight effect.

F. Context normalization in domain adaptation

In this experiment, we show that due to its strength in local representation, context normalization can yield substantial gains in domain adaptation. Domain adaptation [START_REF] Farahani | A brief review of domain adaptation[END_REF] is a technique for using knowledge learned by a model from a related domain with sufcient labeled data to improve the performance of the model in a target domain with insufcient labeled data. In this case, two contexts can be considered: r ∈ {source domain, target domain}. As an illustration, we use context normalization with AdaMatch [START_REF] Berthelot | Adamatch: A unied approach to semi-supervised learning and domain adaptation[END_REF], a method that Section IV-A). Both datasets have various varying factors in terms of texture, viewpoint, appearance, etc. Their domains, or distributions, are different from one another. As in [START_REF] Berthelot | Adamatch: A unied approach to semi-supervised learning and domain adaptation[END_REF], we use wide residual networks [START_REF] Zagoruyko | Wide residual networks[END_REF] for the dataset pairs. The model is trained using the Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] optimizer and a cosine decay schedule to reduce the initial learning rate, which is initialised at 0.03. Context normalization is used as the rst layer of AdaMatch to incorporate the context identier (source domain and target domain) into the image normalization process. In general, the clear improvement that context normalization brings in terms of validation can be seen in Tables IX andX. Normalizing with CN-Channels takes the target task into account. The source domain (MNIST) is labeled as opposed to the target domain (SVHN), which could explain why using MNIST as the context identier in AdaMatch+CN-Channels (ref. Tables IX and X) gives better results.

V. CONCLUSION

We have proposed a novel approach called "context normalization" (CN) that enhances deep neural network training in terms of training stability, fast convergence, higher learning rate, and viable activation functions. Similar to the conventional mixture normalization (MN) method, our approach is driven by the hypothesis that any continuous function can be approximated in some sense by a weighted sum of Gaussian distributions with nite mean vectors and covariance matrices. In other words, our methodology assumes that the data distribution is a mixture of Gaussian models. However, unlike the mixture normalization technique that invokes the expectation maximization (EM) algorithms to estimate the Gaussian components parameters, our proposed methodology relies on the notion of concept that represents a cluster of related data. In fact, a supervised deep neural network is built and trained in order to learn the Gaussian components parameters. Once these optimal values are determined after convergence, they are utilized during the CN procedure performed on a deep neural network activation layer. CN alleviates the slow estimation of Gaussian component parameters inherent to EM in the scenario of large datasets. Furthermore, unlike MN, CN provides non linear decision boundaries between context which reects more reality. Our experimental results demonstrate the superiority of context normalization over batch normalization and mixture normalization, showcasing enhanced convergence and generalization performance. The proposed method, when applied specically to images, introduces CN-Channels and CN-Patches for training, and CN and CN+ for inference. With its exibility to adapt various representations and tasks, context normalization proves to be a valuable tool in some application such as image classication.

Our short-term perspective consists in merging seamlessly a gradient free optimization algorithm with a gradient-based error optimizer in order to reach global convergence. We believe that this objective will boost training of deep neural networks further. This precision margin gained allows gaining insight into the neuron activation level sensitivity.

Fig. 1 :

 1 Fig. 1: A concise overview of the processing steps involved in Batch Normalization (BN), Mixture Normalization (MN), and Context Normalization (CN). The dashed line in the Batch Normalization diagram indicates a mini-batch parameter update, highlighting a key step in the process.

Algorithm 1 :

 1 CN-Context Normalizing Transform Input : Values of x over the mini-batch

Fig. 3 :

 3 Fig. 3: Comparative analysis of validation error curves for the CIFAR ConvNet architecture (ref.Table IV) trained under different learning rate and weight decay congurations.

Fig. 4 :

 4 Fig. 4: Validation loss curves on CIFAR-100 when the ViT architecture is trained with different normalization methods.

  Figure VII) trained on CIFAR-100. The model with CN-Channels gives 50.99% accuracy and 78.55% top-5 accuracy.

Fig. 5 :

 5 Fig. 5: Simulation of a night image on a day image: night image is obtained by normalizing day image with Day parameters, then scale and shift with Night parameters.

Fig. 6 :

 6 Fig. 6: Simulation of day image on night image: day image is obtained by normalizing night image with Night parameters, then scale and shift with Day parameters

Table I

 I for all sections.

	Variable	Denition
	T	number of context
	K	number of components
	N	mini-batch size
	C	channels
	H	height
	W	width
	N et	neural network
	Θ	trainable parameters of neural network
	x ∈ R N ×C×H×W	4-D activation tensor
	B = {x 1:m }	attened x across axis N , H and W
	B i	attened x across axis H and W
	µr	the mean on the context r
	σr	standard deviation on the context r
	onehot(.)	One-Hot Encoding function

TABLE I :

 I Table of notations II. RELATED WORK A. Batch Normalization Let x ∈ R N ×C×H×W denote the activation for a given neuron in a layer of a convolutional neural network where N , C, H and W are respectively the batch, channel, height, and width axes. Batch normalization (BN) [4] standardizes with m samples mini-batch B = {x 1:m

  Algorithm 3: CN-Inference (Context-Normalized Deep Neural Network) Input : Deep Neural Network N et tr CN with trainable parameters Θ ∪ {µ r , σ r } T r=1 (ref. Inference CN deep neural network //with frozen parameters if choice = CN then for i ← 1 to m do Normalize using context(r) parameters (µ r and σ r learned during training) associated to x i . xi ← CN(x i , r) end end if choice = CN+ then for i ← 1 to m do

	Algorithm 2); mini-batch of activation
	{x i } m i=1 ; choice ∈ {CN, CN+} Output: Context-Normalized deep neural network for inference, N et inf CN N et inf CN ← N et tr CN //

• Normalize x i using Equation 9 // Normalization + aggregation end end

  2 and Algorithm 1). Images are grouped by context (each context is a component of the mixture of Gaussian distributions), and for each context, a unique vector (context identier) is associated. The context

	Model	Description
	BN	model with batch normalizing transform (ref. Sec-
		tion II-A)
	MN	model with mixture normalizing transform (ref.
		Section II-B)
	CN	Context Normalization model which uses CN-
		Training (ref. Algorithm 2) in the training step and
		CN method (ref. Algorithm 3) in the inference step.
		Two versions of CN are used in the experiments:
		CN on Patches (CN-Patches) and CN on Channels
		(CN-Channels).
	CN+	Context Normalization model which uses CN-
		Training (ref.Algorithm 2) in training step and
		CN+ method (ref. Algorithm 3) in inference step

TABLE II :

 II Normalization method used in the experiments identier is rst embedded for a better representation, and the resulting vector is used to learn the corresponding parameters µ and σ independently. This process is repeated for each input (image) to the neural network during training. CN-Patches is a way of integrating the context information into the image patch normalization process. It allows the image patch representation to be adapted to the context of the image, which improves the overall performance of the model.

  to estimate

	Dataset	Task
	CIFAR-10	This dataset is used for training CIFAR Con-
		vNet (ref. Table IV), ensuring consistent conditions
		with the study on mixture normalization (ref. Sec-
		tion IV-B).
	CIFAR-100	Three experiments (ref. Sections IV-B, IV-C,
		and IV-D) use this dataset: comparing with mixture
		normalization methods, utilizing superclass struc-
		ture as context, and blending with MNIST digits
		for domain adaptation.
	MNIST digits	MNIST digits is employed in two domain adap-
		tation experiments, serving as the blended dataset
		with CIFAR-100 in the rst (ref. Section IV-D)
		and as the source domain in the second (ref.
		Section IV-F).
	VERI-Wild	This dataset facilitates the analysis of context pa-
		rameters obtained through model training using
		an image similarity measurement strategy (ref.
		Section IV-E), thereby constructing the American
		night.
	SVHN	SVHN is used in conjunction with MNIST dig-
		its as the target dataset to train the AdaMatch
		model in unsupervised domain adaptation (ref.
		Section IV-F).

TABLE III :

 III Summary of datasets and associated training experiments

	layer	type	size	kernel	(stride, pad)
	input	input	3 × 32 × 32
	conv1	conv+bn+relu	64 × 32 × 32	5 × 5	(1, 2)
	pool1	max pool	64 × 16 × 16	3 × 3	(2, 0)
	conv2	conv+bn+relu	128 × 16 × 16	5 × 5	(1, 2)
	pool2	max pool	128 × 8 × 8	3 × 3	(2, 0)
	conv3	conv+bn+relu	128 × 8 × 8	5 × 5	(1, 2)
	pool3	max pool	128 × 4 × 4	3 × 3	(2, 0)
	conv4	conv+bn+relu	256 × 4 × 4	3 × 3	(1, 1)
	pool4	avg pool	256 × 1 × 1	4 × 4	(1, 0)
	linear	linear	10 or 100

TABLE IV :

 IV In mixture normalization, each sample is normalized using the mean and standard deviation of the mixture component to which it belongs. Context normalization treats each Gaussian component as an individual context, enabling sample normalization within each component. It employs separate multi-layer perceptrons (MLPs) to estimate the mean and standard deviation for each context using the one-hot encoded context identier (onehot(.)).Our primary objective is not to achieve state-of-the-art results, which require computationally expensive architectures and careful parameter tuning. Instead, we aim to demonstrate that by replacing or incorporating our context normalization technique, the convergence rate can be improved, leading to superior test accuracy. This showcases the signicant impact of our approach in enhancing model performance. Increasing the learning rate from 0.001 to 0.005 increases the convergence gap, showing the ability of context normalization as mixture normalization to take advantage of higher learning rates for training. From Table V and Figure3we can see that regardless of the weight decay and learning rates, CN models not only

		CIFAR-10
	model	(lr, weight decay)	test accuracy (%)
	BN-1	(0.001, 1e-4)	86.9
	BN-2	(0.001, 2e-5)	85.9
	BN-3	(0.005, 1e-4)	82.56
	BN-4	(0.005, 2e-5)	83.71
	MN-1	(0.001, 1e-4)	87.08
	MN-2	(0.001, 2e-5)	87.9
	MN-3	(0.005, 1e-4)	82.07
	MN-4	(0.005, 2e-5)	84.71
	CN-1	(0.001, 1e-4)	87.32
	CN-2	(0.001, 2e-5)	88.15
	CN-3	(0.005, 1e-4)	83.85
	CN-4	(0.005, 2e-5)	87.09
		CIFAR-100
	model	(lr, weight decay)	test accuracy (%)
	BN-1	(0.001, 1e-4)	57.48
	BN-2	(0.001, 2e-5)	57.69
	BN-3	(0.005, 1e-4)	50.82
	BN-4	(0.005, 2e-5)	52.25
	MN-1	(0.001, 1e-4)	61.6
	MN-2	(0.001, 2e-5)	61.9
	MN-3	(0.005, 1e-4)	53.08
	MN-4	(0.005, 2e-5)	52.7
	CN-1	(0.001, 1e-4)	61.79
	CN-2	(0.001, 2e-5)	60.53
	CN-3	(0.005, 1e-4)	56.08
	CN-4	(0.005, 2e-5)	53.54

CIFAR ConvNet architecture on mixture normalization paper for a comparison of MN and BN for small and large learning rate regimes the parameters of the mixture model θ = {λ k , µ k , Σ k : k = 1, ..., K = 3}.

TABLE V :

 V

Performance Evaluation on CIFAR-10 and CIFAR-100 using the CIFAR ConvNet architecture (ref. Table IV) with the incorporation of Batch Normalization (BN), Mixture Normalization (MN), and Context Normalization (CN). Each algorithm is followed by a corresponding number, representing the learning rate and weight decay.

  Table IV) trained under different learning rate and weight decay congurations. converge much faster than their corresponding BN and MN counterparts but also achieve better accuracy. After model training, each context's parameters are known. The use of CN+ in the inference step leads to the results shown in Table VI. The CN method (ref. Table V) is faster than the

		CIFAR-10
	model	(lr, weight decay)	test accuracy (%)
	CN+1	(0.001, 1e-4)	87.90
	CN+2	(0.001, 2e-5)	88.28
	CN+3	(0.005, 1e-4)	82.85
	CN+4	(0.005, 2e-5)	87.50
		CIFAR-100
	model	(lr, weight decay)	test accuracy (%)
	CN+1	(0.01, 1e-4)	62.01
	CN+2	(0.001, 2e-5)	60.53
	CN+3	(0.005, 1e-4)	55.80
	CN+4	(0.005, 2e-5)	53.15

TABLE VI :

 VI Performance Evaluation on CIFAR-10 and CIFAR-100 using CIFAR ConvNet architecture (ref. Table IV). Introducing CN+ which is an extended version of CN for Inference, encompassing all contexts as Mixture Normalization. Each CN+ variant is denoted by a number, representing the iterative exploration of learning rate and weight decay adjustments.

CN+ method (ref. Table VI) and gives approximately the same results. In the following experiments, we use the CN method in the inference step (ref. Algorithm 3).

TABLE VII

 VII

	: Comparison of the two Context Normalization
	methods on CIFAR-100: Context Normalization on Patches
	(CN-Patches) and Context Normalization on Channels (CN-
	Channels), with normalization to the mean and standard devi-
	ation of the dataset (ViT) and input normalization using batch
	normalization (BN).
	CN-Patches and CN-Channels accelerate the learning process
	and enhance classication performance. These results indicate
	that the proposed approaches stabilize the distribution and
	mitigate internal covariate shift, resulting in a superior data
	representation aligned with the target task.

TABLE VIII :

 VIII Blended dataset CIFAR-100 and MNIST digits: results of models based on the two normalization methods (CN-Patches and CN-Channels). Baseline models collapsed in this combined dataset.

TABLE IX :

 IX Test accuracy of AdaMatch and AdaMatch with context normalization (AdaMatch+CN-Channels) using source domain (MNIST) as context identier.

	model	source data (MNIST)	target data (SVHN)
	AdaMatch	79.39%	20.46%
	AdaMatch+CN-Channels	94.45%	23.22%

TABLE X :

 X Test accuracy of AdaMatch and AdaMatch with context normalization (AdaMatch+CN-Channels) using target domain (SVHN) as context identier.