Arnaud Gissot

Khalid Oumzil

Amit Patwa

Philippe Barthélémy

Hybrid Lipid Oligonucleotide: a versatile tool for supramolecular chemistry

Lipid Oligonucleotides (LONs) are emerging as promising supramolecular tools for biomedical and technological applications. In this contribution we highlight recent advances in the area of LONs with an emphasis on their supramolecular properties and applications. In the first section we focus on the design, self-assemblies and applications of LONs. In the last section, we describe recent biomedical applications thereof.

Introduction

Nucleic acids and lipids are central molecules in living organisms. While the former is responsible for the storing and propagation of genetic information, the latter allow for the protection and compartmentalization of the living matter from the external environment. The chemical combination of a lipid with an oligonucleotide, the so-called Lipid OligoNucleotides (LONs), have been realized with the aim of combining and optimizing the properties of both biomolecules. 1 In fact, the propensity of the lipidic segment of the LONs to aggregate in water brings about new self-assembling opportunities to these bio-inspired amphiphiles. LONs can therefore self-assemble into micelles, 2 liposomes, 3 and nanoparticles. This phenomenon is expected to positively impact the biological effect of the nucleic acid segment of the LON. These different fields have already been covered by us or others in the past. 1,4,5 This contribution will therefore mainly focus on the recent advances with an emphasis on supramolecular and biomedical applications.

Lipid-oligonucleotide conjugates, from molecular to supramolecular self-assemblies

Many biomolecules have been covalently attached to nucleic acid sequences to impart new properties to the oligonucleotides e.g. targeting properties (RGD peptides, carbohydrates), higher affinities for their complementary sequence (oligospermine in zip nucleic acids), 6 or cell membrane crossing capabilities with lipid conjugates. Recent developments in the field of oligonucleotide conjugates have been reviewed lately. [7][8][9] Lipid conjugation to ONs was primarily thought of as a mean to anchor the ON to cell membranes, ultimately leading to the internalization of the ON sequence. Many different lipidantisense ON have consequently been synthesized in the past. Basically there are three different positions for lipid conjugation to ONs: (i) 3-terminal, (ii) 5-terminal and (iii) intra-chain position. At each location, the lipid conjugates can be incorporated at various sites i.e. at the phosphate backbone, at the sugar (at 2-, 3-, 5-positions, etc.) and/or at the nucleobase. The specific site(s) of lipid conjugation has/have an impact on the biological and physical properties of the LONs. Therefore, depending on its application, the conjugation site(s) has/have to be carefully chosen so that it facilitates the function of LONs without compromising its binding properties (usually hybridization with the complementary ONs). The synthetic methods that have been used for the preparation of LONs have been thoroughly reviewed and will not be covered here. 1,5,[START_REF] Patwa | DNA in Supramolecular Chemistry and Nanotechnology[END_REF]

Supramolecular assemblies

The presence of a lipid in ON sequences improves their solubility in organic solvents. If only one lipid does not seem to be sufficient to provide full solubility, a TG4T DNA sequence with dodecyl chains at each phosphotriester linkage is soluble in chloroform or THF. [START_REF] Shibata | [END_REF] Interestingly, this minimal sequence prone to form G4-quadruplexe in water kept the ability to form this secondary structure motif and extract alkali metals in chloroform or THF. Wang et al. took profit of the higher tolerance of LONs toward organic co-solvents to play around the solvent composition to trigger the switch between different supramolecular architectures. 12 While micellar aggregates are formed in the presence of THF, their hybrid DNA-dendron LON formed nanofibers after gently evaporating THF followed by slow cooling. The authors hypothesized that the organic cosolvent eliminated  stacking interactions important for the integrity of the nanofibers. Nevertheless, ONs and LONs are overwhelmingly used in water. Here, the lipidic tether brings about amphiphilic character to the ON. Yet, unlike readily available conventional amphiphiles whose self-aggregation properties have been extensively studied over the years, little is still known about the nature, the thermodynamic and kinetic parameters of LON aggregates. In fact, the solid phase synthesis of ON is the method of choice for the production of LONs (and more generally ONs). ON syntheses are conveniently carried out at the 0.2 up to a few dozen mole scale at best at the laboratory level. These quantities clearly limit the wealth of physicochemical experiences that can be carried out with LONs. As an example, parameters as basic as the critical aggregation concentrations (cac) of LONs are rare in the literature. Besides, these values are usually obtained with hydrophobic fluorescence probes, which can sense polarity change in their environment. With the notable exceptions of LONs with either a cholesterol, or short consecutive alkyl chains at the internucleosidic phosphorus, 13 which self-assemble at millimolar or sub-millimolar concentrations, most of the other LONs cac fall in the nanomolar range. Of note, the ratio of the probe molecules over the LON in the aggregates in such diluted solutions may somehow affect the value of the cac. Alternative techniques would therefore be highly desirable to confirm and shed light on the origin of these low cac numbers. Nevertheless, these investigations are further hampered by the tendency of the LONs to show complex aggregation patterns and kinetics. 14 Recently, Langmuir-Blodgett films were prepared from the 11mer LON below (Figure 1). No stable monolayer at the air/water interface was observed probably because of the too short alkyl chains present in the structure of the LON. 15 The LON molecules started to interact at a surface area of 1390Ų per polar head somewhat ten times larger than traditional phospholipids. This phenomenon was ascribed to the much larger ON polar head. Furthermore, the compression and expansion of the film showed a hysteresis profile: reorganization at the air/water interface took place upon compression with the formation of multilayer structures with the LON. These later structures were indeed observed by atomic force microscopy. The size of the oligonucleotide segment of the LON is one of the important parameters that governs the type of self-assembly formed. This has been investigated in some details by the group of Gianneschi. They were able to reversibly control the nature of the LON aggregates depending on the ON length. 3,16 Shorter ON sequences of the LON favored vesicular aggregates. The latter were reversibly converted into micelles with longer ONs. The presence (or absence) of a spacer between the ON sequence and the lipidic segment of the LON was also shown to impact the morphology of the self-assembly to some extent, 17 as well as the in vitro efficacy of si-RNA conjugates. 18 On the other hand, the influence of other important parameters on the cac like (1) the base composition, (2) the presence of possible secondary structures in the ON, and (3) the ionic strength has not been investigated so far. Indeed, secondary and to a lesser extent tertiary structures are possible in ON in general and always found in the structures of aptamers. Aptamers are small single stranded RNA or DNA molecules that can specifically recognize virtually any desired target. As such, they constitute a chemical equivalent of antibodies. The group of Tan combined the exquisite recognition specificity of aptamers with the self-aggregation/internatinalization properties of lipids to develop a LON aptamer capable of sensing ATP in living cells. 19 Interestingly, these constructs, which showed no toxicity, were present as discrete molecules throughout the cytoplasm after self-internalization of the LON micelles. This greatly favored the detection of ATP in the cytoplasm. Although not investigated in details in this article, the selfaggregation properties of the LONs may be at least in part responsible for the success of this design. Oligomerization of the aptamer molecules is indeed brought about by the micellar architecture. This setting is expected to lower the koff and then prevent the dissociation of the target ATP from the complex.

With an estimated 100 copies of aptamer molecules per micelle, the LON developed by Tan and coll. showed greatly enhanced recognition properties over the unmodified aptamer: the micelle aptamer still binds the targeted cells in the presence of blood, or at 37°C where the soft nature of the micelle proved far superior to the more conventional "hard" nanoparticle conjugates like gold or carbon nanotubes. 2 Besides the "oligomerization" of the ON sequences that takes place at the surface of LON micelles, the sharing of the ions cloud between adjacent ONs, the so-called proximity effect, 20 may also favor the affinity of the aptamer micelles for its target. Cations are actually required in the Stern layer (and above) around the nanoparticle to compensate for the anionic charges present in the ON sequences at the micellar surface. This phenomenon is entropically favorable as fewer cations are required for the neutralization of the same number of negative charges and/or the stabilization of secondary/tertiary ON structures. Similarly, the correct positioning of dodecyl alkyl chains in a DNA duplex was found to markedly increase the temperature of fusion of the duplex. 21 ITC experiments showed that the favorable free energy of association between the 2 strands was primarily entropically driven as expected with a favorable hydrophobic effect. LONs have also found applications in the field of DNA origami. The lipid segments of the LONs can be precisely positioned in these exquisite self-assembling 3-D DNA architectures. It was for instance possible to program the formation of discrete dimers of DNA-cubes through the specific aggregation of 2 DNA-cubes decorated with 4 LON molecules on one of their 6 faces. 22 Interestingly the aggregation of the cubes took place at LON concentrations below their critical micellar concentration (cmc). Aggregates of LONs constitutes an ideal delivery vehicle for drug molecules in vivo: they are non toxic, they self-assemble into micellar structures which are dynamic and capable of hosting hydrophobic guests and the ON sequence can be used as a targeting agent and/or as a signal to trigger the release of the drug molecules. Our group took profit of these different properties to develop a LON-based delivery vehicle for paclitaxel molecules. 23 Interestingly, the drug loading capabilities of the LON aggregates were found to depend not only on the nature of the lipid but also quite unexpectedly on the oligonucleotide sequence of the LON. Furthermore, binding of the complementary DNA to the LON aggregates led to the contraction of the micelles with the concomitant release of part of the drug molecules. Provided the ON sequence is found in the extracellular environment of the tumor cell, this platform could eventually find applications in the spatially controlled delivery of lipophilic drugs into malignant cells. In a similar way, LON micelles where shown to induce capsid formation from the cowpea chlorotic mottle virus. 24 The 28-nm in length icosahedral virus particles were efficiently filled up with the LON micelles whose core can accommodate extra hydrophobic molecules like pyrene or others. These genomefree virus-like particles are very interesting in the context of the specific delivery of drug molecules to cells usually targeted by the virus.

LONs incorporation into membranes

If biotechnological applications of self-assembled LONs are rare in the literature due to their poor availability, they have found many interesting applications as tagging reagents for membranes, mostly self-assembled monolayers (SAM) or vesicles. In fact, DNA chips usually capitalize on "hard" covalent attachment of the ON to solid surfaces. In contrast, soft materials are dynamic in nature. [START_REF] Opdahl | Proc. Natl. Acad. Sci[END_REF][START_REF] Kwiat | [END_REF] They allow ON capture probes to rearrange and give homogeneous kinetics and thermodynamics upon annealing the target ON. [START_REF] Wu | Proc. Natl. Acad. Sci[END_REF] This property was for instance utilized for the reversible assembly of stacked nanodiscs decorated with LONs and with interdisc distances controlled by the length of the ON. [START_REF] Beales | [END_REF] Yet, one has to bear in mind that the adsorption of lipids into artificial or natural membranes is nothing but aspecific. Most of the lipidic membranes are heterogeneous in their lipidic content and distribution. Each lipid will therefore partition preferentially or specifically and with different kinetics into designated domains or rafts in the membrane. 29 This has truly to be considered when LONs are used in a context of living cells. 30,[START_REF] Gartner | Proc. Natl. Acad. Sci[END_REF] In addition, mammalian cells exhibit anionic sialic acid and glycosaminoglycans at their surface which were shown to disfavor LON partitioning due to repulsive electrostatic interactions. [START_REF] Palte | [END_REF] As expected based on electrostatic considerations, shorter ON sequences were more favorably anchored in the membranes. Longer sequences have also a detrimental impact on the kinetics of fusion of the LON micelles with membranes, probably because of a greater distance between the lipids of the membrane and those of the micelles. 2 Finally, LON self-assemble into aggregates whose dynamics have been poorly investigated. LON aggregates may well be kinetically and/or thermodynamically trapped with no possibility to partition into membranes. Clearly, the impact of the ON sequences in the free energy diagram of the LON aggregates and the residence time of each monomer in the aggregate are key elements that will necessitate deeper investigations in the future. The reader is referred to two recent review articles for other examples of surface functionalization with LONs. 33,34

Applications of LONs

The group of Herrmann have designed a block copolymer LON: PFO-b-DNA (PFO = poly(9,9-di-n-octylfluorenyl-2,7diyl)) as a dispersing agent for single-wall Nanotubes (SWNT). This dispersant proved superior to other surfactants commonly employed. 35 The semiconducting SWNT were selectively solubilized as individual entities from the crude nanotubes extract. Moreover, SWNT aqueous suspensions remained stable for more than 3 months. Very interestingly, these LONsolubilized semiconducting SWNT were used in the manufacture of a field-effect transistor by a pure self-assembly process controlled by the ON sequence in contrast to the tedious positioning of individual SWNT usually required in the manufacture of carbon nanotube electronics. The ability of LONs to pair with complementary ONs and to be stably inserted into lipidic membranes has allowed their use as platforms for the detection of nucleic acids. Ideally, a powerful nucleic acid detection format should be able to directly detect sub-picomolar concentrations of DNA or RNA. As a result, nucleic acids are usually detected after some kind of preliminary amplification step (PCR or other), which is usually costly, time-consuming and certain sequences are more or less prone to being amplified by PCR. Signal amplification by any means is therefore necessary to avoid this amplification step and still detect low copy DNA or RNA molecules. The group of Höök developed an elegant strategy wherein LONs are inserted into vesicles with different lipid compositions. Provided the target ON sequence is present, it is sandwiched between the glass substrate and the vesicle decorated with the complementary sequence. 36 Each single molecule target (A or B in Figure 2) binding event will translate into a huge number of co-lipid molecules (POPC or DPPC) immobilized on the surface thus amplifying the signal that can be detected by time of flight (TOF) secondary ion mass spectrometry (MS). Consequently, the MS signal for each co-lipid of the different vesicles is proportional to the concentration of each ON target sequence. Based on a similar approach wherein the liposomes were loaded with a fluorescent dye, the same group was able to follow in real time the residence time of the vesicles once they are immobilized on the surface using total internal reflection fluorescence spectroscopy (TIRF). 37 The immobilization of each single vesicle mediated by DNA duplex formation could be translated not only into the dissociation constant of the vesicle binding to the solid substrate but also into kon and koff. In fact, these kinetic parameters are far superior at discriminating between mutated DNA sequences compared to end-point thermodynamic measurements thus demonstrating the utility of this design for the detection of single point mutations. Not only LONs are useful at detecting nucleic acid sequences, they can also serve as phase transfer agent for the purification of the complementary nucleic acid sequence. 38 LON molecules reversibly partition between the water/organic solvent interface and reverse-micelles. Duplex formation between the LON and the water-soluble complementary DNA takes place at the interface. The targeted ON can therefore be extracted from water to the organic phase in a sequence-specific manner after the LONs form reverse-micelles in the organic phase. Restoring the correct gene expression of cells by means of the administration of synthetic oligonucleotides constitutes the basis of gene therapy. Several ON-based drugs are now in clinical trials or already marketed. For example, therapeutic progress has been achieved lately for patients with metastatic castration-resistant prostate cancer (mCRPC). Custirsen (OGX-011) is a second-generation 2'-methoxyethyl modified phosphorothioate antisense ON currently in phase III trials that inhibits expression of clusterin a cytoprotective chaperone protein that is upregulated in response to various anticancer therapies. Overexpression of clusterin inhibits apoptosis thus favouring malignant cell resistance to chemotherapies. 39 MicroRNAs are natural genetic regulators of numerous cellular pathways and are involved in many diseases, including cancers. Several members of this class of small nucleic acid are currently emerging as new promising therapeutic targets and/or potentially relevant biomarkers. 40 In that context, the LON-Quantum Dot (QD) nanoplatform developed recently in our lab offers new opportunities as a dual diagnosis/therapeutic tool. 41 Quantum dots were stably encapsulated with anti-miR21 LON molecules (Figure 4). miR-21 is a miRNA that is overexpressed in cancer cells. Surface plasmon resonance (SPR) studies indicate that just like in liposomes, 42 the kinetics and thermodynamics of duplex formation at the surface of the QDs remain essentially unaffected compared to the same duplex free in solution. The LON-QDs were found to hybridize exclusively with microRNA-21. This resulted in the down-regulation (30%) of miR-21 expression in the absence of any tranfecting reagents. As an alternative to LONs, fluorocarbon oligonucleotide conjugates (FONs) with either C8F17 or C6F13 hydrophobic and lipophobic chains were synthesized using a ''post synthetic'' grafting of the fluorocarbon moieties. 43 Interestingly, it was demonstrated that these conjugates allows the cellular uptake of the ONs in different human cell lines, including hepatic Huh7, gastric epithelial NCI-N87 and embryonic kidney HEK293. These results suggest that FON aggregates may be used as a new tool for the delivery of oligonucleotides for intracellular targets, such as antisense sequences, aptamers or siRNA. Other examples of LON-based biomedical applications can be found in the literature. [START_REF] Patwa | DNA in Supramolecular Chemistry and Nanotechnology[END_REF]

Conclusion

The numerous recent publications in the domain of hybrid lipid oligonucleotides indicate that these amphiphiles are promising tools in supramolecular chemistry as well as in biomedical applications. As witnessed lately by the burgeoning of new lipid oligonucleotide-based therapies targeting many different diseases, it is unquestionable that biomedical and biotechnological applications using these conjugates should greatly take advantage of the deeper understanding of their complex supramolecular chemistry. Consequently we are confident that this field will greatly evolve in the near future.

Figure 1 .

 1 Figure 1. Example of a LON with short alkyl chains covalently linked to the nucleobases.

Figure 3 .

 3 Figure 3. LONs (in green) associated with target ON (in red) can serve as phase transfer agent for the purification of the complementary nucleic acid sequence (adapted from reference 38).

Figure 2 .

 2 Figure 2. Example of signal amplification using vesicles encoded with LONs. In the presence of the complementary DNA targets (A or B) the vesicles are immobilized on the surface. This event allows an amplification of the number of secondary ions which are detected by mass spectrometry. (adapted from reference 36).

Figure 4 .

 4 Figure 4. Up, schematic representation of a LON (in green) inserted in a membrane (in grey, DOPC or PUPC) that encapsulates the QD solid core (in red). Down, expression levels of miR21 in cancer cells incubated in different conditions, including control experiments. The platform inhibits the expression of miR21 (lines in red) of cancer cells in the absence of a transfecting reagent (lipofectamine) (adapted from reference 41).

Acknowledgements

P.B. acknowledges financial support from the Army Research

Office. This work has been supported by the French National Agency (ANR) in the frame of its programme Emergence (project TCTP-LASOCaP, ANR-11-EMMA-0022).

Notes and references

a INSERM U869, Bordeaux, F-33076, France. b Université de Bordeaux, Bordeaux, F-33076, France.


Titouan Parcollet

Mohamed Morchid

Georges Linarès

E2E-SINCNET: TOWARD FULLY END-TO-END SPEECH RECOGNITION

Keywords:

Modern end-to-end (E2E) Automatic Speech Recognition (ASR) systems rely on Deep Neural Networks (DNN) that are mostly trained on handcrafted and pre-computed acoustic features such as Mel-filter-banks or Mel-frequency cepstral coefficients. Nonetheless, and despite worse performances, E2E ASR models processing raw waveforms are an active research field due to the lossless nature of the input signal. In this paper, we propose the E2E-SincNet, a novel fully E2E ASR model that goes from the raw waveform to the text transcripts by merging two recent and powerful paradigms: SincNet and the joint CTC-attention training scheme. The conducted experiments on two different speech recognition tasks show that our approach outperforms previously investigated E2E systems relying either on the raw waveform or pre-computed acoustic features, with a reported top-of-the-line Word Error Rate (WER) of 4.7% on the Wall Street Journal (WSJ) dataset.

INTRODUCTION

ASR systems are either hybrid DNN-HMM or end-to-end (E2E). The former set of ASR models provides state-of-the-art performances on numerous speech-related real-world tasks [START_REF] Povey | The kaldi speech recognition toolkit[END_REF][START_REF] Ravanelli | The pytorch-kaldi speech recognition toolkit[END_REF] but involves multiple sub-blocks trained separately, and often requires separate and a strong human expertise. E2E systems, on the other hand, propose to directly transcribe a sequence of acoustic input features [START_REF] Graves | Towards end-to-end speech recognition with recurrent neural networks[END_REF][START_REF] Zhang | Towards end-to-end speech recognition with deep convolutional neural networks[END_REF] with a single model usually composed of different Neural Networks (NN) trained jointly in an end-to-end manner. In particular, a major challenge is to automatically generate an alignment from the raw signal that often contains several thousands of data point per second, to the text, only consisting of a single character or concept in the same time scale.

Recently, E2E approaches started to outperform traditional DNN-HMM baselines on common speech recognition tasks with the introduction of more efficient sequence training objectives [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF][START_REF] Wang | Espresso: A fast endto-end neural speech recognition toolkit[END_REF], more powerful architectures [START_REF] Zeghidour | End-to-end speech recognition from the raw waveform[END_REF][START_REF] Li | Jasper: An end-to-end convolutional neural acoustic model[END_REF] and attention mechanisms [START_REF] Bahdanau | End-to-end attention-based large vocabulary speech recognition[END_REF][START_REF] Hori | Advances in joint ctc-attention based end-to-end speech recognition with a deep cnn encoder and rnn-lm[END_REF][START_REF] Dong | Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition[END_REF][START_REF] Salazar | Selfattention networks for connectionist temporal classification in speech recognition[END_REF]. Despite being named "E2E", the latter models still require pre-processed acoustic features such as Mel-filter-banks, alleviating a pure E2E pipeline based on the raw audio signal.

Processing raw waveforms in the specific context of ASR is an active challenge [START_REF] Palaz | End-to-end phoneme sequence recognition using convolutional neural networks[END_REF][START_REF] Tüske | Acoustic modeling with deep neural networks using raw time signal for lvcsr[END_REF][START_REF] Hoshen | Speech acoustic modeling from raw multichannel waveforms[END_REF][START_REF] Zeghidour | End-to-end speech recognition from the raw waveform[END_REF]. Most of these works rely on modified Convolutional Neural Networks (CNNs) to operate over the signal. As an example, in [START_REF] Zeghidour | End-to-end speech recognition from the raw waveform[END_REF], the authors propose to combine a log non-linearity with a CNN architecture that exactly matches an output dimension equivalent to standard Mel-filter-banks features, forcing the input layer to learn the latter signal transformation. Nevertheless, and as demonstrated in [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF], CNNs are not efficient at learning common acoustic features due to the lack of constraint on the numerous trainable parameters. Consequently, the authors proposed SincNet, a specific convolutional layer that integrates common acoustic filters, such as band-pass filters, to replace the convolutional kernel weights drastically reducing the number of parameters. Furthermore, it is demonstrated that the learned filters have a much better frequency response than those learned with traditional CNNs, resulting in better performances in a speaker recognition task. Then, SincNet has been combined with a straightforward fully-connected DNN in the context of a DNN-HMM ASR system also outperforming CNNs trained with both pre-computed acoustic features and raw waveforms [START_REF] Ravanelli | Speech and speaker recognition from raw waveform with sincnet[END_REF].

Unfortunately, there is no available model combining both the efficacy of SincNet to operate over raw signals, and the latest training scheme for E2E systems. Therefore, we propose to bridge this gap by investigating and releasing1 a fully E2E model, named E2E-SincNet, combining SincNet with the joint CTC-attention training scheme [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF] and resulting in a customizable, efficient and interpretable E2E ASR system. Contributions of the paper are summarized as:

1. Enhance the original SincNet to fit bi-directional recurrent neural networks (RNN).

2. Merge the later model with the joint CTC-attention method [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF] to create E2E-SincNet 1 based on the well-known ESPnet toolkit [START_REF] Watanabe | Espnet: End-to-end speech processing toolkit[END_REF] (Section 2).

3. Evaluate the model alongside with other baseline models on the WSJ and TIMIT speech recognition tasks (Section 3).

The conducted experiments show that E2E-SincNet obtains superior and state-of-the-art (SOTA) performances to both traditional E2E models operating on raw waveform with CNNs, and SOTA E2E architectures relying on pre-computed acoustic features.

END-TO-END SPEECH RECOGNITION

This section introduces the necessary building blocks to conceive a fully E2E automatic speech recognition system. First, latent acoustic features are extracted from the raw waveform signal with a specific kernelized CNN, also known as SincNet [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF] (Section 2.1). The latter model is then merged with a joint CTC-attention [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF] training procedure (Section 2.2.1), based on an encoder-decoder architecture [START_REF] Bahdanau | End-to-end attention-based large vocabulary speech recognition[END_REF] (Section 2.2.2).

Processing raw waveforms with SincNet

Traditional parametric CNNs operate over the raw waveform by performing multiple time-domain convolutions between the input signal and a certain finite impulse response [START_REF] Lawrence | Theory and applications of digital speech processing[END_REF] as:

y[n] = x[n] × f [n] = l=0 L x[l].f [n -l], (1)
with x[n] a part of the speech signal, f [n] a filter of length L, and y[n] the output finally filtered. In this case, all the elements of f are learnable parameters. SincNet proposes to replace f with a predefined function g that only depends on much fewer parameters to describe its behavior. In [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF], the authors implemented g as a filterbank composed of rectangular bandpass filters. Such function can be written in the time domain as:

g[n, f1, f2] = 2f2sinc(2πf2n) -2f1sinc(2πf1n), (2)
with f1 and f2 the two learnable parameters that describe the low and high cutoff frequencies of the bandpass filters, and sinc(x) = sin(x) x

. Such parameters are randomly initialized in the interval [0, fs 2 ], with fs equal to the input signal frequency sampling. It is also important to notice that g is smoothed based on the Hamming window [START_REF] Kumar | Digital signal processing: a computer-based approach[END_REF].

Other definitions of the filter g have been proposed including triangular, Gammatone, and Gaussian filters [START_REF] Loweimi | On learning interpretable cnns with parametric modulated kernel-based filters[END_REF] demonstrating superior performances over Eq. 2 due to better filter responses to the signal. As a matter of fact, SincNet allows an important flexibility to efficiently enhance traditional acoustic-based CNNs with prior and well-investigated knowledge. Finally, SincNet filters are facilitating the interpretability of the model by being easily extracted and applied over any signal for further investigations of the transformations [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF][START_REF] Ravanelli | Interpretable convolutional filters with sincnet[END_REF][START_REF] Loweimi | On learning interpretable cnns with parametric modulated kernel-based filters[END_REF].

Unfortunately, SincNet has only been investigated with a mere fully-connected DNN based on a hybdrid DNN-HMM setup [START_REF] Ravanelli | Speech and speaker recognition from raw waveform with sincnet[END_REF][START_REF] Loweimi | On learning interpretable cnns with parametric modulated kernel-based filters[END_REF]. We propose to connect SincNet to a recurrent encoder-decoder structure trained in a complete E2E manner following the joint CTCattention procedure [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF].

Joint CTC-attention models

Connectionist Temporal Classification

In E2E ASR systems, the task of sequence-to-sequence mapping from an input acoustic signal X = [x1, ..., xn] to a sequence of symbols T = [t1, ..., tm] is complex due to: 1) X and T could be in arbitrary length; 2) The alignment between X and T is unknown in most cases; 3) T is usually shorter than X in terms of symbols.

To alleviate these problems, connectionist temporal classification (CTC) has been proposed [START_REF] Graves | Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks[END_REF]. First, a softmax is applied at each timestep, or frame, providing a probability of emitting each symbol X at that timestep. This probability results in a symbol sequences representation P (O|X), with O = [o1, ..., on] in the latent space O. A blank symbolis introduced as an extra label to allow the classifier to deal with the unknown alignment. Then, O is transformed to the final output sequence with a many-to-one function z(.) defined as follows:

z(o1, o2, -, o3, -) z(o1, o2, o3, o3, -) z(o1, -, o2, o3, o3)    = (o1, o2, o3). (3)
Consequently, the output sequence is a summation over the probability of all possible alignments between X and T after applying the function z(O). Accordingly to [START_REF] Graves | Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks[END_REF] the parameters of the models are learned based on the cross entropy loss function:

X,T ∈train -log(P (O|X)). (4)
During the inference, a best path decoding algorithm is performed. Therefore, the latent sequence with the highest probability is obtained by performing the argmax of the sof tmax output at each timestep. The final sequence is obtained by applying the function z(.) to the latent sequence.

Attention-based encoder-decoder

Conversely to CTC, encoder-decoder models [START_REF] Bahdanau | End-to-end attention-based large vocabulary speech recognition[END_REF] do not suffer from a forced many-to-one mapping. Indeed, the input signal X = [x1, ..., xn] is entirely consumed by a first encoder neural network (e.g. a recurrent neural network), before being fed to a second one that is free to emit any number of outputs T = [t1, ..., tm] starting from the information contained in the last latent space of the encoder. Major bottlenecks are therefore related to the ability of the encoder to map correctly an entire sequence to an arbitrary latent space, and to the decoder that is not aware of the sequential order of the input signal. To alleviate these issues, attention-based encoder-decoder have been proposed [START_REF] Bahdanau | End-to-end attention-based large vocabulary speech recognition[END_REF].

From a high-level perspective, an attention-based decoder is able to look over the complete set of the hidden states generated by the encoder, making it feasible to "choose" the relevant information in the time-domain [START_REF] Bahdanau | End-to-end attention-based large vocabulary speech recognition[END_REF]. More precisely, an attention-based encoderdecoder consists of two RNNs. The encoder part remains mostly unaltered and maps an input sequence of arbitrary length n, X = [x1, ..., xn] to n hidden vectors h, (h1, ..., hn). Then, the attentiondecoder generates m output distributions O = [o1, ..., om] corresponding to m timesteps, by attending to both the n encoded h and the previously generated token ot-1. Two special tokens, denoting the start-of-sentence and end-of-sentence are added to integrate boundaries. The loss function is nearly identical to CTC, except that a condition on the previous ground-truth token (o truth t-1 ) is added [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF]:

L enc,dec = m t=1 -log(P (ot|xt, o truth t-1 )). (5)
Following [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF], our proposed E2E model relies on a locationbased attention method [START_REF] Luong | Effective approaches to attention-based neural machine translation[END_REF]. The attention weight vector at time step t named at identifies the focus location in the entire encoded hidden sequence h at time step t with a context vector ct:

ct = i at,ihi, at,i = exp(γet,i) m k=1 exp(γe t,k ) , (6)
γ is the sharpening factor [START_REF] Chorowski | Attention-based models for speech recognition[END_REF] and e t,k relates the importance or energy of the k annotation vector for predicting the i output token. In the case of a location-based attention, e t,k is computed as:

e t,k = w T tanh(W st-1 + vh k + U (F * at-1) + b, (7)
with w, W ,V ,F ,Y and b are different trainable parameters, st-1 is the hidden state of the decoder at the previous time step, and * is the convolution product. It is important to note that the implementation of e varies accordingly to the type of attention mechanism employed [START_REF] Bahdanau | End-to-end attention-based large vocabulary speech recognition[END_REF]. Then, the decoder generates an output token ot from an input vector based on both the context vector ct and the previous state st-1 alongside with updating the current state st following RNNs equations with st-1, ct and ot. Unfortunately, ASR systems solely relying on an attention mechanism are highly perturbed by noisy data that generate wrong alignments [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF]. Furthermore, it has been shown that it is difficult to train such models from scratch on wide input sequences [START_REF] Chorowski | Attention-based models for speech recognition[END_REF][START_REF] Bahdanau | End-to-end attention-based large vocabulary speech recognition[END_REF].

Joint CTC-attention

To overcome the limitations of both CTC training and attentionbased encoder-decoder models and to benefit from their strengths, [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF] introduced the joint CTC-attention paradigm. The key idea of the latter method relies on the introduction of the CTC loss as an auxiliary task to the attention-based encoder-decoder training. More precisely, both losses are combined and controlled with a fixed hyperparameter λ (0 ≤ λ ≤ 1) as:

Ljoint = (1 -λ)L enc,dec + λLCT C . (8)

E2E-SincNet

SincNet has only been combined with mere feed-forward NNs [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF][START_REF] Ravanelli | Speech and speaker recognition from raw waveform with sincnet[END_REF], while the joint CTC-attention approach has only been applied to pre-computed acoustic features such as MFCCs and Mel-filterbanks [START_REF] Kim | Joint ctcattention based speech recognition using multi-task learning[END_REF][START_REF] Watanabe | Espnet: End-to-end speech processing toolkit[END_REF]. We propose to combine SincNet to the latter training procedure in a efficient, interpretable and fully E2E ASR approach (Figure 1). The E2E architecture is composed of three components: 1) An encoder that operates over the raw audio signal with a first SincNet layer followed by N one-dimensional convolutional layers. The latent features are then consumed by a traditional bidirectional RNN. 2) A simple CTC decoder that produces a token for each time step encoded. 3) An attention-based decoder that looks out over the entire encoded hidden sequence to output the right symbol. The model is trained following the joint CTC-attention loss function (Eq. 8).

EXPERIMENTS

In this Section, E2E-SincNet is compared to other state-of-the-art end-to-end ASR systems with two different speech recognition tasks. First, datasets alongside with pre-computed and raw acoustic features are detailed (Section 3.1). Then, baselines and proposed models architectures are described (Section 3.2). Finally, we report and discuss the results in Section 3.3.

Speech recognition datasets and acoustic features

E2E-SincNet is evaluated in two different tasks of phoneme recognition with the TIMIT dataset, and word recognition with the Wall Street Journal corpus.

The TIMIT phoneme recognition task

The TIMIT [START_REF] John S Garofolo | Darpa timit acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1.1[END_REF] dataset is composed of a standard 462-speaker training dataset, a 50-speakers development dataset and a core test dataset of 192 sentences for a total of 5 hours of clean speech. During the experiments, the SA records of the training set are removed and the development set is used for early stopping. The accuracy is reported in terms of Phoneme Error Rate (PER). TIMIT is considered as a challenging task for E2E systems due to its very limited amount of available training data (less than 5 hours).

The Wall Street Journal speech recognition task

Only the full "train-si284" dataset is considered as a training set (81 hours), due to the fact that the models have already been evaluated on the smaller TIMIT dataset. The usual "test-eval92" is used at testing time, while "test-dev93" is considered as a validation dataset. The accuracy is reported in terms of Word Error Rate (WER).

Acoustic features

In the original SincNet proposal [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF], chunks of raw signal are created every 400ms with a 10ms overlapping. Instead, we propose to split the waveform of each speech sentence into blocks of 25ms. Indeed, [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF] introduce a SincNet followed by a DNN that requires both right and left contexts to be trained properly. Our approach relies on a combination of SincNet with a RNN allowing the latter context to be captured within the recurrent connections, making it feasible to drastically reduce both the input dimension and the VRAM consumption at training time (i.e. by a factor of 5). Then, other E2E systems usually process either pre-computed acoustic features. Therefore, 23 and 80 Mel-filter-banks are extracted for the TIMIT and WSJ datasets respectively, based on windows of size 25ms with a 10ms overlapping.

Models architectures

Two different E2E ASR models operating on the raw waveform and relying on the encoder-decoder approach with the joint CTCattention training scheme are introduced (see Figure 1). E2E-SincNet. The encoder is made of a specific SincNet layer and 3 one-dimensional convolutional layers with 256 -128 -128 filters, followed by a bidirectional LSTM composed with 4 or 6 layers of size 512 for the TIMIT and WSJ tasks respectively. A onedimensional maxpooling of length 3 is applied after the convolutional and SincNet layers to reduce the signal dimension. In [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF], the authors introduced a SincNet layer composed of 128 filters of size 251. We propose to increase the number of filters to 512 and to decrease their size to 129 to enhance the local resolution of the filters, better fitting to the task of speech recognition. Finally, the decoder relies on a simple attention layer of size 512 combined with the CTC loss (Section 2.2.3).

E2E-CNN. This architecture is proposed to highlight the impact of the SincNet layer in E2E-SincNet. More precisely, E2E-CNN is identical to E2E-SincNet but with a traditional convolutional layer with 512 filters to replace the SincNet one.

Models are trained based on the Adadelta optimizer with vanilla hyperparameters [START_REF] Matthew | Adadelta: an adaptive learning rate method[END_REF] for 20 and 15 epochs during the TIMIT and WSJ tasks respectively. The joint CTC-attention loss control hyperparameter λ (Eq. 8) is set to 0.5 for the TIMIT experiments and decreased to 0.2 with WSJ. No dropout is applied and the results observed on the test dataset are reported with respect to the best performances obtained on the validation dataset.

Results and discussions

Table 1 reports the results obtained by our approaches compared to a more traditional E2E model operating on Mel-filter-banks on the TIMIT dataset. First, it is worth underlining that the E2E-SincNet obtains the best performances with a PER of 19.3% on the test compared to 20.5% for the baseline and 21.1% for the non-SincNet alternative representing a relative improvement of 1.2% and 1.3% respectively. Unfortunately, TIMIT is a very challenging task for E2E systems due to the small amount of available training data (less than 5 hours), resulting in worse performances in comparison to hybrid DNN-HMM ASR systems [START_REF] Ravanelli | The pytorch-kaldi speech recognition toolkit[END_REF]. Therefore, it is of crucial interest to scale the E2E-SincNet model to a larger dataset to validate its suitability to real-world tasks.

Table 2 reports the performances obtained by various SOTA E2E models on the WSJ dataset by integrating a 3-gram recurrent language model (RNNLM) [START_REF] Watanabe | Espnet: End-to-end speech processing toolkit[END_REF]. "Jasper" [START_REF] Li | Jasper: An end-to-end convolutional neural acoustic model[END_REF] uses a transformerXL language model. First, the proposed E2E-SincNet obtains a top-of-line WER of 4.5% on the "test eval92" dataset, outperforming all the baselines. Indeed, a previous best score of 5.9% was reported in [START_REF] Salazar | Selfattention networks for connectionist temporal classification in speech recognition[END_REF], highlighting a relative improvement of 1.2%. The E2E-SincNet outperforms the E2E-CNN with a relative gain of 1.8% on both TIMIT and WSJ task. This demonstrates the efficacy of the SincNet layer to learn an expressive filtered signal enabling a better and lossless latent representation of the raw waveform. It is interesting to note that "transformer" models have recently obtained better performances on multiple ASR tasks [START_REF] Karita | Improving transformer-based end-to-end speech recognition with connectionist temporal classification and language model integration[END_REF]. Nonetheless, transformers are a specific architecture that differ significantly from the presented models and are therefore not considered in our benchmarks.

CONCLUSION

Summary. In this paper, we introduced E2E SincNet, a fully endto-end automatic speech recognition system able to process the raw waveform based on an adaptation of the recent SincNet with the powerful joint CTC-attention training paradigm. The conducted experiments on two different speech recognition-related tasks have demonstrated the superiority of our approach over various other E2E systems based on both pre-computed acoustic features and the raw waveform, achieving one of the best result observed so far with an E2E ASR model on the Wall Street Journal dataset. Future work. SincNet currently suffers from various issues. First, it is important to investigate other filters to efficiently operate over the raw signal. Then, an alternative to the maxpooling must be explored to alleviate the risk of aliasing in the filtered signal.

Fig. 1 .

 1 Fig. 1. Illustration of the proposed E2E-SincNet. The batch dimension is omitted for readability. The raw signal is first encoded with a SincNet layer, followed by multiple 1D convolutions and a bidirectional RNN. Then a CTC and an attention-based decoders emit a sequence of text symbols and are trained jointly.

Table 1 .

 1 Results obtained with different E2E ASR systems on the TIMIT phoneme recognition tasks. "Fea." details the type of input features employed, and "Valid." denotes the validation dataset. Results are expressed in Phoneme Error Rate (i.e. lower is better).

	Models	Fea.	Valid. % Test %
	E2E-CNN	RAW	18.9	21.1
	ESPnet (VGG) [18] FBANK	17.9	20.5
	E2E-SincNet	RAW	17.3	19.3

Table 2 .

 2 Results obtained with different E2E ASR systems on the WSJ dataset. "Fea." details the type of input features employed, "Valid." denotes the validation dataset, "-ASG" is the auto segmentation criterion (i.e a variation of CTC) and "-Att." is attention only. Results are expressed in Word Error Rate (i.e. lower is better).

	Models	Fea.	Valid. Test
	BiGRU-Att. [9]	FBANK	-	9.3
	Wav2Text [28]	FBANK	12.9	8.8
	Jasper [8]	FBANK	9.3	6.9
	E2E-CNN	RAW	9.8	6.5
	ESPnet (VGG) [18]	FBANK	9.7	6.4
	CNN-GLU-ASG [7]	RAW	8.3	6.1
	SelfAttention-CTC [12] FBANK	8.9	5.9
	E2E-SincNet	RAW	7.8	4.7

Code is available at: https://github.com/TParcollet/E2E-SincNet

Acknowledgments. This work was supported by the AISSPER project through the French National Research Agency (ANR) under Contract AAPG 2019 ANR-19-CE23-0004-01 and by the Engineering and Physical Sciences Research Council (EPSRC) under Grant: MOA (EP/S001530/).


Britain To-day, Bulletins from Britain, and Britain: Some Semi-official British Periodicals in the United States During the Second World War Alice Byrne, Aix Marseille Université, LERMA, Aix-en-Provence, France By the end of the Second World War, British Information Services (BIS) employed over 300 people and possessed a well-oiled machinery for distributing news and background information about the United Kingdom across the United States.1 Based in New York, BIS had been formed in the spring of 1941 in order to coordinate British publicity in the United States under the authority of the British Embassy in Washington but employing staff from the MoI. BIS greatly expanded the British propaganda effort which had grown out of the modest prewar British Library of Information (BLI), controlled by the Foreign Office (FO). One of the channels for BIS's information campaign was the magazine Britain produced in conjunction with the MoI based in London. Britain had originally appeared under the title Bulletins from Britain, but it had been renamed as part of a major revision in 1942. BIS was not, however, the only agency to produce a British propaganda magazine: the British Council, although not physically present in the United States, continued to circulate its magazine Britain To-day to a number of American readers.2 Indeed, the British propaganda effort in the United States was marked by interdepartmental rivalry and conflicting views of what information should be given to the American public and through which channels. This parallel history was largely absent from the "official" history by John Wheeler-Bennett deposited in the National Archives, which tended to focus on the gradual construction of an effective propaganda organization. This chapter seeks to flesh out our understanding of how British information to the United States evolved during the war by focusing on the actual material produced and, specifically, on two periodicals initially distributed by the BLI in New York: Britain To-day and Bulletins from Britain/Britain.

Competing Periodicals?

Britain To-day first appeared in March 1939 as a simple newsletter with no cover or illustrations. It declared its objective as follows:

The purpose of Britain To-day is to bring the friends, and, for that matter, the critics of Great Britain into closer touch with current happenings in our country, by recording and commenting upon important features and developments in our national life, whether they be experiments in the reorganisation of industry, inquiries into social or economic problems, new methods of co-operation in the sphere of the central or local government or in the relations between Great Britain and the Dominions, new movements in art or literature, anything indeed which may be of interest to the citizens of other countries occupied with problems like our own.3 It sought, therefore, to paint an image of the UK with a broad brush, defining British culture in terms of the "life and thought" of the country and not merely in the restricted sense of the arts.

However, since Britain To-day was conceived of as a response to German propagandists, concern with refuting anti-British propaganda led it to adopt a defensive tone.4

The outbreak of war gave Britain To-day a clearer purpose but also brought administrative uncertainty. It seemed at first that the British Council and its publications would be absorbed by the newly formed MoI. However, after having been transferred to the MoI at the start of the war, within a few months Britain To-day was moved back to the British Council, which was assigned the task of dealing with cultural and educational topics, leaving war "publicity" and political propaganda to the ministry.5 In practice, the line of demarcation between the two bodies was difficult to establish; despite this, or perhaps because of it, the council's chairman, Lord Lloyd, was determined to prevent the then Minister of Information, Sir John Reith, from curtailing the actions of the council.

Although the MoI had ceded control of Britain To-day to the British Council (operating under the aegis of the FO), Lloyd was conscious of the fact that "Britain To-day was in fact, though not in name, 'Britain at War' in that it attempted to paint a picture of the life of this country in war-time." As such he feared that Reith would insist that Britain To-day be restricted entirely to cultural subjects, thus limiting its appeal. Indeed, without articles of "topical war interest" the magazine risked becoming unsuitable for its task in the eyes of embassy officials who assured its distribution.6 While the MoI continued to press the council to relinquish its activities in the field of press and broadcasting, Britain To-day was specifically excluded from the negotiations.7 Subsequently, the MoI appeared content to leave the running of Britain Today to the British Council, though it did make occasional suggestions of suitable topics.8

The German invasion of France and the Low Countries in spring 1940 brought an end to both the phoney war and the Chamberlain government. Churchill's invitation to Lord Lloyd to join his new cabinet as Secretary of State for the Colonies strengthened the British Council's position. At the same time, the editor of Britain To-day, Maurice Ashley, left to join the army and was replaced in June 1940 with the journalist and literary critic Rolfe Arnold Scott-James. Scott-James had previously edited the London Mercury which, like many cultural periodicals, had ceased publication shortly after the outbreak of war. Scott-James would play a major role in determining the voice of Britain To-day until it finally ceased publication in December 1954.

The occupation of much of Europe by Germany led to a dramatic drop in circulation for

Britain To-day which, however, was soon offset by its expansion in the United States. Americans had had their first taste of Britain To-day at the New York World Fair in 1939 and the magazine was subsequently distributed by the BLI in the same city. As the war intensified, so did British attempts to win over American public opinion, while ostensibly adhering to the official policy of "no propaganda" in the United States. Although active American support had become more vital than ever, this need was matched by a rise in American fears of being manipulated by foreign propaganda.9 Hence the FO and MoI's extreme caution when addressing US public opinion. Various methods were employed, including encouraging British writers to undertake lecture tours in the United States, tailoring BBC programs to US tastes, and increasing the production of British books and articles published in the United States.10

Official involvement in such projects was kept to a minimum so that writers could plausibly claim to be simply meeting American demands for their written and spoken material.

In the summer of the 1940, as Britain began to experience its first air raids, British propagandists expanded their efforts to direct factual information to the United States which would elicit both sympathy and respect for the British war effort. A more direct approach was adopted, notably with the launch of the BBC's North American Service in July 1940.11 The launch of Bulletins from Britain in August 1940 similarly sought to open a new channel for information of a more official nature. This newsletter carried short texts on the British war effort, taken from the cables and mail sent to British officials in the United States. It was distributed from the BLI and was, as Wheeler-Bennett later recalled, intended to "convey factual information to an unselected mailing list."12 Shortly after the launch of Bulletins from Britain, the BLI also increased its monthly order of Britain To-day to 8,000 issues which it distributed across the United States. Britain To-day also compiled its own list of a further 6,000 addresses of individuals and institutions to which it sent the magazine. Such was the success of Britain To-day that by the end of the year the United States was its biggest market with a circulation of 22,000, a figure which the director of the BLI expected to rise.13 Both publications were distributed free of charge, bearing the stamp of the BLI. Although the official origin of Bulletins from Britain was made explicit, Britain To-day only carried the address of the BLI with no mention of the British Council.

Given the success of Britain To-day and the BLI's involvement in increasing its circulation, it may be wondered why Bulletins from Britain was deemed necessary. By 1940, Britain To-day had developed into a more sophisticated fortnightly production, with a cover by Paul Nash, sixteen pages of text and eight pages of illustration. All the articles, except for the editorial, were signed, frequently by men and occasionally by women with established reputations, whether as journalists and critics (Dilys Powell, Ivor Brown), academics (Prof.

Ernest Barker), writers (Laurence Binyon) or experts in other specialized fields (Sir William Bragg, Patrick Abercrombie). With an average of four articles, each four pages long, including the editorial, Britain To-day sought to cover a limited number of topics in some depth. Bulletins from Britain was an entirely different proposition. Published weekly, it consisted of only five pages: most of the texts were anonymous and were only a few paragraphs long; longer twopage articles occasionally carried a byline but none were by famous authors.

In terms of content, almost half of Britain To-day was dedicated to cultural topics with the remainder split evenly between articles dealing with the war and the armed forces, and those dealing with the home front. Bulletins from Britain barely concerned itself at all with cultural matters and was focused purely on the war. However, like Britain To-day, it offered roughly equal space to the armed forces and the home front. To a large extent the two publications reflected the separation of political from cultural propaganda. In practice, though, there was much overlap. The earliest issues of Bulletins from Britain were dominated by the air war, covering such topics as the role of Empire and Polish pilots, the use of US aircraft, and German losses. This is hardly surprising given that its launch came at the height of the Battle of Britain.

Britain To-day also devoted its mid-September editorial to the subject of air raids, both against and by Britain. Its August editorials may also fairly be described as war propaganda dealing with the blockade of Germany, in particular the interests of the neutral nations, and British morale. The latter text ended with a declaration aimed at convincing sympathetic readers in neutral countries, predominantly the United States, of the importance of their support:

The British people are sure of the rightness of their cause and fortified by it. Millions of people in the conquered countries are no less sure of it, and the power of public opinion among neutrals is a spiritual asset of incalculable worth. To the British people immense material force is not lacking. Animated by moral force it will be irresistible. [START_REF] Editorial | The Will to Victory[END_REF] Concerning the home front, Britain To-day tended to present somewhat technical explanations, detailing, for example, the organization of the Local Defence Volunteers or the treatment of civilian casualties from air raids; while Bulletins from Britain placed more emphasis on the theme of Britain "carrying on" despite the Blitz.15 All of these articles highlighted British unity and the emergence of a "new spirit." Britain To-day 's cultural articles went further, for example by criticizing the class tradition of English education and anticipating how the breaking down of class divisions during the war would lead to the emergence of a "more unified, extended and deeper purpose in the whole education system" after the war. This article can be seen as a precursor of the material produced by both the MoI and the British Council from 1942 onwards which placed increasing emphasis on the postwar reform in the fields of welfare, housing, and education. [START_REF] Lindsay | British Education -Past and Future[END_REF] Finally, both Britain To-day and Bulletins from Britain carried articles on wartime theater featuring the presence of American drama in Britain, thereby drawing attention to a shared Anglo-American culture.17

Despite the fact that Britain To-day had to address a wider international audience, it purveyed broadly the same messages as Bulletins from Britain, which were clearly tailored to counter US doubts about the UK's ability to resist Nazi aggression and its suitability as a potential ally. More significant were the differences in form and style between the two publications, indicative of the readers each sought to address. Britain To-day had from the outset been aimed at "those who count" and, without being fully highbrow, tended to be more "academic and literary field." However, this approach was criticized by more recently appointed propagandists who convinced Lord Lothian, the UK ambassador to the United States, that the BLI was struggling to adapt to its new role as the nerve center of British propaganda in the United States. The launch of Bulletins from Britain, and the expansion of Britain To-day in the United States, coincided with a shift in British policy marked by the launch of a new British Press Service in the autumn of 1940. However, the library remained under the aegis of the FO and continued to facilitate the "supply of information to academic and cultural institutions and to the general public," which included the distribution of these periodicals.19

Bulletins from Britain failed to impress FO officials back in London who, by 1941, were already discussing whether Angus Fletcher, the director of the BLI, should take steps to improve it. The problem was partly one of staff, since Fletcher was thought to lack "the right people to produce a readable publication of the kind required."20 Britain To-day also came in for criticism, with the head of the FO's North American Division concluding "it is common ground that the two publications now issued in the USA under HMG's auspices are dreary in the extreme and do our cause little good."21 Their doubts appeared to be confirmed by a memorandum produced by the press committee of the interventionist American Defense, Harvard Group, which criticized Britain To-day for being "wordy" (this was also a common criticism of many of the MOI's earlier publications) and "filled with an all-out optimism." The fundamental problem was that it was not written with an American reader in mind: "The editorials might be interesting to an Englishman or a colonial, but they have no interest for an American reader. If Britain To-day is specifically intended for the Empire and not for the United States, then a similar bulletin, tailored for American consumption, should be worked out."22

The FO had already reached the same conclusion earlier in the year and had thrown its weight behind a proposal for a new magazine by a South African publisher serving in the Canada Corps, Norman Kark, who also had experience of the United States. Kark provided further evidence of the unsuitability of Bulletins from Britain and Britain To-day, claiming that members of the Canadian Corps with working experience of US newspapers to whom he had shown the publications agreed they could "serve no useful purpose." Kark argued that American readers were "nauseated by the present uplift type of article" which should be replaced by more "indirect propaganda" based on "reader interest and entertainment value." Kark's projected magazine was to be designed for American readers and marketed as "a sincere effort to cement the friendship of the two English Speaking Nations."23 An outline of the magazine comprised digests and reprinted articles as well as specially commissioned texts and included cartoons, jokes, and sketches which all contributed to a livelier tone.

Kark's credentials were drawn from his experience publishing a magazine entitled Courier, described by the FO as being produced in a "bright" and "semi-American" style.

Courier differed from Britain To-day and Bulletins from Britain in the emphasis it placed on high-quality photographs and illustrations and its inclusion of sections dedicated to humor, cinema, and fashion. This, combined with its lack of faith in the BLI's ability to produce an appropriate magazine, led the FO to invite the MoI to consider scrapping the existing publications in favor of Kark's new project provisionally entitled Britain Calling. The MoI did not show the same level of enthusiasm, fearing that the new publication would be too much like Courier, but promising to seek Kark's advice if Britain To-day and Bulletins from Britain were to be amalgamated. There is no evidence that the MoI ever contacted Kark but his ideas would later resurface.

A New Approach

By the end of 1941, Britain To-day 's circulation in the United States had reached 40,000. At the request of the BLI, it was decided that Britain To-day should be placed on sale. This offered a means not only to reduce costs but also to remove the taint of propaganda associated with free distribution. In January 1942 Britain To-day was launched at the price of 10 cents and with a new look intended to make the magazine more attractive. Henceforth, Britain To-day would be a 28-page monthly with more illustrations, extra articles, and book reviews. The editorial accompanying the first such issue suggested a shift away from a straightforward "projection of Britain" approach in favor of a more reciprocal exchange: Robert Louis Stevenson used to say that he liked to think of his books as circular letters written to his friends. That is how we like to think of Britain To-day. . . . Friendship implies intellectual intercourse. Each side wants to know what the other has to tell, and to give back his story in return. [START_REF] Editorial | A Circular Letter[END_REF] This was not, however, the kind of bright and breezy publication Britain To-day 's critics had had in mind, but aimed to resemble a worthy, upper middlebrow cultural review along the lines of the London Mercury, the defunct magazine formerly edited by Scott-James.25 Indeed the war news of the earlier format gave way to articles relating to social issues, education (the subject of a series which ran throughout 1942), and reconstruction. Following the US entry into the war in December 1941, the conflict itself was henceforth described as a war for democracy, defined by Britain To-day as "not a national, but an international conception," citing Abraham Lincoln to justify this claim.26 Increasing emphasis was placed on the international dimension of reconstruction and on the importance of "winning the peace."27 Britain To-day's attempts to develop its subscription base were disrupted by a major reorganization of British publicity in the United States which effectively placed responsibility in the hands of the MoI. In May 1942 the BLI was amalgamated with the British Press Service and its functions assumed by British Information Services (BIS), which took over the BLI's mailing lists.28 It thus fell to BIS to send out the first promotion letters, which brought in about 3000 paying subscribers. However, when Scott-James asked BIS to carry on promotion of Britain To-day by sending free copies to suitable people, BIS agreed to do so but only on a limited scale due to lack of staff. The British Council suspected that the real reason for BIS's lack of commitment to Britain To-day was because it preferred to invest its energy in its own publications. Given the unwillingness of BIS to promote Britain To-day, Scott-James received permission to engage a commercial firm to take charge of its distribution in the United States.

An agreement was reached with the Toronto office of the publisher Dent's, which had already taken over responsibility for the printing of the North American edition using molds sent from the UK. This did not however fully resolve the issue as BIS only sent Dent's the list of paying subscribers but not of those who had failed to renew.29 Lack of access to the BIS's mailing list proved a major obstacle to the development of sales in the United States. This was made particularly acute given the political sensitivity about the distribution of unsolicited material. Suggestions that complimentary copies of Britain Today might be sent to American officials or that British Consuls might identify potential subscribers were discouraged by both the FO and BIS. 30 The ambiguous status of Britain Today complicated matters: BIS refused to share its mailing lists with a commercial house while drawing attention to the need to respect US regulation of foreign propaganda.31 Despite its attempt to remarket itself as a commercial periodical, the development of sales of Britain Today remained constrained by the fact that ultimately it was a semi-official publication produced using public funds with the aim of influencing public opinion. Indeed, Dent's was obliged to submit details of all its publications produced for foreign principals, including Britain To-day, under the Foreign Agents Registration Act (1938). These records testify to the fact that Britain To-day failed to convert its circulation of free issues into a significant subscription base as documents filed for 1943 and 1944 give its average monthly circulation as only 2,200 (although this figure does not appear to include copies distributed by BIS).32 BIS finally agreed to recircularize its entire mailing list to promote Britain To-day in the summer of 1944, but there is no evidence that this had a significant impact on sales. [START_REF]Exchange between MoI[END_REF] The MoI was not in fact certain that BIS possessed a separate mailing list for Britain To-day and it is possible that the same list was used to distribute Bulletins from Britain. If this was indeed the case, those readers of Britain To-day who had decided not to subscribe when the magazine ceased free distribution would have continued to receive Bulletins from Britain.

This would offer a further explanation as to why BIS did not wish to share this mailing list with Dent's. Moreover, as was noted by the British Council, at the same time that BIS was pleading lack of staff to justify its failure to promote Britain To-day, it was also planning to launch a new publication. BIS commissioned a poll of Bulletins from Britain's readers by a commercial subsidiary of Gallup to determine what improvements needed to be made.34 As a result, Bulletins from Britain was scrapped and replaced with a new publication, entitled simply Britain. Launched in November 1942, Britain was modelled on the hugely successful Reader's Digest. [START_REF] Bentley | O Dreams, O Destinations" An Autobiography[END_REF] It carried condensed versions of previously published texts, poetry, and fiction, and transcripts of speeches as well as specially commissioned texts. It also included jokes, cartoons and quizzes, and overall bore a close resemblance to the sort of publication suggested by Normal Kark in 1941. Indeed, like the projected Britain Calling, it was specifically marketed as seeking to strengthen Anglo-American relations, with a subtitle announcing, "Toward a greater knowledge and better understanding of America's allies, the people of Britain". As the United States had moved from the status of neutral nation to ally, the need was less for war news than for cultural propaganda aimed at fostering mutual comprehension. This was also the consequence of a new propaganda policy agreed by the FO and the MoI, which anticipated the need for Anglo-American cooperation in the postwar world.36 Both Britain To-day and Britain were concerned with educating their readers about Britain. However, Britain targeted a different audience from Britain To-day and the selection of articles in the UK was entrusted to the popular middlebrow author Phyllis Bentley.37 John Wheeler-Bennett described it thus: 'Britain' was designed for popular consumption so that a non-specialised audience would have an opportunity to read stories about Britain which by their emotional or dramatic appeal would imprint on their minds a picture of the British war effort. [START_REF] Wheeler-Bennett | History of British Information in the USA[END_REF] Like Britain To-day, Britain was put on sale for 10 cents but was far more successful, building up a list of 84,000 subscribers by the following year.39 This suggests that the new format adopted by Britain was generally more appealing to American readers than that of Britain Today. Nevertheless, Britain also benefitted from more effective promotion and distribution by BIS. Indeed, John Wheeler-Bennett would later claim that by 1943, BIS possessed "the only efficient machine in America for distributing information concerning Britain. however, continued to publish articles such as "Shopkeepers or Humanists" by Gilbert Murray, celebrating the "old aristocratic tradition" of humanism, which was exactly the type of rather smug article that the Harvard Group had criticized.45

The war remained the primary focus of Britain particularly through first-hand accounts, whether of a family returning to their blast-damaged house or that of a secret agent parachuted into France.46 Every issue contained an article written by a member of the armed forces while the accompanying photographs often included action shots of combat at sea or, later, soldiers liberating France. This continued until the final issue in April 1945. In the months following D-Day, Britain To-day, however, preferred to focus on questions such as the humanitarian aid delivered in liberated Europe. [START_REF] For Example | Britain To-day 103[END_REF] In the final year of war, the magazine was particularly preoccupied with the postwar order and published a number of articles arguing for the need for international cooperation in order to ensure a lasting peace.48 To a certain extent, Britain Today was already carving out a new post-conflict role for itself as an advocate of cultural internationalism.

Magazines were considered an important vector for British propaganda in the United States, which is unsurprising given the dominance of this form of media in mid-twentieth century America.49 Although British policy stipulated that British information in the United States had to be honest and factual, the different bodies involved in producing British magazines disagreed as to their form and content. In the early years of the war, when the United States was still neutral, Britain To-day and Bulletins from Britain both provided free news about the British war effort distributed through the BLI, with the former laying greater emphasis on cultural matters. However, both magazines were criticized for being ill-adapted to an American readership. The FO failed in its attempt to merge the publications into a new magazine targeting the American public and it was left to the British Council and the MoI to make their own changes. Britain To-day developed into a cultural periodical aimed at a small elite readership with a greater interest in promoting postwar reconstruction projects than in carrying news of the war. Britain, launched in November 1942, was modelled on popular mass-circulation American magazines and informed its readers about the British war effort through a combination of factual and fictionalized accounts. Britain was by far and away the most successful British propaganda magazine in terms of circulation, though it is unclear to what extent this was due to its innate virtues given the fact that it was the only such magazine to benefit from the effective organization of the BIS.

Despite their differences, Britain To-day and Britain were both placed on sale in 1942 in an attempt to pass for commercial publications. Both magazines increasingly described themselves as seeking to inform their American readers about Britain in the interests of mutual comprehension. Although at the beginning of the war, the cultural propaganda produced by the British Council seemed destined to disappear, in reality the war created a need for this type of approach to convince potential allies that Britain was deserving of support. Once the United States had joined the war as the United Kingdom's ally, the need for greater understanding only grew. It would therefore be inaccurate to distinguish the magazines along the lines of cultural propaganda (British Council) versus political propaganda (MoI) since both were concerned with presenting a positive image of British culture and way of life. However, Britain's emphasis on the war, meant that it was ultimately ill-suited to continue publication in the postwar period, unlike Britain To-day. Nevertheless, the reason Britain To-day outlived Britain was primarily an administrative one: while the MoI was converted into the Central Office of Information after the war, the British Council continued to function under its royal charter as a semi-independent organization and as such it was able to maintain its publications. Yet arguably Britain To-day, with its limited circulation and problems of distribution, as well as its rather 1930s feel, was badly placed to survive in the competitive world of American postwar magazines. It ceased publication in 1954 without ever reaching the circulation figures enjoyed by Britain.

  "40 At sixty-eight pages, Britain was longer and more varied in its style and content than Britain To-day. The first number relied heavily on extracts from a book published by Harper entitled London Calling, edited by the novelist Storm Jameson.41 London Calling had itself been discreetly solicited by the MoI and was intended to foster Anglo-American friendship.42 Extracts included poems by John Masefield, Walter de la Mare, and Dorothy L. Sayers, a short story by Rose Macaulay, and reflections on the interwar period by Harold Butler, the head of BIS, who had also published in Britain To-day. The latter article, which anticipated the emergence of "a new kind of society" which would "safeguard rights" and "exact obligations" resembled those published by Britain To-day.43 Indeed, Britain To-day 's editorial of the same month, November 1942, struck a similar note to Butler arguing that the expansion of education was essential to equip every individual to "fulfil his duties as a public-spirited citizen."44 Although Britain carried weighty and informative articles, the overall tone was leavened by the inclusion of humor and fiction. Moreover, semi-fictional accounts dramatized the experiences of those serving on Atlantic convoys, refurbishing ships or in munitions factories. The content was much closer to what the Harvard Group had called for back in 1941. Britain To-day,