Angela Fan
email: angelafan@fb.com

Claire Gardent
email: claire.gardent@loria.fr

Chloé Braud
email: chloe.braud@irit.fr

Antoine Bordes
email: abordes@fb.com

Augmenting Transformers with KNN-Based Composite Memory for Dialog

Various machine learning tasks can benefit from access to external information of different modalities, such as text and images. Recent work has focused on learning architectures with large memories capable of storing this knowledge. We propose augmenting generative Transformer neural networks with KNN-based Information Fetching (KIF) modules. Each KIF module learns a read operation to access fixed external knowledge. We apply these modules to generative dialog modeling, a challenging task where information must be flexibly retrieved and incorporated to maintain the topic and flow of conversation. We demonstrate the effectiveness of our approach by identifying relevant knowledge required for knowledgeable but engaging dialog from Wikipedia, images, and human-written dialog utterances, and show that leveraging this retrieved information improves model performance, measured by automatic and human evaluation.

Introduction

Machine learning approaches to various tasks, such as game-playing or dialog, are often dependent on external information. This information can take multimodal forms, including structured knowledge bases, free text, and images, and also comes in overwhelmingly large quantities. A pressing challenge is to create models that can identify which specific elements of multiple information sources are relevant in a particular context, and incorporate them into standard architectures on each task. In this work, we focus on humanmachine dialog and how to efficiently retrieve external knowledge that is relevant to the dialog. We consider two scenarios and for each scenario, retrieve two types of knowledge: (i) knowledge about similar dialog contexts and (ii) external knowledge used to ground the conversation into real world information.

Knowledge about similar dialog contexts allows for a hybrid retrieval/generative approach to dialog where the system response is generated based not only on a representation of the current dialog context and of the relevant world knowledge, but also based on a response retrieved from a similar dialog context. The retrieved knowledge can be viewed as providing informa-tion about structure and dialog sentences, or utterances: which response is likely given a similar context?

External knowledge is also retrieved to improve the semantic content of the dialog model. In one scenario, Wizard of Wikipedia [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF], general topics are provided to crowdworkers, who are asked to have in-depth and specific conversations about these topics by referencing specific Wikipedia sentences as knowledge. In this scenario, external knowledge is retrieved from a pre-selected set of Wikipedia sentences associated with the current dialog topic. Retrieval aims to select the sentence that is most relevant at each step of the dialog and thereby to ground system responses in relevant world knowledge (e.g. by referring to Star Wars when talking about science fiction).

In the other scenario, Engaging ImageChat [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF], crowdworkers are provided with images and asked to have a conversation inspired by or about the image. In this case, the retrieved external knowledge is images and their associated dialogs. By retrieving images that are similar to the image being talked about, we aim to enrich system responses with knowledge about what is typically mentioned when describing similar images (e.g. when talking about an image with dogs, mentioning their breed).

Our work on incorporating different types and modalities of knowledge is related to methods that strive to add external memory, such as knowledge bases, to neural networks. Previous work has explored incorporating large external memories into neural network layers [START_REF] Weston | Memory Networks[END_REF][START_REF] Sukhbaatar | End-to-End Memory Networks[END_REF][START_REF] Sukhbaatar | Augmenting Self-attention with Persistent Memory[END_REF][START_REF] Lample | Large Memory Layers with Product Keys[END_REF]. Many existing approaches focus on using attention over the memory slots, which is computationally intensive and becomes less effective as the the size of the memory grows. In this work, we propose representing multiple sources of external information as fixed encodings and using K Nearest Neighbors search to fetch relevant information. KNN search is computationally efficient and scalable, and libraries like faiss [START_REF] Johnson | Billion-scale similarity search with GPUs[END_REF] allow KNN to be easily used on GPUs and integrated into neural networks. Further, the external memories are pre-encoded, so the information encoding is only computed once. As the external memories are kept fixed, they do not require any training to learn the memories along with the model. We can thus scale easily to larger memories by learning only the KNN-based read operation to identify relevant information from the memory.

Our core contribution proposes an efficient, KNNbased Information Fetching (KIF) module that can access relevant external knowledge, combine knowledge from different sources, and integrate this information into standard sequence to sequence architectures. We apply these flexible modules to two dialog datasets that challenge generative models to leverage external information to write coherent, on-topic responses. Both of our chosen tasks require models to leverage external information, such as information from Wikipedia or images, to engage in the conversation. We show that relevant information can be identified from hundreds of thousands of candidates in a multi-modal, multiknowledge-source setting to improve the performance of generative dialog models. Further, the output of the KIF modules is interpretable as specific humanreadable knowledge elements are selected, allowing users to better understand the information the generative model conditions upon when writing the subsequent utterance. On both datasets, we achieve state-ofthe-art results compared to generative models and find there is no statistically significant difference in the interestingness or human preference of our model output compared to state-of-the-art retrieval models.

Related Work

We discuss related work on learning to incorporate external knowledge into neural networks and efficiently access relevant information. We then describe work in generative dialog that incorporates knowledge.

Incorporating External Knowledge

Augmenting neural networks with memory, or longer term components that can be accessed with read and write operations, has been explored in various proposed architectures. For example, Memory Networks [START_REF] Weston | Memory Networks[END_REF][START_REF] Sukhbaatar | End-to-End Memory Networks[END_REF][START_REF] Sukhbaatar | Augmenting Self-attention with Persistent Memory[END_REF] introduce attention mechanisms over large external memories. Neural cache models (Grave et al., 2017b) simplify these to access previous memories with a dot product. Previous work has also studied how to read and write into these memory architectures [START_REF] Rae | Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes[END_REF][START_REF] Graves | Neural Turing Machines[END_REF][START_REF] Joulin | Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets[END_REF]. In contrast, we focus on how to read large memories.

Another line of research has focused on computational scalability for larger external memories to allow efficient access of information. For example, [START_REF] Chandar | Hierarchical Memory Networks[END_REF] propose a hierarchical memory network rather than a flat one and [START_REF] Rae | Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes[END_REF] learn sparse operations to read and write. [START_REF] Lample | Large Memory Layers with Product Keys[END_REF] focus on learning memories of up to one million slots and how to efficiently access the slots using product keys. [START_REF] Khandelwal | Generalization through Memorization: Nearest Neighbor Language Models[END_REF] use nearest neighbor operations to augment language models by performing retrieval at the token level -in contrast, we focus on multi-modal retrieval of multiple pieces of knowl-edge based on an entire dialog context. Beyond explicit memory representations, it may be possible to store information implicitly during training time by memorizing common patterns present in text [START_REF] Petroni | Language Models as Knowledge Bases?[END_REF]. We focus on learning to fetch relevant information from multiple explicit external multi-modal knowledge sources and integrate them into one network. Further, our work allows the retrieved information to be interpreted as each memory slot is an explicit fact that can be read as text, rather than a learned vector such as in [START_REF] Lample | Large Memory Layers with Product Keys[END_REF].

Work has also focused on computationally efficient softmax operations [START_REF] Mnih | A Scalable Hierarchical Distributed Language Model[END_REF]Grave et al., 2017a;[START_REF] Chen | Strategies for Training Large Vocabulary Neural Language Models[END_REF]. Many approximate softmax techniques use KNN-like operations to form clusters, and the overall softmax operation is constrained by the slow calculation of the exponential. Our usage of KNN benefits from efficient and scalable libraries such as faiss and nmslib.

Generative Dialog

We develop a general architecture for incorporating external information and apply it to the case of generative dialog models. Previous work in dialog has leveraged knowledge as necessary information to accomplish the task. For example, airline and restaurant booking tasks often use API calls to access information about reservation times and availability [START_REF] Bordes | Learning End-to-End Goal-Oriented Dialog[END_REF]. In contrast, our work focuses on how to incorporate unstructured knowledge, such as free text found on the web. Previous work has employed architectures that attend over the available knowledge and identify relevant pieces of information, which scales poorly with large quantities of information [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF][START_REF] Qin | Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading[END_REF][START_REF] Lian | Learning to Select Knowledge for Response Generation in Dialog Systems[END_REF]. We replace the use of attention over external information with the output of a KNN module. Other work has investigated incorporating information retrieval in language modeling and question answering [START_REF] Chen | Reading Wikipedia to Answer Open-Domain Questions[END_REF][START_REF] Fan | Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs[END_REF][START_REF] Seo | Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index[END_REF][START_REF] Guu | Retrieval Augmented Language Model Pre-Training[END_REF], while we focus on dialog applications and flexibly incorporating knowledge from multiple, multi-modal sources.

On the modeling side, work has explored both generative (Serban et al., 2016b,a) and retrieval based models [START_REF] Zhang | Personalizing Dialogue Agents: I have a dog, do you have pets too[END_REF], which identify the best utterance from the training set to return as the dialog response. This often leverages self-attention or crossattention mechanisms [START_REF] Humeau | Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring[END_REF]. Further work has explored hybrid models, for example using the output of a retrieval model as input for a generative model [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF][START_REF] Weston | Retrieve and Refine: Improved Sequence Generation Models For Dialogue[END_REF][START_REF] Cai | Retrieval-guided Dialogue Response Generation via a Matching-to-Generation Framework[END_REF][START_REF] Zhu | ReBoost: a retrieval-boosted sequenceto-sequence model for neural response generation[END_REF]. Some of this work has specialized to use both types of models to generate conversations in an ensemble [START_REF] Song | Two are better than one: An ensemble of retrieval-and generation-based dialog systems[END_REF] or to specifically improve consistency [START_REF] Song | Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation[END_REF]. We extend these approaches by augmenting generative models with retrieval-like operations based on KNN search, allowing dialog models to flexibly incorporate Figure 1: KIF modules fetch relevant information from multi-modal external knowledge. External knowledge sources E 1 and E 2 are pre-encoded by encoder M (green). In the model, input x i is encoded by encoder M (blue) to produce M (x i ). KIF modules (orange) operate on M (x i ) and identify the nearest neighbors encoded in M (E 1 ) and M (E 2 ) using KNN. Identified relevant elements from E 1 and E 2 are re-encoded by M in a gating mechanism with a weighted sum (represented by σ(WS1 i ) • WS1 i , where WS stands for weighted sum), then concatenated to M (x i ). Full description with notation can be found in Section 3. various sources of external knowledge at the same time and scale to large quantities of retrieval candidates.

KNN-based Information Fetching Modules

Broadly, the KNN-based Information Fetching (KIF) module assumes an encoder model M can access inputs X = {x 1 , x 2 , . . . , x n }. For example, X can be a collection of sentences, and x i represents an individual sentence. In a setting without additional supporting information, the encoder will process an input x i and produce the encoder output M (x i ). If x i is a sequence such as a sentence, then M (x i ) is a representation of the variable size of the sequence length by the fixed size encoder M 's hidden size. However, in many tasks, additional information is present, represented as E = {e 1 , e 2 , . . . , e m }. We encode each element of X and E into a vector representation using the encoder. To identify the closest information in E that is relevant to x i , our general approach will be to use K Nearest Neighbors by comparing the representation of x i with the representation of each element in the set E. K Nearest Neighbors is a fully differentiable operation [START_REF] Plötz | Neural Nearest Neighbors Networks[END_REF], so can be incorporated in a straightforward way into neural models. The most relevant information in E will then be available in the model. We display a KIF-Augmented model in Figure 1 and describe how the KIF module operates.

One challenge to overcome is that the representation of all elements of the knowledge source E are pre-computed and kept fixed, creating M (E) -we do not backpropagate to affect the embeddings of the pre-encoded knowledge. In the early stages of training, the model receives large amounts of loss, which would affect the quality of the pre-encoded embeddings if we backpropagated to them. Further, encoding the fixed external knowledge once and re-using it allows for greater scalability. However, this lack of backprop-agation can introduce a mismatch between the encoding of E and the encodings produced by a model that is training, as the training model has constantly changing representations because the weights are being learned. We use M to represent the original encoder model used to encode E and M to represent the constantly training model that is encoding X. The model must learn a function to align M (x i ) to the pre-encoded elements of the external memory M (E).

To circumvent this misalignment, we learn a mapping operator f E (M (x i )) that trains to map elements of the model's representation of X, or M (X), into the additional information representation space M (E). Concretely, f E (M (x i )) is a multi-layer perceptron with ReLU nonlinearities. From the input elements of X, f E (M (x i )) learns representations of an output close to the corresponding projection of X into E. This can be interpreted as learning a read operation on a fixed external memory. If there was no change to the encoding of the model compared to the pre-computed knowledge, then the ideal mapping operator would be the identity function (as M would equal M ). However, as the model changes significantly during the training process, the nonlinear mapping capability of f E (M (x i )) is essential to be able to identify the correct knowledge E from the input X.

Thus, a model augmented with KIF will incorporate external knowledge in the following manner. First, we find the k nearest elements to f E (M (x i )) in M (E), based on KNN search with inner product. Then, the relevant elements identified by KNN are re-encoded by M . For example, if element e j is retrieved by KIF, it would produce M (e j ). We use the optimized faiss library for KNN search, which can conduct billionscale KNN efficiently on GPUs.

The KNN output for an element x i is produced by using faiss to search for the k nearest representations to f E (M (x i )) in M (E). Note that as the en-coders M and M produce output representations of variable length (for example, in the case where x i is a variable length sequence, such as a sentence), we average across the length dimension to produce a fixed-size representations r to conduct the KNN search.

r xi = Avg f E (M (x i ))
(1)

R E = Avg(M (e)) | e ∈ E (2)
KNN xi = KNearest k, r xi , R E (3)
Then, the KIF module output for an element x i is the set of all re-encoded representations of the KNNretrieved knowledge:

KIF xi = M (e) | e ∈ KNN i (4)
These elements are weighted by their normalized nearest neighbor scores and then summed. This is subsequently concatenated to M (x i ) to form the final encoder output:

[M (x i ), WeightedSum(KIF i )] (5)
This can be easily extended to using multiple modules simultaneously. For instance, two sources of external information, E 1 and E 2 , can be combined by identifying the top candidates of each information source. The weighted sum of the KIF output on each information source is concatenated with the encoded input M (x i ). The KIF output dimensionality is the same size as the hidden size of M (x i ), so they can be directly concatenated.

Finally, different sources of information may not be required for every prediction and some information sources can be more important than others. To allow the model to make more fine-grained decisions about what information to use from what source, and how much of it, we add a gating mechanism using a sigmoid function around each weighted sum of KNN representations. KIF1 i and KIF2 i denote the KIF module from Equation 4applied to E 1 and E 2 respectively.

WS1 i = WeightedSum(KIF1 i ) (6) WS2 i = WeightedSum(KIF2 i ) (7)
which produces the final encoder output, a concatenation of M (x i ) with the output of multiple KIF modules:

M (x i ), σ(WS1 i ) • WS1 i , σ(WS2 i ) • WS2 i (8)
This concatenation represents the output of the encoder M and can be used for various purposes, such as providing the encoder output to a decoder in a sequence to sequence model.

Applying KIF to Dialog Tasks

We describe how to apply KIF to the task of generative dialog, a setting where models must generate engaging and on-topic responses. We investigate dialog for two reasons: first, dialog agents must be able to consult relevant information to maintain the topic of the conversation. Second, retrieval-based agents have strong performance compared to generative ones, due to their ability to copy dialog utterances from the training set. Using KIF, we can incorporate the benefits of retrieval architectures into generative, knowledge-based models.

KIF for Generative Dialog

In dialog, x i represents the text of the conversation i. A conversation consists of multiple back-and-forth utterances (or turns). For example, a conversation could consist of 4 turns:

x i = [x i,1 , x i,2 , x i,3 , x i,4 ] where x i,4
is the direct utterance the model should respond to, and the earlier utterances are the conversation context.

Standard generative dialog models use a Transformer neural network as the encoder M and want to produce an output that is an appropriate response to the conversation. However, in many cases, the conversation history alone does not include all of the information required to produce an appropriate response. For example, if a model needs to chat about a specific movie, it can be helpful to provide the model with more information about that movie so a more interesting dialog response could be produced. To incorporate knowledge, models often concatenate a knowledge source E such as Wikipedia to x i and use attention modules to identify the most relevant knowledge. However, this approach is computationally intensive when handling large quantities of information. Further, attention mechanisms have been found to operate poorly over long sequences, as the mechanism becomes blurry due to the softmax and struggles to make fine-grained decisions (Fan et al., 2018b). The same is true for hierarchical approaches, which lack scalability.

We augment Transformer sequence to sequence (seq2seq) networks on the encoder side with KIF to improve generative dialog models. We experiment on two dialog tasks, Wizard of Wikipedia [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF] and Engaging ImageChat [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF]. In both datasets, models must leverage information external to the dialog history alone -in Wizard of Wikipedia, the chat requires access to knowledgeable facts and in Engaging ImageChat, discussion about a specific image. As models must process multiple inputs and ground responses in the knowledgeable facts or images, these tasks challenge existing seq2seq approaches.

Wizard of Wikipedia

The goal of the Wizard of Wikipedia dataset is to train knowledgeable agents that can chat in any domain. The dataset contains 1,365 various topics discussed in 18,430 dialogs in the training set, totalling 166,787 training utterances. Each topic is a general concept, such as dogs or ice cream, and is included as the first utterance of the conversation. The conversation is meant to be in-depth and detailed, so individual utterances must reference specific knowledge as a basis for the utterance. The knowledge takes the form of Wikipedia sentences. For example, the chat utterance I love Toy Story! It was released in 1995 would reference the Wikipedia sentence Toy Story is a 1995 American computer-animated buddy comedy [...]. For each utterance, a set of sentences are identified by an information retrieval system, and the crowdworker selected one knowledge sentence as the basis for their utterance.

Knowledge Sources. Our model for Wizard of Wikipedia has access to two sources of external information, E 1 and E 2 :

• E 1 is Wikipedia Knowledge provided by the dataset as evidence to support knowledgeable chitchat (initially curated by the information retrieval system used in [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF]). The scale of this KNN search is to filter through an average of 34 sentences. The KIF module uses dialog features to fetch relevant knowledge to condition upon to generate the subsequent utterance. • E 2 is Training Utterances. To incorporate the benefits of retrieval-based dialog models to the generative setting, we use KIF to identify relevant utterances from the training set and take their responses as input. If many conversations about dogs have already occurred, models should be able to take advantage of these human-written examples to improve their generations. For example, likely conversation could occur about the breed of the dog, daily routine with a pet, and similar topics. There are around 170K dialog utterances as inputs to KNN search. This can be interpreted as incorporating the benefits of retrieval models by identifying an utterance with similar structure as the text the model would like to generate. We do not allow the module to fetch the correct response of the current conversation context.

Access to these two sources of knowledge can be seen as learning a template and a topic separately. Sample templates can be identified from the training utterances, and topic-specific information learned by accessing the Wikipedia knowledge.

Additional KNN Features. To better identify relevant training utterances from the large quantity available, we break down x i into conversation sub-features for a more fine-grained match in the KNN search step. By conducting KNN on more features, we can achieve higher quality retrieval. We leverage the nature of dialog to decide these features.

We concatenate the encoding of the most recent dialog utterance (e.g. x i,last ) with the encoding of the dialog context from the current conversation and the turn number t, such that M (x i,last ), M (x i,-last ), t is the representation used for KNN search. Concretely, if the model is trying to produce the 5th turn of the conversation, then x i,last is the most recent utterance from the dialog partner, x i,-last would be the last 3 turns of exchange, and t would be 4. Note that the turn number is represented as a standalone number. These are known to be salient conversation features. The most recent dialog utterance is the direct turn the model is responding to, and the dialog context may provide additional clues. The turn number is important, as earlier turns are often generic (e.g. how are you doing today) and later turns are more specific.

Engaging ImageChat

The goal of Engaging ImageChat is to create agents capable of chitchatting about images selected from the YFFC100M dataset [START_REF] Thomee | YFCC100M: The new data in multimedia research[END_REF]. The dataset contains 186,782 dialogs in the training set, each about a unique image, totalling 355,862 utterances. Agents are assigned one of 215 personalities (e.g. sweet, caring, excited) to increase engagingness. Previous work [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF][START_REF] Shuster | Engaging Image captioning via Personality[END_REF] identified that both crowdworkers and models, when provided with personalities, produced more diverse, interesting responses, as evaluated by humans.

We use a Multi-Modal neural network designed to handle both image input and text input. Following [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF], the images are encoded using a pre-trained ResNeXt network [START_REF] Xie | Aggregated Residual Transformations for Deep Neural Networks[END_REF]. To extract the final image representation, we project the 2048-dimensional output of the image encoder to 512dimensions using a deep multi-layer perceptron with ReLU activation units. The conversation history, which includes the one-word personality, is encoded with a Transformer encoder network. The image and conversation are integrated using the Multimodal-Sum-Combiner module proposed in [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF].

Knowledge Sources. Our model for Engaging Im-ageChat has access to two sources of external information, E 1 and E 2 : Additional KNN Features. To identify relevant information from training utterances, we use the same dialog features as Wizard of Wikipedia in the KNN search step, with one modification: we add the personality provided by the dataset. We represent the personality feature as the personality word, such as caring, and embed it with the encoder M . As utterances from speakers with the same personality are more likely to be similar, this feature improves the quality of the fetched information. For example, conversations with the sweet personality often include similar text such as aww, that's wonderful. We use two additional features for the KNN search: t, the turn number, and p, the personality. This feature is explicitly used in [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF] to improve the engagingness and flow of the conversation. Similar to Wizard of Wikipedia, we represent the conversation turn t as a number. The Transformer model is used to encode text x i and produce a representation of the text, then the turn number t and personality p are represented separately. As the personality is a word, we use the same Transformer to encode it. The concatenation of features used for KNN search is: M (x i,last ), M (x i,-last ), t, p.

• E 1 is
5 Experimental Setup

Implementation Details

Parameter Settings. We use parl.ai [START_REF] Miller | parl.ai: A Dialog Research Software Platform[END_REF] to implement our models. The data for both datasets used is available for download from parl.ai as well. We use byte-pair encoding [START_REF] Sennrich | Neural Machine Translation of Rare Words with Subword Units[END_REF] to represent the text to better handle the rare word problem [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF]Fan et al., 2018a). Our generative Transformer models have 8 encoder layers and 8 decoder layers, with FFN size 2048, embedding dimension 512, and 4 attention heads. We optimize using Adam (Kingma and Ba) and the inverse square root learning schedule [START_REF] Vaswani | Attention is All You Need[END_REF] with 10k warmup updates. The initial learning rate is 0.0001 and we optimize for model perplexity. We use a dropout of 0.5 and set gradient clipping to 0.1. We set k = 5 for all cases. For both datasets, we model a vocabulary size of 54944 based on the BPE-based vocabulary from the Reddit pre-training. We tuned the learning rate and batchsize hyperparameters together.

Pre-training. We pre-train the Transformer seq2seq model used for both datasets on 250M comments from Reddit. The Reddit dataset was made available by pushshift.io. The comments are parsed to maintain conversational threads of users responding to each other, so the encoder network has been exposed to conversational context at training time. Note that the Reddit dataset does not include aspects such as personality, as those are unique to specific datasets such as Engaging ImageChat. The context size in pre-training is set to 512 tokens. The ResNeXt encoder used to model images for the Engaging ImageChat dataset was pretrained on 3.5 billion images [START_REF] Mahajan | Exploring the limits of weakly supervised pretraining[END_REF].

Evaluation

Generation. We generate with beam search, setting the beam size to 4. We use 3-gram blocking. This technique disallows repeated n-grams from being generated multiple times and reduces repetition.

Automatic Metrics. Following Dinan et al. ( 2018), we compute F1, a metric of unigram overlap, between the generated utterance and the human-written reference utterance from the dataset. For generative models, utterances are generated using beam search. For retrieval models, the next utterance is predicted by ranking the entire set of training utterances, and the highest scoring utterance is chosen.

In Wizard of Wikipedia, there are two test sets: a set of seen topics, or topics that have been seen at training time with new test-time dialogs. The second set is unseen, or topics that have not been encountered at all during training time. We evaluate on both subsets.

Model Test F1

Retrieval Baselines Retrieval Transformer [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF] 9.8 1

Generative Baselines

Generative Transformer MemNet [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF] 7.1 + Reddit Pre-Training 12.8 Retrieve and Refine [START_REF] Weston | Retrieve and Refine: Improved Sequence Generation Models For Dialogue[END_REF] 13.6 Response Generation with MR [START_REF] Qin | Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading[END_REF] 13.2 KIF-Augmented Transformer 14.4 Human Evaluation. We follow the setup and use the analysis questions proposed in the Acute-Eval dialog evaluation system [START_REF] Li | ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons[END_REF]. For reproducibility, we adopt this existing evaluation setting that has been applied to several dialog datasets. We use the question wording suggested by Acute-Eval and follow their self-chat procedure and interface. As one of the original datasets assessed in this system was Wizard of Wikipedia, their evaluation setting extends naturally to ours. We collect 100 human-bot conversational dialogs on a crowdsourcing platform for both datasets.

The dialogs are eight turns long. Then, we show pairs of the collected conversations side by side, one conversation with a human and model A and the other conversation with a human and model B. We ask annotators the following questions:

• Who would you prefer to talk to for a long conver-sation? • If you had to say one of the speakers is interesting and one is boring, who would you say is more interesting? • Which speaker sounds more human?

• Which speaker has more coherent responses in the conversation? • If you had to say that one speaker is more knowledgeable and one is more ignorant, who is more knowledgeable? (Wizard of Wikipedia only)

We measure the percentage of time one model was chosen over the other, taking the majority agreement between three evaluators. To reduce variance, dialogs paired in the evaluation were collected on the same topic for Wizard of Wikipedia and collected on the same image and personalities for Engaging ImageChat. Topic and images selected for evaluation are unique and taken randomly from the test set.

Baselines

We compare Transformers augmented with KIF to other existing approaches on Wizard of Wikipedia and Engaging ImageChat. The best approaches, judged by human evaluation, are retrieval models, the Retrieval Transformer Memory Network from [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF] and the Retrieval Transformer from [START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF]. These have been shown to be strong baselines compared to other retrieval techniques based on TF-IDF [START_REF] Chen | Reading Wikipedia to Answer Open-Domain Questions[END_REF]. Thus, we report the existing retrieval models for both datasets, but focus on Human: I love watching disney movies! Do you like Disney? Model: I love disney, I love watching disney movies and different animations and characters that make them unique Fetched Knowledge: disney consumer products and interactive media -the company specializes in producing animated films Fetched Training Utterance: i love music, I love going to music shows and dancing and having fun Human: What's your favorite disney movie? There are so many good ones. Model: I love the incredibles, they are my favorite disney movie.

Fetched Knowledge: disney announced intentions to develop additional superhero films after the success of the incredibles Fetched Training Utterance: i love kiteboarding, it is one of my favorite activities on the water. Human: omg, me too! the second one just came out i think. I don't watch a lot of sequels though, I think the last one I saw was toy story. Model: That's cool! I remember when toy story was first released in 1995.

Fetched Knowledge: talk of a second film following toy story began around a month after the film's opening, in december 1995 Fetched Training Utterance: that's neat! i like so many of her songs! i remember when her first album was released in 2008. Human: I've actually never seen toy story before. Kind of embarrassing... Model: Ohhh ... I think toy story is a classic? It was first released in 1999.

Fetched Knowledge: toy story 2 is a 1999 american computer-animated comedy film produced by pixar animation studios for walt disney pictures Fetched Training Utterance: Ohhh ... What do people with color blindness do to cope with the effects? comparing to other generative baselines.

We compare to three additional generative baselines. Note that in Wizard of Wikipedia, the construction of the dataset is that sentences of Wikipedia knowledge are provided with the utterances in a concatenated form. Models must identify the relevant information in this provided knowledge, or can access more Wikipedia knowledge beyond the provided sentences. The fol-lowing baseline methods always have access to the information provided in the dataset already, but no additional Wikipedia knowledge beyond that.

• Transformer Memory Networks. To contrast the ability of KIF to existing work, we compare our models to published Transformer Memory Networks [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF]. These models encode each piece of external information independently with a Transformer Encoder, and these are stored as memory slots. To access information in the memory slots, a model performs dot-product attention between the memory slots and the dialog context. In [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF], the knowledge selection from Wikipedia was supervised with either (a) a two-stage model where the first model was trained to predict the right knowledge and a second model conditions on the predicted knowledge to generate the next utterance, or (b) an end-toend model with an auxiliary loss for knowledge prediction accuracy. • Retrieve and Refine. We implement a hybrid model [START_REF] Weston | Retrieve and Refine: Improved Sequence Generation Models For Dialogue[END_REF] that incorporates top retrieval candidates as additional input to Generative Transformer MemNets. In Retrieve and Refine, a fixed number of candidates are retrieved and concatenated to the conversational history in the encoder, making the input much longer. For both datasets, the Retrieve and Refine mechanism that fetches a fixed number of training utterances is added to the Generative Transformer MemNet with Reddit Pre-Training baseline.

Unlike the KIF-Augmented Transformer, the retrieval is conducted with a separate model so there is no backpropagation to affect the retrieval. With KIF, models can alter the retrieved candidates by learning the mapping operator. Further, a fixed amount of information is always retrieved, without the capability to easily rescale to focus on specific candidates. KIF modules have weighting mechanisms to focus more on certain information, and the modules are combined with gating so models can learn which knowledge sources are more important and adjust flexibly. Lastly, Retrieve and Refine is only used to retrieve one source of information: training set utterances. • Response Generation with MR. We implement the model proposed in [START_REF] Qin | Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading[END_REF], which encodes the conversation history and document contextually with a biLSTM before generating the next dialog utterance. The initial model was applied to a machine reading task where a knowledge document was provided along with the conversation history. For Wizard of Wikipedia, we replace the knowledge document with the Wikipedia sentences provided in the dataset. The model then uses the conversation to identify the most relevant information in the document using a cross-attention mechanism. For the Engaging Im-ageChat dataset, as there is no document provided with the dataset, we replace the expected document with the conversation history, and use the most recent utterance in the conversation to attend to the conversation history.

We make an additional improvement to this baseline: in [START_REF] Qin | Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading[END_REF], the embeddings used pre-trained CoVE vectors [START_REF] Mccann | Learned in Translation: Contextualized Word Vectors[END_REF].

We found our Reddit pre-trained Transformer embeddings to work more effectively as they are trained for dialog. Thus, we replace CoVE embeddings with domain-specific ones.

All of Transformer generative baselines are initialized with the same pre-training on Reddit that we use for our models for fair comparison on modeling quality.

Results

We describe the results of incorporating KIF modules into Transformer networks. We display an example conversation between a human and our model in We compare to various baselines using automatic and human evaluation, and discuss our experiments. We present various ablation settings to understand the key features that make our method function.

6.1 KIF is Effective for Incorporating Knowledge Automatic Evaluation. Comparing KIF augmented Transformer networks to published baselines and Retrieve and Refine, we find improved results.

For Wizard of Wikipedia, the improvement in F1 score over the best baseline is around 8 points (see Table 1). A major contributing factor is the construction of the dataset -as each dialog turn is grounded in a specific knowledge sentence from Wikipedia, improving the ability to identify the relevant fact strongly improves performance. Contrasting the results from the seen and unseen test sets in Table 1, the improvement on unseen is worse -it is harder to fetch training utterances for unseen topics.

While Imagechat has no explicit dependency on knowledge, we still see a 2 point improvement compared to the Generative Transformer MemNet (with the additional Reddit pre-training), indicating that KIF can be generally useful (see Table 2). Compared to an even stronger baseline that we tune in this work, Retrieve and Refine, we see 1 point improvement.

Human Evaluation. Results are shown in Figure 2. On both datasets, we find there is large improvement over existing generative models (green bars) that is statistically significant for some of the evaluation questions. Evaluators agree that KIF-augmented Transformers are generally more coherent and humansounding compared to the Generative MemNet.

Compared to existing retrieval models (blue) is more nuanced. Along the lines of existing work [START_REF] Zhang | Personalizing Dialogue Agents: I have a dog, do you have pets too[END_REF][START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF], we find that retrieval-based models score very well in human evaluations that ask how human or interesting a dialog sounds. This is because retrieval models return human-written utterances from the training set and do not suffer from decoding mistakes present in generative models. For example, on Engaging ImageChat, while our model has significantly improved over the generative baseline (see green bars in Figure 2, right), it does not beat retrieval based methods in sounding more human or being more interesting (see blue bars in Figure 2, right). As the Retrieval baseline returns human-written text for other humans to evaluate, we hypothesize that humans score each other's writing quite well. Compared to generative models, which we focus on improving, retrieval models often produce longer text with more interesting, nuanced vocabulary usage, and do not make generation mistakes such as repetition. These factors often lead to the stronger performance of retrieval models.

A surprising result is that KIF-augmented Transformers are more human sounding than retrieval models on Wizard of Wikipedia. This is because the dataset's utterances are long and factual due to the tendency of crowdworkers to copy Wikipedia. Sometimes humans chatting with the retrieval bot would respond uh... that's an interesting fact? Otherwise, our model scores similarly to retrieval models, with most evaluations not having statistically significant difference.

We conduct a second evaluation on the Unseen Test Set of the Wizard of Wikipedia dataset. Results are shown in Figure 3. Trends are similar compared to the results on the Seen Test set, though the preference for the KIF-augmented Transformer is greater over the retrieval baseline. We hypothesize that because the Unseen Test Set is on entirely held out topics, the retrieval baseline can struggle to identify relevant utterances. In contrast, the KIF-augmented Transformer, similar to the generative baseline from [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF], can use the generative capability to produce utterances.

Lastly, we conduct an additional study to examine the variance of the comparative dialog judgements. The evaluation study for Wizard of Wikipedia is repeated three times on different days, and evaluators who have answered on previous days are not allowed to evaluate again in any subsequent experiments. Overall, we find reasonable interannotator agreement rates, around 73% averaged across all evaluations, which is similar to the agreement rates reported in [START_REF] Li | ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons[END_REF]. We find there is greater variance on questions asking which dialog is more human and more interesting, most likely as different evaluators can interpret these in different ways. Further, we see that comparison with the Retrieval model has less variance compared to the Gen-erative model, possibly because the Retrieval model's human written text is devoid of mistakes. Overall, we find that the conclusions (and statistical significance) are stable across multiple evaluations.

Analysis of Fetched Knowledge

Example conversations from our KIF-augmented generative model are shown in Figure 4 on Wizard of Wikipedia. We find that relevant knowledge is identified that affects the content of the generated utterance. For example, the model finds knowledge sentences about Disney movies as the human conversationalist starts the conversation discussing Disney. The model leverages the fetched knowledge to write the content of the generated utterance. In a concrete example, the fetched sentence disney announced intentions [...] after the success of the incredibles leads the model to generate the utterance i love the incredibles, they are my favorite disney movie.

In contrast, the model uses the form of the fetched training utterance often as a template for writing a response. For example, the model copies the training utterance Ohhh ... what do people with color blindness do to cope with the effects? and starts the model generation with Ohhh ... and continues with the question i think toy story is a classic? following the form of the selected training utterance.

Figure 5 displays the top-3 fetched training set utterances and knowledge sentences on the Wizard of Wikipedia dataset when responding to a human utterance. KIF modules can identify multiple relevant items. In response to the human question about blue skies the 1946 movie the model identifies both the comedy film and the band.

Finally, the elements retrieved by KIF modules provide a more interpretable understanding of what the model is conditioning upon to generate a dialog response. In Table 3 while changing the fetched training utterance changes the form of the generated sentence.

Scaling KIF to Challenging Retrieval Settings

KIF modules can be used in more realistic and challenging settings for knowledge retrieval that test the scalability of the module. In Figure 6(a), we compare the Generative Transformer MemNet Baseline with KIF-Augmented Transformers in three settings. The first is the standard Wikipedia sentences provided by the dataset (average 34 sentences). Then, we extend to providing the model with the full Wikipedia article (on average, 57 sentences) and finally to multiple Wikipedia articles (on average, totaling 205 sentences), identified using the conversation's topic. This increasing size of available knowledge could be realistic for settings where it is unclear what information is most relevant, if filtering steps to preprocess the data remove potentially relevant information, or if information synthesis from multiple knowledge sources is necessary to produce a high quality generation. As the Wikipedia knowledge becomes more difficult to identify, performance decreases, but still outperforms the baseline that uses the dataset-provided set of 34 sentences.

Comparing the scaling capability of KIF to the standard Generative Transformer MemNet Baseline highlights the advantage of using KNN. The attention-based mechanism used in [START_REF] Dinan | Wizard of Wikipedia: Knowledge-Powered Conversational Agents[END_REF] struggles to identify salient information when given increasingly larger quantities of knowledge, unlike the KNN information fetch. We hypothesize the attention mechanism is challenged by softmax-ing over a larger quantity of inputs, as it can be difficult to make sharp distinctions.

Ablations

Importance of Multiple Knowledge Sources. One benefit of the KIF module approach is that several modules can be combined, each capturing information from a different source. In both settings, Wizard of Wikipedia and Engaging ImageChat, two modules were used to incorporate multiple forms of knowledge -training utterances to capture the capability of a retrieval-based model and knowledge from Wikipedia or related chats based on image features. We perform here an ablation study to evaluate the impact of using only one source of information. As can be seen in Table 4, performance decreases when only one source of information is used (see Table 4).

For Engaging ImageChat, this study also underlines the importance of being able to fetch in a multi-modal fashion. The general form of the KIF module -requiring only a feature vector to find nearest neighbors from -allows fetching on multiple modalities such as text and images. In Table 4 text with the knowledge to identify relevant pieces of information. In Table 5, we show that matching on more features can improve the quality of the retrieved information. Using only the encoding of the immediate previous utterance can improve results on Wizard of Wikipedia by 7 F1 points, but this is further improved by also leveraging the encoding of context (+1.8 F1) and using the dialog turn number (+1 F1). These features are available in the datasets, and we leverage them to improve the relatedness of retrieved knowledge.

Multi-Hop Retrieval with KIF. Work in memory networks [START_REF] Weston | Memory Networks[END_REF][START_REF] Sukhbaatar | End-to-End Memory Networks[END_REF] employed multi-hop mechanisms. be useful when multiple sources are necessary or information is incrementally fetched. To emulate multi-hop memory mechanisms, we use KIF to retrieve relevant information for N = 2 or N = 3 fixed hops. As the number of hops is fixed, the multi-hop operation remains differentiable. We do not allow the model to retrieve the same information in a second hop. We experimented in two settings. First, the same KIF module is used multiple times to fetch different information, and then all of the fetched knowledge is concatenated. Results are shown in Table 6 (top). Second, we examine spreading the fetches into different KIF modules at various encoder depths. This could be interpreted as the model learning to access more information each layer. As the model progresses deeper, more abstract and high level representations are built, which could allow different knowledge to be retrieved. Results are shown in Table 6 (bottom).

In both multi-hop settings, no improvement in performance on the Wizard of Wikipedia dataset is observed. We hypothesize this can be partially attributed to the construction of the dataset -as humans explicitly based their written dialog utterance on one knowledge sentence. Further, it is possible that concatenation brings together too much information for the model to incorporate, and thus adding additional fetches makes the retrieval more noisy.

Effect of Gating. We analyze the effect of the gating mechanism by evaluating the capability of the gate to identify and focus on salient information. On Wizard of Wikipedia, we concatenate a third source of information: dialog turns from a completely different corpus called PersonaChat [START_REF] Zhang | Personalizing Dialogue Agents: I have a dog, do you have pets too[END_REF]. This dataset looks quite different -short utterances without factual knowledge -and should be easy for the model to identify as distinct from Wizard of Wikipedia. As shown in Figure 6(b), if KIF on PersonaChat is included without gating, it has a harmful effect as the model includes irrelevant information. When equipped with gating, the model learns to use the gate to ignore some inputs, and can recover almost the full performance of a model without this irrelevant information source.

Size of K in KNN. Figure 6(c) shows the performance on Wizard of Wikipedia when varying the amount of knowledge. Being able to access multiple relevant pieces of information is helpful, but too much information can be harmful. This is likely because the weighted sum becomes blurry if too many sentences are incorporated.

Conclusion

We present a KNN-based Information Fetching module that learns to identify relevant information from external knowledge sources by learning a mapping-based read operation. KIF modules benefit from the scalability and efficiency of K Nearest Neighbors search, enabling computation with large external memories. We show in the context of two dialog datasets that relevant knowledge can be identified and incorporated to create more engaging, high quality dialog.

Figure 2 :

 2 Figure 2: Human Evaluation Results on both Datasets. More than 50% indicates the KNN Model is preferred. Stars indicate statistical significance at p < 0.05.

Figure 3 :

 3 Figure 3: Human Evaluation on the Unseen Test set of Wizard of Wikipedia. More than 50% indicates the KNN Model is preferred. Stars indicate statistical significance at p < 0.05.

Figure 4 :Figure 5 :

 45 Figure 4: Conversation between Human and KIF-Augmented Transformer on Wizard of Wikipedia. The top-scoring Wikipedia knowledge and training utterances fetched by KIF are displayed with model output.

  Figure 4, and show the top scoring Wikipedia knowledge and Training Utterance fetched by KIF modules.

Figure 6 :

 6 Figure 6: Ablations on Wizard of Wikipedia. (a) KIF can scale to hundreds of relevant sentences (blue) while the baseline model, the Generative Transformer MemNet (gray), scales poorly (b) Gating can remove irrelevant information. In the 3 Sources case, one source of external information is unrelated. (c) Performance as k varies.

Table 1 :

 1 Chat on Similar Images. While there are over 180K different images in this dataset, many of the images are similar. For example, conversations associated with two pictures of dogs could be relevant to each other. The model is able to use KIF directly on the current image features to fetch from around 180K different images and return 6 turns of related chat for each fetched image.Fetching from E 1 consists of identifying related image chats, or conversations on related topics. • E 2 is Training Utterances. Similar to the motivation for the previous dataset, we allow the model to identify training utterances that could be useful for responding in the current conversation. The scale of this fetching task is large: 350K dialog utterances. This could be interpreted as identifying utterances with similar structure to what the model would like to generate, and is complementary to the topic-based related image chats. Results on the Wizard of Wikipedia dataset. We implement the Retrieve and Refine and Response Generation with MR approaches, all with Reddit Pre-Training, and evaluate them on Wizard of Wikipedia. The Seen test set consists of conversations on topics seen at training time, and the Unseen test set consists of conversations about new topics that were not in the training set.

	Model	Test F1	Test F1
		(Seen) (Unseen)
	Retrieval Baselines
	Retrieval Transformer MemNet (Dinan et al., 2018)	15.4	12.4
	Generative Baselines
	2-Stage Generative MemNet (Dinan et al., 2018)	18.9	17.4
	Generative Transformer MemNet (Dinan et al., 2018)	16.9	14.4
	+ Reddit Pre-Training	17.6	16.3
	Retrieve and Refine (Weston et al., 2018)	18.2	17.9
	Response Generation with MR (Qin et al., 2019)	17.5	16.8
	KIF-Augmented Transformer	25.9	22.3

Table 2 :

 2 Results on the Engaging ImageChat dataset. We implement the Generative Transformer Memory Network, Retrieve and Refine, and Response Generation with MR approaches, all with Reddit Pre-Training, and evaluate them on Engaging ImageChat.

  , we display for the same dialog history, changing the model's fetched training utterance and knowledge sentence for our own examples. The model heavily incorporates our manual changes of the fetched information into the generated utterance. For example, changing the knowledge directly affects what the model generates as the favorite characterfrom buzz lightyear to mr potato head to slinky dog -

	Knowledge	Training Utterance	Generation
	buzz lightyear's name is in honor of	my favorite character in that book	cool! my favorite character in that
	astronaut edwin 'buzz' aldrin	series is hermione granger	movie is buzz lightyear
	mr potato head is based on the real-life	my favorite character in that book	my favorite character in that movie is
	mr. potato head toy	series is hermione granger	real-life mr potato head
	slinky dog is a toy dachschund with a	my favorite character in that book	cool! my favorite character is the
	metal slinky for a body	series is hermione granger	slinky dog
	slinky dog is a toy dachschund with a	i really like the character hermione	cool! i really like slinky dog
	metal slinky for a body	granger
	slinky dog is a toy dachschund with a	my favorite character of all time has	i love that movie, my favorite charac-
	metal slinky for a body	to be hermione granger	ter has to be slinky dog the dachshund
	slinky dog is a toy dachschund with a	i agree with you! that's my favorite	i think so too! my favorite is slinky
	metal slinky for a body	character as well

Table 3 :

 3 Effect of Fetched Information on Generated Utterances. The top section provides examples for a fixed training utterance, changing the knowledge -the generated text maintains the construction of the training utterance but changes the favorite character to match the knowledge. The bottom section provides examples for fixed knowledge but changing the training utterance -the generated text modifies its form to match the training utterance, but the favorite character information remains consistent.

Table 4 :

 4 , using the Image-based KIF to fetch text from Related Images is important to reach the strongest performance (compare Training Utterances Only that uses text-based KIF and using both Training Utterances and Related Images). dialog Features for KNN Performance. The quality of the KNN search is critical to the performance of KIF modules. As the external knowledge is kept fixed, KIF must be able to align the dialog con-Using Multiple KIF Modules on Multiple Sources is important for improved performance.

	Using

Table 5 :

 5 Important Features for KNN Search using KIF. Salient conversation features improve performance on both datasets.

Table 6 :

 6 Multi-hop with KIF to retrieve information with multiple fetch steps.

	Such capacity could
	Model	Valid F1
	KIF-Augmented Transformer	27.4
	One KIF Module fetches multiple times
	2 Fetches	26.9
	3 Fetches	26.0
	Multiple KIF Modules fetch once each
	2 Fetches	26.5
	3 Fetches	25.9

[START_REF] Shuster | Image-chat: Engaging Grounded Conversations[END_REF], retrieval Transformer models report Hits@N using a fixed candidate set of 99 distractor candidates and 1 true candidate. We compute F1 using their open-sourced model by scoring the entire training set of over 350K utterances with the model and taking the top scoring candidate as the response.

Acknowledgements

We thank the reviewers and action editor for their comments and insightful discussion. We thank Emily Dinan and Kurt Shuster for providing assistance to reproduce their original works.


Mattia Melosso
email: mattia.melosso2@unibo.it

Luca Bizzocchi

Aleksandra Adamczyk

Elisabetta Cané

Paola Caselli

Laura Colzi

Luca Dore

Barbara M Giuliano

J.-C Guillemin

Elisabetta Canè

Jean-Claude Guillemin

Marie-Aline Martin-Drumel

Olivier Pirali

Andrea Pietropolli Charmet

Domenico Prudenzano

Víctor M Rivilla

Filippo Tamassia
email: filippo.tamassia@unibo.it

Extensive ro-vibrational analysis of deuterated-cyanoacetylene (DC3N) from millimeter-wavelengths to the infrared domain

Keywords: Cyanoacetylene, Interstellar species, Ro-vibrational spectroscopy, Spectral analysis, Anharmonic resonances, Line catalog

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Highly unsaturated molecules account for a large portion of the known interstellar species [START_REF] Mcguire | Census of Interstellar, Circumstellar, Extragalactic, Protoplanetary Disk, and Exoplanetary Molecules[END_REF]. For instance, the presence of several carbon-chain molecules is one of the most characteristic features of the chemical composition of starless cores, such as the Taurus Molecular Cloud (TMC-1), one of the brightest source of carbon-chain species [START_REF] Yamamoto | Introduction to Astrochemistry: Chemical Evolution from Interstellar Clouds to Star and Planet Formation[END_REF]. Among the unsaturated molecular species, cyanopolyynes, i.e., linear molecules of general chemical formula HC 2n+1 N, are widespread in the interstellar medium (ISM) and all members up to HC 11 N have been detected to date [START_REF] Loomis | The third time's a charm? a rigorous investigation of spectral line stacking techniques and application to the detection of HC 11 N[END_REF]. Cyanoacetylene (HC 3 N, IUPAC name prop-2ynenitrile), the simplest member of the cyanopolyynes family, was found to be an abundant species in a large variety of astronomical objects: starless cores [START_REF] Suzuki | A survey of CCS, HC 3 N, HC 5 N, and NH 3 toward dark cloud cores and their production chemistry[END_REF], post-AGB objects [START_REF] Wyrowski | Physical conditions in the proto-planetary nebula CRL 618 derived from observations of vibrationally excited HC 3 N[END_REF], carbon-rich circumstellar envelopes [START_REF] Decin | Warm water vapour in the sooty outflow from a luminous carbon star[END_REF], massive star-forming regions [START_REF] Li | Large-scale kinematics, astrochemistry, and magnetic field studies of massive star-forming regions through HC 3 N, HNC, and C 2 H mappings[END_REF], protoplanetary disks [START_REF] Chapillon | Chemistry in disks. VII. first detection of HC 3 N in protoplanetary disks[END_REF], solar-type protostars [START_REF] Al-Edhari | History of the solar-type protostar IRAS 16293-2422 as told by the cyanopolyynes[END_REF], external galaxies [START_REF] Rico-Villas | Super hot cores in NGC 253: witnessing the formation and early evolution of super star clusters[END_REF], and Galactic Center molecular clouds [START_REF] Zeng | Complex organic molecules in the Galactic Centre: the N-bearing family[END_REF].

The deuterated form of cyanoacetylene (DC 3 N) has been detected in the ISM as well. The first astronomical observation of DC 3 N has been reported towards TMC-1 [START_REF] Langer | Detection of deuterated cyanoacetylene in the interstellar cloud TMC-1[END_REF] by the detection of the J = 5 → 4 emission rotational transition around 42 GHz. Consecutively, DC 3 N has tentatively been detected in the high-mass star-forming regions Orion KL [START_REF] Esplugues | Combined IRAM and Herschel/HIFI study of cyano(di)acetylene in Orion KL: tentative detection of DC 3 N[END_REF] and Sagittarius B2 [START_REF] Belloche | Exploring molecular complexity with ALMA (EMoCA): Deuterated complex organic molecules in Sagittarius B2 (N2)[END_REF]. In these regions, deuterium fractionation is not as effective as in dark clouds, thus preventing a strong enhancement above the deuterium cosmic abundance. Recently, DC 3 N has been detected in some low-mass cores (see e.g., Refs. [START_REF] Al-Edhari | History of the solar-type protostar IRAS 16293-2422 as told by the cyanopolyynes[END_REF], [START_REF] Bianchi | Astrochemistry as a tool to follow protostellar evolution: The class I stage[END_REF]) and in a sample of 15 highmass star-forming cores [START_REF] Rivilla | DC 3 N observations towards high-mass star-forming regions[END_REF]. The latter work, based on the spectroscopic results presented in this paper, suggests that DC 3 N is enhanced in the cold and outer regions of star-forming regions, likely indicating the initial deuteration level of the large-scale molecular cloud within which star formation takes place. Rivilla et al. [START_REF] Rivilla | DC 3 N observations towards high-mass star-forming regions[END_REF] also summarize all the astronomical observations of DC 3 N so far.

Microwave (MW) transitions of DC 3 N were first reported for the ground and the four lowest singlyexcited states during the course of an extensive study of cyanoacetylene isotopologues [START_REF] Mallinson | The microwave spectrum of cyanoacetylene in ground and excited vibrational states[END_REF]. A larger number of vibrationally excited states was re-examined in depth some years later and a rigorous determination of the effective molecular parameters was attained [START_REF] Plummer | Rotational spectra of DC 3 N in some excited vibrational states[END_REF]. Recently, the laboratory investigation of the rotational spectrum of DC 3 N has been extended to the THz regime for the ground and the v 7 = 1 states [START_REF] Spahn | Rotational spectra and hyperfine structure of isotopic species of deuterated cyanoacetylene, DC 3 N[END_REF]. In the same paper, the authors revised the 14 N and D hyperfine-structure constants derived in Refs. [START_REF] Fliege | Deuterium and nitrogen quadrupole coupling in cyanodeuteroacetylene[END_REF][START_REF] Tack | Beam-maser spectroscopy on cyanoacetylene-D[END_REF] from supersonic-jet Fourier-Transform Microwave (FT-MW) spectroscopy.

As far as its infrared (IR) spectrum is concerned, the experimental position and intensity of all fundamentals but the weak ν 4 mode have been determined from low resolution (0.5 cm -1 ) studies [START_REF] Uyemura | Infrared intensities of bending fundamentals in gaseous HCCCN and DCCCN[END_REF][START_REF] Bénilan | IR band intensities of DC 3 N and HC 15 3 N: Implication for observations of Titan's atmosphere[END_REF]. Some combination and overtone bands were also observed in the same works. In addition, two medium resolution (0.025-0.050 cm -1 ) IR studies were performed by Mallinson & Fayt [24] and Coveliers et al. [START_REF] Coveliers | Far-infrared spectra of DCCCN[END_REF]. In the former, the band center of the three stretching modes of DC 3 N (ν 1 , ν 2 , and ν 3 ) has been determined; in the latter, the far-infrared (FIR) spectrum was recorded between 200 and 365 cm -1 and the ν 7 fundamental was analyzed together with the bands ν 6 -ν 7 , ν 5 -ν 7 , and ν 4 -ν 6 , and their hot-bands.

In this work, a detailed investigation of both millimeter/submillimeter-wave and infrared spectra of DC 3 N is reported. Pure rotational transitions within all the vibrational states with energy lower than 1015 cm -1 have been detected and 27 fundamental, overtone, combination, and hot ro-vibrational bands have been analyzed at high resolution (0.001-0.01 cm -1 ). The new measurements have been combined in a fit containing almost 6700 distinct transition frequencies, thus allowing the determination of a consistent set of spectroscopic parameters. This work represents the most exhaustive spectroscopic characterization of DC 3 N so far and provides a robust line catalog useful for astronomical applications. Moreover, the large number of vibrational excited states are of interest for harmonic/anharmonic force field computations.

The paper is structured as follows. First, the synthesis of the sample and the spectrometers used for spectral recording are described ( §2). Then, the effective Hamiltonian employed for the energy levels description is given ( §3). Successively, the general features of the spectra and their analysis are discussed ( §4). Finally, the results are summarized and the conclusions are presented ( §5).

Experimental details

Synthesis of deuterocyanoacetylene

Methyl propiolate (HC CCOOCH 3 ) was purchased from TCI-Europe and used without further purification. The DC 3 N sample was synthesized in Rennes following the procedure described in Ref. [START_REF] Bénilan | IR band intensities of DC 3 N and HC 15 3 N: Implication for observations of Titan's atmosphere[END_REF]. Briefly, HC CCOOCH 3 was added dropwise to liquid ammonia resulting in a 100 % conversion into HC CCONH 2 . The propiolamide was then mixed with phosphorous anhydride (P 4 O 10 ) and calcined white sand; the whole system was heated up to 470 K over 2 h while connected to a liquid nitrogen-cooled trap where pure cyanoacetylene was collected. Cyanoacetylene (3 g), heavy water (D 2 O, 4 mL) and potassium carbonate (K 2 CO 3 , 50 mg) were mixed together in an inert atmosphere. The biphasic mixture was then stirred for about 20 min at room temperature. Subsequently, on a vacuum line, partially deuterated cyanoacetylene was condensed in a 77 K cooled trap, while water was blocked in a first 220 K trap. The operation was repeated 3 times by addition of D 2 O and K 2 CO 3 to the partially deuterated cyanoacetylene. The residual D 2 O was removed by vaporisation on P 4 O 10 and DC 3 N was finally condensed in a trap cooled to 150 K. Deuterocyanoacetylene with an isotopic purity greater than 98 % was obtained in a 67 % yield. The sample can be stored indefinitely at 250 K without decomposition.

Infrared spectrometers

The FIR spectrum of DC 3 N was recorded at the AILES beamline of the SOLEIL synchrotron facility using a Bruker IFS 125 FT interferometer [START_REF] Brubach | Performance of the AILES THz-Infrared beamline at SOLEIL for High resolution spectroscopy[END_REF] and a white-type multipass absorption cell whose optics were adjusted to obtain a 150 m optical path length [START_REF] Pirali | Rotationally resolved infrared spectroscopy of adamantane[END_REF][START_REF] Pirali | The far infrared spectrum of naphthalene characterized by high resolution synchrotron FTIR spectroscopy and anharmonic DFT calculations[END_REF]. For the present experiment, we used the far-IR synchrotron radiation continuum extracted by the AILES beamline. The interferometer was equipped with a 6 µm Mylar-composite beamsplitter and a 4 K cooled Si-bolometer. Two 50 µm-thick polypropylene windows isolated the cell from the interferometer, which was continuously evacuated to 0.01 Pa limiting the absorption of atmospheric water. Vapor of DC 3 N was injected into the absorption cell at a 25 Pa pressure. The spectrum covers the range 70-500 cm -1 and consists of the co-addition of 380 scans recorded at 0.00102 cm -1 resolution.

IR spectra in the 450-1600 cm -1 range were recorded in Bologna using a Bomem DA3.002 Fourier-Transform spectrometer [START_REF] Tamassia | Spectroscopy of a low global warming power refrigerant. infrared and millimeter-wave spectra of trifluoroethene (HFO-1123) in the ground and some vibrational excited states[END_REF]. It was equipped with a Globar source, a KBr beamsplitter, and a liquid nitrogen-cooled HgCdTe detector. A multi-pass cell with absorption-lengths from 4 to 8 m was employed for the measurements. Sample pressures ranging between 25 and 650 Pa were used to record the spectra. The resolution was generally 0.004 cm -1 , except for the very weak ν 4 band, which was recorded at a lower resolution of 0.012 cm -1 . Several hundreds of scans, typically 800, were co-added in order to improve the signal-to-noise ratio (S/N) of the spectra.

All the spectra have been calibrated using residual water or CO 2 absorption lines whose reference wavenumbers were taken from Refs. [START_REF] Matsushima | Frequency-measurement of pure rotational transitions of H 2 O from 0.5 to 5 THz[END_REF][START_REF] Horneman | Transferring calibration from CO 2 laser lines to far infrared water lines with the aid of the ν 2 band of OCS and the ν 2 , ν 1 -ν 2 , and ν 1 + ν 2 bands of 13 CS 2 : Molecular constants of 13 CS 2[END_REF] and from HITRAN [START_REF] Gordon | The HITRAN2016 molecular spectroscopic database[END_REF], respectively. No apodization functions were applied to the interferograms.

Millimeter and submillimeter spectrometers

Rotational spectra have been recorded using two frequency-modulation (FM) millimeter/submillimeter spectrometers located in Bologna and in Garching.

The Bologna spectrometer has been described in details elsewhere [START_REF] Degli Esposti | Determination of a semi-experimental equilibrium structure of 1-phosphapropyne from millimeter-wave spectroscopy of CH 3 CP and CD 3 CP[END_REF][START_REF] Melosso | The sub-millimeter rotational spectrum of ethylene glycol up to 890 GHz and application to ALMA Band 10 spectral line data of NGC 6334I[END_REF]. Briefly, a Gunn diode oscillator operating in the W band (80-115 GHz) was used as primary source of radiation, whose frequency and phase stability are ensured by a Phase-Lock Loop (PLL). Spectral coverage at higher frequencies was obtained by coupling the Gunn diode to passive frequency multipliers in cascade (doublers and triplers, Virginia Diodes, Inc.). The output radiation, sine-wave modulated in frequency (f = 48 kHz), was fed to the glass absorption cell containing DC 3 N vapors at a pressure between 1 and 15 Pa, depending on the intensity of the lines under consideration. The outcoming signal was detected by a Schottky barrier diode and sent to a Lock-in amplifier set at twice the modulation-frequency (2f scheme); the demodulated signal is then filtered into a resistor-capacitor (RC) system before data acquisition.

In Garching the CASAC spectrometer developed at the Max-Planck-Institut für extraterrestrische Physik was used. Full details on the experimental set-up are given in Ref. [START_REF] Bizzocchi | Accurate sub-millimetre rest frequencies for HOCO + and DOCO + ions[END_REF]; here, we report only a few key details which apply to the present investigation. The instrument is equipped with an active multiplier chain (Virginia Diodes) as a source of radiation in the 82-125 GHz band. Further multiplier stages in cascade allow to extend the frequency coverage up to ∼ 1.1 THz with an available power of 2-20 µW. The primary millimeter radiation stage is driven by a cm-wave synthesizer (Keysight E8257D) operating in the 18-28 GHz band, which is locked to a Rb atomic clock to achieve accurate frequency and phase stabilisation. A closedcycle He-cooled InSb hot-electron bolometer operating at 4 K (QMC) is used as a detector. As in Bologna, frequency (f = 50 kHz) modulation technique is employed and the second derivative of the actual absorption profile is thus recorded by the computer-controlled acquisition system after lock-in demodulation at 2f . The absorption cell is a plain Pyrex tube (3 m long and 5 cm in diameter) fitted with high-density polyethylene windows. The measurement were performed using gaseous samples at pressure of a few Pa. In this condition, DC 3 N is stable for ca. 2 h without significant decomposition due to hydrogen exchange.

The spectra were recorded in the frequency ranges 80-115 GHz and 920-1070 GHz in Garching, and in the window 240-440 GHz in Bologna. [a] From low-resolution integrated band-intensity measurements at 296 K (Ref. [START_REF] Bénilan | IR band intensities of DC 3 N and HC 15 3 N: Implication for observations of Titan's atmosphere[END_REF]).

[b] From low-resolution integrated band-intensity measurements at 293 K (Ref. [START_REF] Uyemura | Infrared intensities of bending fundamentals in gaseous HCCCN and DCCCN[END_REF]).

Theoretical background

From a spectroscopic point of view, DC 3 N is a closed-shell linear rotor. It has 7 vibrational modes: 4 stretchings (ν 1 -ν 4 ; Σ symmetry) and 3 doubly-degenerated bendings (ν 5 -ν 7 ; Π symmetry). They are summarized in Table 1. In the present work, only the low-lying vibrational states (v 4 , v 5 , v 6 , and v 7 , with one of multiple quanta of excitation) have been investigated for two main reasons: (i) transitions associated to the lower energy states are of astrophysical interest, and (ii) some of the vibrational states are connected by a network of anharmonic resonances fully described within our chosen energy threshold of 1015 cm -1 ; above this limit the states are either unperturbed or involved in higher-order resonances. Therefore, the stretching modes ν 1 , ν 2 , and ν 3 , lying above this threshold, have not been investigated. Conventionally, we labelled a given vibrational state with the notation (v 4 , v l5 5 , v l6 6 , v l7 7 ) e/f , where l t is the vibrational angular momentum quantum number associated to the bending mode t and the e/f subscripts indicate the parity of the symmetrized wave functions [START_REF] Brown | The labeling of parity doublet levels in linear molecules[END_REF]. When the l t and e/f labels are not indicated, we refer to all the possible sub-levels of a state.

The full ro-vibrational wave-function is then given by the ket

v 4 , v l5 5 , v l6 6 , v l7 7 ; J, k e/f
. The vibrational part of the wave-function is expressed as combination of one-or two-dimensional harmonic oscillators, whereas the rotational part is the symmetric-top wave-function whose quantum number k is given by k = l 5 + l 6 + l 7 . A substate is denoted as Σ when k = 0, Π for |k| = 1, ∆ for |k| = 2, and so on.

The following Wang-type linear combinations [START_REF] Yamada | Effective Hamiltonian for polyatomic linear molecules[END_REF] lead to symmetry-adapted basis functions:

v 4 , v l5 5 , v l6 6 , v l7 7 ; J, k e/f = 1 √ 2 v 4 , v l5 5 , v l6 6 , v l7 7 ; J, k ± (-1) k v 4 , v -l5 5 , v -l6 6 , v -l7 7 ; J, -k , (1a) v 4 , 0 0 , 0 0 , 0 0 ; J, 0 e = v 4 , 0 0 , 0 0 , 0 0 ; J, 0 . (1b)
The upper and lower signs (±) correspond to e and f wave-functions, respectively. For Σ states (k = 0), the first non-zero l t is chosen positive. Here, the omission of the e/f label indicates unsymmetrised wavefunctions. The Hamiltonian used to reproduce the ro-vibrational energy levels is equivalent to the one used for HC 3 N [START_REF] Bizzocchi | Rotational and high-resolution infrared spectrum of HC 3 N: Global ro-vibrational analysis and improved line catalog for astrophysical observations[END_REF]:

H = H rv + H l-type + H res , (2)
where H rv is the ro-vibrational energy including centrifugal distortion corrections, H l-type represents the l-type interaction between the l sub-levels of the excited bending states, and H res accounts for resonances among accidentally quasi-degenerate ro-vibrational states. The resonance network active in DC 3 N resembles the one found for HC 3 N and will be described later.

The Hamiltonian matrix is built by using unsymmetrised ro-vibrational functions. It is subsequently factorized and symmetrized using Eqs. [START_REF] Mcguire | Census of Interstellar, Circumstellar, Extragalactic, Protoplanetary Disk, and Exoplanetary Molecules[END_REF]. The matrix elements of the effective Hamiltonian are expressed using the formalism already employed for the analysis of HC 3 N [START_REF] Bizzocchi | Rotational and high-resolution infrared spectrum of HC 3 N: Global ro-vibrational analysis and improved line catalog for astrophysical observations[END_REF].

General features and analysis

Vibrational spectra

Although infrared spectra were recorded up to 1600 cm -1 in this study, our analysis is limited to the portion of the electromagnetic spectrum below ∼ 1040 cm -1 . This is because the highest energy state within our threshold of 1015 cm -1 is the (0110) state, whose combination band falls in the region 999-1035 cm -1 . In total, 27 ro-vibrational bands have been observed at high resolution for the first time and successfully analyzed. They include fundamental, overtone, combination, and hot-bands, and are listed in Table 2 along with the observed sub-bands, frequency and J ranges, number of data used in the analysis, and the rootmean-square (rms) error of the final fit. All the observed bands are also graphically displayed in Figure 1.

Figure 2 shows a general overview of portions of the FIR (180-460 cm -1 range, upper panel) and midinfrared (MIR, 450-800 cm -1 range, bottom panel) spectra recorded in this work. The most prominent bands in the FIR region are the ν 7 fundamental, ν 6 -ν 7 , ν 5 -ν 7 , ν 4 -ν 6 , and 2ν 7 overtone bands. The MIR region is dominated by the very strong fundamentals ν 6 and ν 5 . The low-frequency side of the spectrum is particularly crowded due to the proximity of the two fundamentals, the presence of their associated hotbands, and of the ν 6 of HC 3 N centered at 500 cm -1 . HC 3 N is present in the sample as result of the H/D exchange in the cell.

Having a medium IR intensity, the combination bands ν 6 + ν 7 and ν 5 + ν 7 are well visible in the highfrequency part of the MIR spectrum as seen in the bottom panel of Figure 2. Although not displayed in Figure 2, the overtone 2ν 6 and the combination ν 5 + ν 6 bands centered around 975-1018 cm -1 and 999-1035 cm -1 , respectively, are clearly detectable as well, despite the presence of strong absorption lines due to HDO. The very weak (< 0.1 atm -1 cm -2 ) ν 4 fundamental at 830-865 cm -1 had to be recorded at higher pressure (400 Pa) and lower resolution (0.012 cm -1 ). In this case, up to 2600 scans were co-added to improve the S/N of the spectrum.

Rotational spectra

Rotational spectra were recorded for all the 14 states whose vibrational energy do not exceed our threshold of 1015 cm -1 . Literature data were available for some of these states, as pointed out in Section 1. However, line positions of some millimeter-wave transitions from Ref. [START_REF] Mallinson | The microwave spectrum of cyanoacetylene in ground and excited vibrational states[END_REF] are affected by large uncertainties (up to 300 kHz) and many data are limited to low frequencies. For these reasons, we decided to re-investigate and extend the spectrum for all these vibrational states. The largest improvements have been realized for the states (1000), (0110), (0020), and (0004) involved in a network of anharmonic resonances, for which extended data-sets were obtained. In particular, the (0110) state, not included in the analysis of Ref. [START_REF] Plummer | Rotational spectra of DC 3 N in some excited vibrational states[END_REF], has been assigned for the first time in this study and its interaction with the (1000) state has been identified and properly accounted for.

Table 3 summarizes the set of rotational data used in the analysis, specifying the observed sub-levels, J and frequency ranges, number of distinct fitted frequencies, the rms error of the final fit, and the corresponding references used.

With the exception of v 4 = 1, all the states possess a rotational constant B greater than that of the ground state and therefore their rotational lines lie at frequencies higher than those of the corresponding 3, where the broad scan covers the J = 13 ← 12 transitions for many vibrational satellites. In this excerpt, the l-type resonance patterns of all the excited bending states analyzed are visible. From a visual inspection, it is easy to associate some of these patterns to the pertaining state: the ground and v 4 = 1 exhibit a single line, while each bending state has = t (l t +1) lines (even though not always resolvable).

Analysis of the spectra

The sample of pure rotational and ro-vibrational data contains 6691 distinct frequencies involving 14 vibrational states of DC 3 N. This work represents the first-ever investigation of its ro-vibrational spectrum in the region between 365 and 1040 cm -1 . Moreover, the FIR spectrum has been thoroughly re-investigated at higher resolution with an accuracy two or three orders of magnitude better than Ref. [START_REF] Coveliers | Far-infrared spectra of DCCCN[END_REF]. As far as the rotational spectrum is concerned, this work extends the observation of excited states transitions to the submillimeter-wave region. In addition, rotational transitions with J up to 126 were recorded at THz frequencies (1.069 THz) although only for the ground state.

In the combined fit, a different weight was given to each datum in order to take into account the different measurements precision. Uncertainties spanning from 0.0004 to 0.00075 cm -1 were used for the infrared measurements performed in Bologna; the weak ν 4 band being the only exception, for which an uncertainty of 0.001 cm -1 was assumed. FIR data recorded at higher resolution with the FT-IR spectrometer of the AILES beamline have been given uncertainties between 0.00005 and 0.0001 cm -1 , based on calibration residuals and the S/N of spectral lines. As far as pure rotational transitions are concerned, we assumed a typical experimental error of 10-20 kHz for our new millimeter/submillimeter measurements. Data from literature were used with the uncertainty stated in the original papers [START_REF] Mallinson | The microwave spectrum of cyanoacetylene in ground and excited vibrational states[END_REF][START_REF] Plummer | Rotational spectra of DC 3 N in some excited vibrational states[END_REF][START_REF] Spahn | Rotational spectra and hyperfine structure of isotopic species of deuterated cyanoacetylene, DC 3 N[END_REF]. Only few lines from Ref. [START_REF] Mallinson | The microwave spectrum of cyanoacetylene in ground and excited vibrational states[END_REF], whose residuals were far off their declared errors, were not used in the fit.

The spectral analysis was performed using a custom PYTHON code that employs the SPFIT program [START_REF] Pickett | The fitting and prediction of vibration-rotation spectra with spin interactions[END_REF] as computational core (see Ref. [START_REF] Bizzocchi | Rotational and high-resolution infrared spectrum of HC 3 N: Global ro-vibrational analysis and improved line catalog for astrophysical observations[END_REF] for further details about the code). The data were fitted to the Hamiltonian of Eq. ( 2) and its coefficients optimized in an iterative least-squares procedure. Some spectroscopic parameters could not be determined from the available experimental data. In these cases, the constant of a given vibrational level were derived from the corresponding optimized values obtained for other levels belonging to the same vibrational manifold considering, whenever feasible, a vibrational dependence.

In other cases, they were simply fixed to zero. The spectroscopic parameters obtained from the combined fit procedure are collected in Tables 4567.

As anticipated, the analysis of DC 3 N follows the approach successfully adopted for HC 3 N [START_REF] Bizzocchi | Rotational and high-resolution infrared spectrum of HC 3 N: Global ro-vibrational analysis and improved line catalog for astrophysical observations[END_REF]. The main difference is the set-up of the anharmonic resonances network, which arises from the different energy of some vibrational levels due to the isotopic substitution. In particular, the ν 5 vibrational energy, 663.36848(3) cm -1 in HC 3 N, drops to 522.26378(2) cm -1 in DC 3 N. For HC 3 N, two resonant systems were described: (i)

v 5 = 1 ∼ v 7 = 3 and ii) v 4 = 1 ∼ v 5 = v 7 = 1 ∼ v 6 = 2 ∼ v 7 = 4.
Of the two systems, the former is not present in DC 3 N while the latter is almost the same, except for v 5 = v 7 = 1, replaced by v 5 = v 6 = 1. The treatment of such perturbations led to the determination of the corresponding interaction parameters, C 30 for the cubic Table 5: Spectroscopic constants derived for DC 3 N in singly-excited bending states.

Constant Unit Moreover, a centrifugal distortion parameter C J 50 was included in the analysis. For the states involved in this resonance system, many experimental data are available. In the MIR region, we recorded the ν 4 , ν 5 +ν 6 , and 2ν 6 bands that provide the energy position for most of the interacting levels. The energy of the v 7 = 4 was determined through the FIR spectrum, where the 4ν 7 ← 3ν 7 hot-band and the 4ν 7 ← ν 6 band were detected. A large pure rotational data-set is also available for the polyad of interacting states. Besides several rotational transitions observed within the vibrational states, a small set of interstate transitions between the (1000) and (0004) states were identified. The coefficients C mn of the resonance Hamiltonian are given in Table 8.

v 7 = 1 v 6 = 1 v 5 = 1 G v cm -

Conclusions

In this work, a large set of high-resolution rotational and ro-vibrational data of DC 3 N has been recorded and analyzed in order to achieve a detailed knowledge of all the vibrational states approximately below 1000 cm -1 of energy. To reach this goal, infrared spectra of DC 3 N have been recorded in the range 150-1600 cm -1 at high resolution (0.001-0.004 cm -1 ). In this region, 27 fundamental, overtone, combination, and hot-bands have been observed and analyzed. Notably, the very weak ν 4 fundamental has also been detected, even though at lower resolution (0.012 cm -1 ). Also, pure rotational transitions for 14 states have been recorded to extend the investigation of the spectrum to the submillimeter-wave region up to ca. 500 GHz.

Almost 6700 experimental transitions were included in a least-squares fit procedure thanks to which a large number of rotational and ro-vibrational spectroscopic parameters have been determined for 14 different vibrational states. The whole set of data has been fitted with an overall weighted standard deviation σ of 0.95, meaning that on average all data are well-reproduced within their given uncertainties. The vibrational energies were determined experimentally for all the investigated states, without any assumption. The combination of both high-resolution ro-vibrational data and pure rotational measurements allowed an accurate modeling of the spectrum of DC 3 N, including perturbations produced by the observed anharmonic resonances. The interaction between the (1000) and (0110) states has been introduced for the first time, with the effect to eliminate the residual discrepancies described in Refs. [START_REF] Plummer | Rotational spectra of DC 3 N in some excited vibrational states[END_REF][START_REF] Coveliers | Far-infrared spectra of DCCCN[END_REF].

The present work shows once again the success of a combined analysis of data from different spectral regions, like infrared and millimeter-wave fields. The results are generally more coherent and fewer assumptions are needed, if not any. Also, a more extended set of spectroscopic parameters can be obtained with reliability.

This study provides an extensive line catalog (deposited as Supplementary Material) which can be used to assist future astronomical observations of DC 3 N and is suitable for modeling both cold and hot regions of the interstellar medium.

Constant Unit

v 6 = v 7 = 1 v 5 = v 7 = 1 v 5 = v 6 = 1 v 6 = 1, v 7 = 2 v 5 = 1, v 7 = 2 G v cm -

Acknowledgement

This study was supported by Bologna University (RFO funds) and by MIUR (Project PRIN 2015: STARS in the CAOS, Grant Number 2015F59J3R). This work has been performed under the SOLEIL proposal #20190128; we acknowledge the SOLEIL facility for provision of synchrotron radiation and would like to thank the AILES beamline staff for their assistance.

Figure 1 :

 1 Figure 1: Vibrational energy-level diagram of DC 3 N up to 1015 cm -1 , where the arrows represent the 27 IR bands analyzed in this work. Red and blue arrows indicate the bands observed at SOLEIL and in Bologna, respectively.

Figure 2 :

 2 Figure 2: Portions of the FIR (upper panel) and MIR (lower panel) spectra of d-cyanoacetylene (black traces). A simulation spectrum of the most intense bands is also reported for both DC 3 N (red) and HC 3 N (grey). Lines belonging to CO and various H 2 O isotopologues were removed from the spectra.

Figure 3 :

 3 Figure 3: A 2 GHz broad scan of the J = 13 ← 12 rotational transition of DC 3 N around 110 GHz. The spectrum was recorded at room temperature, with DC 3 N at a pressure of 0.05 Pa, RC= 3 ms, frequency step 50 kHz, FM = 120 kHz, scan speed = 0.4 MHz/s, 2 scans. The arbitrary units of the y-axis are set so that the intensity of the ground state (GS) transition, out of scale in the figure, is 1.

  [b] Kept fixed to ground state value. terms, (v 4 = 1)-(v 6 = 2) and (v 4 = 1)-(v 5 = v 6 = 1), and C 50 for the quintic term (v 4 = 1)-(v 7 = 4).

  Number in parenthesis are one standard deviation in units of the last quoted digit. [a] Constrained value, see text. [b] Kept fixed to ground state value.

  are one standard deviation in units of the last quoted digit.

  L.B., P.C., and B.M.G. acknowledge the support by the Max Planck Society. V.M.R. has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sk lodowska-Curie grant agreement No 664931. LC acknowledges support from the Italian Ministero dell'Istruzione, Università e Ricerca through the grant Progetti Premiali 2012 -iALMA (CUP C52I13000140001). J.-C.G. thanks the Centre National d'Etudes Spatiales (CNES) for a grant.

Table 1 :

 1 Energy and intensity of all fundamental modes of DC 3 N.

	Modes Description	Energy	Reference Abs. intensity
			(cm -1 )		(atm -1 cm -2 )
	ν 1	C D stretching	2608.520(3)	[24]	81.3 ± 5.7 a
	ν 2	C C stretching	2252.155(3)	[24]	50.5 ± 2.4 a
	ν 3	C N stretching	1968.329(3)	[24]	38.7 ± 4.0 a
	ν 4	C C stretching	867.60(6)	This Work	< 0.1 b
	ν 5	CCD bending	522.263933(7) This Work	83.8 ± 4.7 a
	ν 6	CCC bending	492.759896(7) This Work	106. ± 8 a
	ν 7	CCN bending	211.550293(5) This Work	0.89 ± 0.11 b

Table 2 :

 2 Ro-vibrational bands recorded and analyzed in this work.

	Band	Sub-bands	Freq. range J range No. of lines rms × 10 4
			(cm -1 )			(cm -1 )
	ν 7	Π -Σ +	190-240	1-93	258	0.5
	ν 6	Π -Σ +	466-522	2-109	267	3.6
	ν 5	Π -Σ +	500-557	0-117	255	3.7
	ν 4	Σ + -Σ +	830-865	0-61	109	9.6
	2ν 7	Σ + -Σ +	405-445	2-78	136	1.0
	2ν 6	Σ + -Σ +	975-1018	2-101	141	5.2
	ν 6 + ν 7	Σ + -Σ +	686-736	1-89	166	2.5
	ν 5 + ν 7	Σ + -Σ +	715-769	1-105	170	3.0
	ν 5 + ν 6	Σ + -Σ +	999-1035	3-63	102	3.8
	ν 6 -ν 7	Π -Π	257-306	1-89	309	0.6
	ν 5 -ν 7	Π -Π	288-333	1-86	291	0.7
	ν 4 -ν 6	Σ + -Π	329-375	1-79	222	0.6
	4ν 7 -ν 6	Σ + -Π	351-352	59-60	2	0.4
	2ν 7 -ν 7	Σ -Π	193-236	1-78	391	0.8
	3ν 7 -2ν 7	Π -Σ +	191-235	4-77	178	0.7
	4ν 7 -3ν 7	Σ + -Π	193-219	10-58	89	0.9
	3ν 7 -ν 7	Π -Π	405-441	4-68	208	1.0
	ν 6 + ν 7 -ν 7	(Σ, ∆) -Π	478-508	5-56	329	6.9
	ν 6 + 2ν 7 -2ν 7 Π -(Σ + , ∆)	476-512	12-65	93	5.2
	2ν 6 -ν 6	Σ -Π	500-519	10-68	43	7.0
	ν 5 + ν 7 -ν 7	(Σ + , ∆) -Π	505-545	2-75	464	4.5
	ν 5 + 2ν 7 -2ν 7 Π -(Σ, ∆)	505-542	2-82	106	3.5
	ν 6 + 2ν 7 -ν 7	Π -Π	690-718	4-53	153	4.7
	ν 5 + 2ν 7 -ν 7	Π -Π	721-748	2-45	296	4.1
	ν 5 + ν 7 -2ν 7	(Σ + , ∆) -(Σ + , ∆)	290-330	2-81	261	0.9
	ν 6 + ν 7 -2ν 7	(Σ + , ∆) -(Σ + , ∆)	256-306	2-84	402	0.9
	4ν 7 -2ν 7	(Σ + , ∆) -(Σ + , ∆)	406-437	5-67	302	1.0
	ground state transition. This can be seen in Figure

Table 3 :

 3 Summary of the rotational data used in the analysis.

	State	|k|	J range Freq. range No. of lines	rms	Reference
				(GHz)		(kHz)
	Ground state	0	3-126	33-1069	52	13.2	TW, Ma78, Pl88, Sp08
	v 7 = 1	1 e,f	5-105	50-896	67	10.9	TW, Ma78, Pl88, Sp08
	v 6 = 1	1 e,f	7-44	67-381	42	17.0	TW, Pl88
	v 5 = 1	1 e,f	7-44	67-381	42	13.7	TW, Pl88
	v 4 = 1	0	7-51	67-439	32	23.6	TW, Pl88
	v 7 = 2	0, 2 e,f	7-44	67-383	61	24.0	TW, Ma78, Pl88
	v 7 = 3	(1, 3) e,f	7-44	68-384	77	20.0	TW, Ma78, Pl88
	v 7 = 4	0,(2, 4) e,f	7-48	68-419	85	19.8	TW, Pl88
	v 6 = 2	0, 2 e,f	7-44	67-382	54	44.7	TW, Pl88
	v 6 = v 7 = 1	(0, 2) e,f	7-44	67-382	93	30.8	TW, Ma78, Pl88
	v 5 = v 7 = 1	(0, 2) e,f	7-44	67-382	78	20.8	TW, Pl88
	v 5 = v 6 = 1	(0, 2) e,f	9-44	84-381	63	14.6	TW
	v 6 = 1, v 7 = 2 (±1, 3) e,f	7-46	68-400	95	21.9	TW, Pl88
	v 5 = 1, v 7 = 2 (±1, 3) e,f	9-44	85-383	97	17.4	TW
	interstate a		44-49	364-429	10	19.7	TW

Abbreviations are used as follow: TW This work, Ma78 Mallinson & De Zafra (1978) [17], Pl88 Plummer et al. (1988) [18], Sp08 Spahn et al. (2008) [19]. [a] Transitions between the interacting states (1000) and (0004).

Table 4 :

 4 Spectroscopic constants derived for DC 3 N in the ground and v 4 = 1 states.Number in parenthesis are one standard deviation in units of the last quoted digit. [a] Kept fixed to ground state value.

	Constant Unit	Ground state	v 4 = 1
	G v	cm -1	0.0	867.594(75)
	B v	MHz	4221.580853(37)	4212.271(16)
	D v	kHz	0.4517857(89)	0.45312(11)
	H v	mHz	0.03949(78)	0.03949 a
	L v	nHz	-0.154(23)	-0.154 a

Table 6 :

 6 Spectroscopic constants derived for DC 3 N in overtone states. Number in parenthesis are one standard deviation in units of the last quoted digit. [a] Constrained value, see text. [b] Kept fixed to ground state value.

	Constant Unit	v 7 = 2	v 7 = 3	v 7 = 4	v 6 = 2
	G v	cm -1	422.3753581(61) 632.510162(64)	841.9860892(95) 983.021(78)
	X L (tt)	GHz	19.354043(62)	19.1988(19)	19.03874(52)	56.39(58)
	y L (tt)	MHz	...	...	1.82(12)	...
	B v	MHz 4247.45224(11)	4260.38118(15)	4273.30576(21)	4236.558(16)
	D v	kHz	0.491827(21)	0.512469(34)	0.53448(11)	0.471993(61)
	H v	mHz	0.03949 b	0.03949 b	0.176(21)	0.03949 b
	L v	nHz	-0.154 b	-0.154 b	-0.154 b	-0.154 b
	d JL (tt)	kHz	-10.426(30)	-10.947(24)	-11.368(21)	141.6(39)
	h JL (tt)	Hz	...	...	-0.0552(64)	...
	q t	MHz	5.93258(10)	5.95888(12)	5.98281(15)	3.15095 a
	q tJ	Hz	-13.897 a	-14.149(36)	-14.288(46)	-1.571 a
	q tJJ	µHz	43.37 a	43.37 a	43.37 a	...

Table 7 :

 7 Spectroscopic constants derived for DC 3 N in combination states.

Table 8 :

 8 Resonance parameters.