Takahisa Uchida
email: uchida.takahisa@irl.sys.es.osaka-u.ac.jp

Nicolas Lair
email: nicolas.lair@inserm.fr

Hiroshi Ishiguro
email: ishiguro@irl.sys.es.osaka-u.ac.jp

Peter Ford Dominey
email: peter.dominey@inserm.fr

Affiliations

Dissociable neural mechanisms for human inference processing predicted by Static and Contextual Language Models

published or not. The documents may come   L'archive ouverte pluridisciplinaire

Abstract:

Language models (LMs) continue to reveal non-trivial relations to human language performance and the underlying neurophysiology. Recent research has characterized how word embeddings from a LM can be used to generate integrated discourse representations in order to perform inference on events. The current research investigates how such event knowledge may be coded in distinct manners in different classes of LMs and how this maps onto different forms of human inference processing. To do so, we investigate inference on events using two well documented human experimental protocols from [START_REF] Metusalem | Generalized event knowledge activation during online sentence comprehension[END_REF]McKoon &Ratcliff 1986 (hereafter Metusalem andMcKoon), compared with two protocols for simpler semantic processing. Interestingly, this reveals a dissociation in the relation between local semantics vs event-inference depending on the language models. In a series of experiments, we observed that for the static language models (word2vec/GloVe), there was a clear dissociation in the relation between semantics and inference for the two inference tasks. In contrast, for the contextual language models (BERT/RoBERTa), we observed a correlation between semantic and inference processing for both inference tasks.

The experimental results suggest that inference as measured by Metusalem and McKoon rely on dissociable processes. While the static models are able to perform Metusalem inference, only the contextual models succeed in McKoon inference. Interestingly, these dissociable processes may be linked to well-characterized automatic vs strategic inferences processes in the psychological literature. This allows us to make predictions about dissociable neurophysiological markers that should be found during human inference processing with these tasks.

INTRODUCTION

We are witnessing an interesting conjuncture in the science and technology of language.

Language Models (LMs) in machine learning are beginning to display remarkable performance capacities, with human-like performance in question answering [START_REF] Talmor | Commonsenseqa: A question answering challenge targeting commonsense knowledge[END_REF], semantic similarity judgement, translation and other domains [START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF][START_REF] Reimers | Sentence-bert: Sentence embeddings using siamese bert-networks[END_REF]. In certain aspects they are similar enough to human performance that specific measures of human language comprehension from psycholinguistic experiments are now being used to characterize and evaluate these language models [START_REF] Ettinger | What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models[END_REF][START_REF] Ettinger | Modeling N400 amplitude using vector space models of word representation[END_REF].

At the same time, language models are beginning to display underlying representations and mechanisms that provide insight into human brain processes in language processing [START_REF] Dehghani | Decoding the neural representation of story meanings across languages[END_REF][START_REF] Mitchell | Predicting human brain activity associated with the meanings of nouns[END_REF][START_REF] Schrimpf | Artificial Neural Networks Accurately Predict Language Processing in the Brain[END_REF].

A central requirement in language comprehension is inference, the ability to complete missing information to allow disambiguation of meaning, as characterized in the tradition of research demonstrating that discourse comprehension often involves instantiation of relevant, unstated information accessed from long-term memory [START_REF] Graesser | Constructing inferences during narrative text comprehension[END_REF][START_REF] Mckoon | Inferences about predictable events[END_REF][START_REF] Mckoon | Inference during reading[END_REF]. [START_REF] Metusalem | Generalized event knowledge activation during online sentence comprehension[END_REF] demonstrated that humans discriminate expected vs. unexpected words based on the discourse context that requires causal or bridging inferences that engage general event knowledge that is not explicitly stated in the discourse. For example, knowledge about car accident events allows one to expect that if a car comes racing though a red light, a crash is imminent. [START_REF] Mckoon | Inference during reading[END_REF] made the distinction between prevalent constructivist theories of inference, and their minimalist proposal. They proposed that readers do not automatically construct inferences to fully represent the situation described by a text. Rather, in the absence of specific, goal-directed strategic processes, only inferences that establish locally coherent and that rely on information that is quickly and easily available are processed. We refer to this as implicit inference. These minimal representations provide the basis for more strategic inference processes that readers invoke to achieve more specific comprehension goals [START_REF] Mckoon | Inference during reading[END_REF][START_REF] Van Den Broek | Inference generation in text comprehension: Automatic and strategic processes in the construction of a mental representation[END_REF].

We refer to this as explicit inference.

The current research investigates how event knowledge may be coded in language models developed in the context of natural language processing (NLP) in machine learning.

We set out to examine how language models may be able to demonstrate human-like performance in tasks that are thought to require knowledge of general events that is not directly available in the discourse being processed. This has recently been demonstrated as wikipedia2vec, a word2vec language model that learns to predict the current word based on the surrounding context words, trained on the entire Wikipedia corpus [START_REF] Yamada | Wikipedia2Vec: An efficient toolkit for learning and visualizing the embeddings of words and entities from Wikipedia[END_REF], was used to predict brain responses during discourse processing that requires inference on events [START_REF] Uchida | A Model of Online Temporal-Spatial Integration for Immediacy and Overrule in Discourse Comprehension[END_REF]. The human-like performance of the model was characterized by a specific linking hypothesis between the model behavior and human behavior and ERP responses as in [START_REF] Brouwer | A neurocomputational model of the N400 and the P600 in language processing[END_REF].

As noted by [START_REF] Talmor | oLMpics--On what Language Model Pre-training Captures[END_REF], "if a pre-trained language model is based on word occurrence statistics, we would imagine that it would struggle on tasks that require symbolic reasoning, such as making inferences about predictable events. If a pretrained model succeeds in such tasks, then the representations that it develops must be useful for these tasks". This motivates the current research, which explores in a more systematic way the properties that allow language models to display human-like performance in these inferencing tasks. [START_REF] Metusalem | Generalized event knowledge activation during online sentence comprehension[END_REF] set out to address "how activation of particular elements of event knowledge is constrained in real time by the specific cues present in an unfolding sentential context." We have a similar objective which is to address this same question in the context of discourse processing by artificial language models. We will exploit the ability of current LMs to generate embeddings which can be used to resolve inference problems. Following [START_REF] Goldstein | Thinking ahead: prediction in context as a keystone of language in humans and machines[END_REF] we consider two types of LMs and their corresponding word embeddings: static embeddings (e.g., word2vec [START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF] , GloVe [START_REF] Pennington | Glove: Global vectors for word representation[END_REF]), which assign a single vector to each word in the lexicon irrespective of context, and contextual embeddings (e.g. BERT [START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF]) in which the same word is assigned different embeddings (vectors) as a function of the surrounding words. We test the hypothesis that these dissociable forms of language models will correspond to dissociable forms of inference processing. If our hypothesis bears out, then it will lead to predictions about dissociable neurophysiological signatures for these distinct inferencing processes.

Part of the originality of our approach is to examine potentially dissociable aspects of the human behavior called inference. In the following we attempt to establish a link between these processes and different classes of LMs, with the goal of then providing a mechanistic basis for making predictions about corresponding dissociable neural processes in the human brain.

MATERIALS AND METHODS

We investigate inference on events using two well documented protocols from [START_REF] Metusalem | Generalized event knowledge activation during online sentence comprehension[END_REF] and [START_REF] Mckoon | Inferences about predictable events[END_REF]) (hereafter Metusalem and McKoon), in an effort to demonstrate dissociation in the relation between implicit vs explicit inference depending on the language models.

In the Metusalem task, subjects are exposed to a sentence, or to the same sentence preceded by an event-evoking discourse. They are then tested on one of three types of words:

Expected (expected both in the context of the sentence alone, and in the extended discourse), Unexpected-Related (unexpected in the context of the sentence alone, but related to the extended discourse), Unexpected-Unrelated (unexpected in the context of the sentence alone, and unrelated to the extended discourse). The measure of performance is the N400 brain responses recorded by EEG scalp electrodes. We should note that there is no explicit or strategic task performed by the subject, and we thus consider this an implicit inference task.

We recall that the N400 is a cortically evoked brain potential that occurs ~400ms after a given word onset. Its amplitude reflects a form of semantic dissonance or surprise [START_REF] Kutas | Reading senseless sentences: brain potentials reflect semantic incongruity[END_REF], or the difficulty of understanding a word in a given context. An example from Metusalem of such a sentence, a discourse and the three types of words are given in Table 1. In the sentence context, N400s are increased for both Unexpected types with respect to Expected. In the event-evoking discourse, the N400 for the Unexpected-Related type is rescued (i.e. the discordance response is reduced, revealing access to event knowledge that allows inference). That is, the N400 for the Unexpected-Related word is reduced, because the discourse context has made more explicit its relatedness. In the automatic vs. strategic dissociation [START_REF] Mckoon | Inference during reading[END_REF], this task would tend towards automatic inference processing. *** Table 1 About Here *** In McKoon's inference task, subjects are exposed to one of two sentences that either evokes a context, e.g. about writing a letter, or use many of the same words but does not evoke that context. Subjects are then asked if a specific word, related to the context, appeared in the sentence. The experiment shows that subjects are slower to report that the target word (that can be inferred by the contextual but not control sentence) did not appear in the sentence, only for the context-evoking sentences, revealing access to event representations that prime the target word. An example of two such sentences and a target word are given in Table 2.

Given that the subjects have a strategic task to perform that requires inference processing, we consider that on the automatic vs. strategic dissociation, this task tends towards strategic. *** Table 2 about here *** We previously reproduced the Metusalem results using a model that constructed a discourse vector as a form of average of Wikipedia2Vec embeddings [START_REF] Uchida | A Model of Online Temporal-Spatial Integration for Immediacy and Overrule in Discourse Comprehension[END_REF], implemented numerically or approximated by a recurrent reservoir network. In the current research, we compare inference performance and semantic performance for the static and contextual models, respectively, based on word2vec/GloVe and BERT. BERT is designed to encode sentence context, and we thus predicted that it would demonstrate an inference processing capability that can be dissociated from that of the static language models. The objective of this study is thus to evaluate these static vs. contextual LMs in different semantic and inference tasks, in order to identify dissociable computational processes, which can then be used to generate predictions about corresponding dissociations in neurophysiological processing when humans perform these same tasks.

NLP models and Simulation

In order to broadly evaluate the two classes of LMs, and in order to have sufficient data points to perform correlation analyses, we studied a number of instances of static and contextual models. The different instances of static models were derived from word2vec and GloVe models trained on different corpora, with different embedding vector dimensions.

Similarly, the different instances of contextual models were derived from variants of BERT and RoBERTa in the Sentence-BERT context, that were pretrained and fine-tuned on various corpora as indicated in Table 1. While the static and contextual models thus varied in terms pretraining and fine tuning they were exposed to, the principal distinction remains the underlying static vs. contextual model architecture.

We compared inference performance and semantic performances for 22 static language models based on word2vec and GloVe and for 23 contextual language models based on BERT/RoBERTa. These are listed in Appendix 1 in the Supplementary Material. The processing pipeline for evaluating a model is illustrated in Figure 1. In the inference tasks, the given text is run through the language model. For the Static models, embeddings are produced for each word and accumulated as an average vector. For the Contextual models the integration is performed by the model, and we retrieve the contextual embedding. For this we use the sentence transformer adaptation of BERT and RoBERTa [START_REF] Liu | Roberta: A robustly optimized bert pretraining approach[END_REF], SBERT [START_REF] Reimers | Sentence-bert: Sentence embeddings using siamese bert-networks[END_REF]. SBERT adds a pooling operation to the output of BERT / RoBERTa to derive a fixed sized sentence embedding which renders BERT capable of producing semantically meaningful sentence embeddings. The SBERT.net infrastructure provides access to a number of pretrained BERT and RoBERTa models that we use.

Pretraining corpora include the STSb semantic textual similarity baseline corpus [START_REF] Cer | Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation[END_REF] and the Stanford natural language inference corpus SNLI [START_REF] Bowman | A large annotated corpus for learning natural language inference[END_REF].

We then calculate the distance between the text embedding and the target word in the inference task using the cosine similarity, which is a measure of the vector similarity. This is explained in more detail for each task below, including the linking hypotheses between the cosine similarity measure, and the human performance measures. *** Figure 1 About Here ***

Semantic and Inference tasks based on published human experiments

In this study, we adopted two inference tasks: based on hallmark studies of Metusalem [START_REF] Metusalem | Generalized event knowledge activation during online sentence comprehension[END_REF] and [START_REF] Mckoon | Inferences about predictable events[END_REF]. In order to compare inference performance with more basic semantic processing baselines, we also adopted two measures of semantic processing from Chwilla [START_REF] Chwilla | The N400 as a function of the level of processing[END_REF], and a measure derived from the Metusalem study. These are all defined in detail below.

Chwilla Semantic Task:

In this task [START_REF] Chwilla | The N400 as a function of the level of processing[END_REF], subjects were exposed to a priming word and then a target word, and the N400 in response to the target word was measured. Chwilla et al. provide 40 pairs of related words, and 40 pairs of unrelated words. In a lexical decision task, targets that were unrelated to the prime produced larger N400 responses than related targets.

In the linking hypothesis for this measure, our measure of cosine similarity is inversely related to the N400 in the Chwilla task. For our model testing, the Chwilla semantic score is computed by the following formula.

(1) Chwilla Semantics = CosSim(Related pair) -CosSim(Unrelated pair)

To evaluate the performance of a given LM model, we compare the semantic relatedness predicted by the model for the related vs. unrelated pairs. We then perform a ttest on the semantic relatedness scores for the 40 related vs. 40 unrelated pairs. The t-score of the comparison is the indicator of performance. A significant result corresponds to a modeling of the human N400 behavior according to the linking hypothesis stated above.

Metusalem Semantic Task:

In order to obtain an alternative measure of semantic processing, we use the Metusalem task and compare responses to the expected vs. unexpected (related and unrelated) words. We generate an average vector for the words in the sentence context (see Table 1), and then compare cosine similarity between sentence and the expected word, vs. the mean of the similarities of the sentence to the two unexpected words. This difference is a measure of semantic processing. In the linking hypothesis for this measure, our measure of cosine similarity is inversely related to the N400 that would be elicited under these conditions. The score of semantics based on the Metusalem study is computed by the following formula.

(2) Metusalem Semantics =.

CosSim(SentVect, Expected) -

(CosSim(SentVect, Related) + CosSim(SentVect, Unrelated))/2
We performed a paired t-test on the semantic relatedness scores for the expected vs unexpected stimuli in the 72 experimental trials provided by the original article. The t-value of the comparison is the measure of performance.

Metusalem inference task:

Downloaded from http://direct.mit.edu/nol/article-pdf/doi/10.1162/nol_a_00090/2057535/nol_a_00090.pdf by guest on 23 November 2022

In the Metusalem inference task [START_REF] Metusalem | Generalized event knowledge activation during online sentence comprehension[END_REF], the N400 was measured in response to the expected, related, and unrelated words. The response to the related words decreases in the discourse vs. sentence context. In the current study, we use a measure of cosine similarity between sentence/discourse vectors and word vectors. This measure increases as these vectors are more similar. In the linking hypothesis for this measure, our measure of cosine similarity is inversely related to the N400 in the Metusalem task. Thus, the score of inference based on the Metusalem task is calculated as follows :

(3

) Metusalem Inference = CosSim(Context, Related) -CosSim(Context, Unrelated) - (CosSim(Sentence, Related) -CosSim(Sentence,Unrelated))
Where Context is the vector representation of the discourse context along with the sentence, and Sentence is the vector representation of the sentence out of context (see Table 1). In other words, we measure the advantage of the complete discourse for differentiating between Related words vs Unrelated words (second line in Eqn 3), with respect to the single sentence in differentiating between the Related, and Unrelated (third line in Eqn.3). The idea is that similarity to the Related words will be "rescued" by the extended discourse, and not by the sentence alone.

To evaluate a model, we run the model on the sequence of words in the sentence or the extended context discourse to generate the sentence/context vector. We then generate the embedding vector for the target word and measure the cosine similarity between the target word and sentence/context vector. We performed a paired t-test on the semantic relatedness In the McKoon task [START_REF] Mckoon | Inferences about predictable events[END_REF], subjects displayed greater reaction times in response to a word when it was evaluated in a matching inference context sentence, vs. a control sentence made with most of the same words, but not evoking the event context.

We thus create a linking hypothesis, such that the value that we calculate, based on cosine similarity, varies inversely with the reaction time in the original McKoon task. In our study, the score for evaluating performance on this task is calculated as follows:

(4) McKoon Inference = CosSim(Context,Target) -CosSim(Control, Target)
To evaluate a model, the model is run on the context sentence and the control sentence, and the resulting vectors are compared to the target word vector, for the 32 experimental trials provided in the original article. We performed a paired t-test on these semantic relatedness values. The t-value of the comparison is the measure of performance.

Analysis code and data for these experiments is openly available at this repository: https://github.com/TakahisaUchida/InferenceSemanticsExperiment. Eqns 1-4. Note that while we mention significance here, in the main analyses that follow we use these t-values in correlation analyses, and do not interpret them in terms of significance.

Experiments and Results

Globally

Thus we are not concerned with the possibility of false positives in these multiple comparisons.

Here we provide a brief roadmap for the four experiments we performed. In the first Experiment we examine the correlation between semantics and inference for the two types of LM. This yields a dissociation between the two forms of inference for both LMs, as semantics and inference are correlated for Metusalem but not McKoon inference. In Experiment 2, we perform the same comparisons, but this time using a measure of semantics that takes into account contextual processing. This further clarified the dissociation between the two forms of inference, such that the correlation between semantics and inference is rescued for McKoon inference, only for the Contextual LMs. In Experiment 3 we directly contrast performance on the inference tasks for the two language models, which clarifies that McKoon inference requires contextual processing. Experiment 4 performs a complimentary analysis, this time distinguishing between semantic tasks that have different requirements on integrative processing.

Experiment1:

The objective of Experiment 1 was to test the prediction that for a given LM, performance in a semantic task should predict or be correlated with performance in an inference task. For semantics we used the Chwilla task, and for inference we used the Metusalem and McKoon tasks. We tested the correlations between semantic and inference performance using two populations of LMs, Static and Contextual.

Thus, for each of the Static and Contextual models, we tested the model on the semantic task and the two inference tasks, and we compared performance on semantics vs. inference for the two inference tasks. Figure 3 shows the result of comparison between

Chwilla semantics and Metusalem and McKoon inference for Static and Contextual language

models. There we observe a correlation between performance for semantic and inference processing for Metusalem inference, such that increased performance on semantics corresponds to increased performance on inference. This is the case for both for the Static (Pearson correlation r= 0.88, p=7.16e-08; regression slope 0.49) and Contextual (r =0.68, p=0.0004; regression slope 0.57) language models. In contrast, for McKoon inference, increased performance on semantics is not associated with increased performance in inference, neither for the Static nor Contextual language models. For the Static model, while the correlation is significant (Pearson r=0.60, p=0.003), the slope of the regression line is nearly flat, at 0.06. This low slope indicates that as performance improves with semantics, it changes marginally for inference. Likewise, for the Contextual language model, the correlation is not significant (r=-0.13, p = 0.5) and the slope of the regression is -0.11.

These results demonstrate a dissociation between inference processing as evaluated by the Metusalem and McKoon tasks when compared with semantic processing as evaluated by the Chwilla task. This is observed both for static and contextual LMs. It is of interest to This shows a coherence across these model types with respect to performance on these tasks. *** Figure 3 about here ***

Experiment2:

Experiment 1 revealed that the correlation between classes of language processing tasks semantics vs inferencevaries depending on the specific inference task. This suggests that there is some non-trivial difference between the two inference tasks. However, it is also possible that the semantic task plays a role in this dissociation. We thus set out in Experiment 2 to examine the correlation between semantic and inference processing using a different semantic task. We use a measure of semantics that exploits a dimension of the Metusalem task which compares expected vs unexpected words in the context of a target sentence. This semantic task is thus more elaborate than a simple difference between related vs unrelated word pairs. We compared this measure of Metusalem That is, we see that with the contextual LMs, when the more elaborate measure of semantics is used the correlation between inference and semantics is recovered for McKoon inference (regression slope 0.44, Pearson r = 0.51, p = 0.01). This is not the case for the static models (regression slope 0.07, Pearson r = 0.48, p = 0.02). For Metusalem inference, there is a clear relation between semantics and inference for the static (regression slope 0.74, Pearson r = 0.91, p = 2.6e-9) and contextual LMs (regression slope 0.67, Pearson r = 0.75, p = 3.3e-5). This reveals a complex interactions between language model classes, inference types, and semantic types.

Experiment3:

In order to gain a more direct view of the relation between Metusalem and McKoon inference, we compare them directly, again for the Static and Contextual models. In Figure 5 we observe that static models display increased performance for Metusalem inference,

Experiment 4

As we observe significant task related difference for the inference tasks, we complete the analysis by directly comparing our two semantic tasks in the context of the two language model classes. Figure 6 illustrates the comparisons between performance on the two semantic tasks for the Static and Contextual models. Performance is characterized by the t-statistic value for the models on the inference tasks. Interestingly, we observe that the Static models display superior performance for the Chwilla vs Metusalem semantic tasks, and indeed that on the Chwilla task, the Static models actually outperform the Contextual models.

These observations are confirmed by the 2x2 ANOVA on Model (Static, Contextual).

x Semantic Task (Metusalem, Chwilla). The Static models outperform the Contextual models (main effect for Model F(1,21) = 5.1, p < 0.05). The Chwilla task yields superior overall performance (main effect for Semantic Task F(1,21) = 114; p < 0.001. The significant performance increase specifically for the Static model with Chwilla semantics is confirmed by the significant Model x Task interaction (F(1,21) = 86, p < 0.001).

This advantage for Static models on the Chwilla task might be understood in terms of the behavioral requirements of the semantic tasks, and the nature of the Static and Contextual LMs. The Chwilla semantic task is based on evaluating relations between single word pairs. This corresponds to the word centered representations in the Static models, thus providing a potential advantage for the Static models.

DISCUSSION

The current research explores how event knowledge that is not explicitly stated in the narrative may be coded in language models, and how this can be used to better understand the computational neurophysiology of human inference processing. This research takes place in the context of an exciting new development in human language neurophysiology, that exploits a form of "big science" based on massive availability of datasets and multiple models (e.g. [START_REF] Schrimpf | The neural architecture of language: Integrative modeling converges on predictive processing[END_REF]). Within this context, the current research takes a complimentary approach, that focuses on specific tasks within the same domain (inference) and the differences between them.

We focused on two well-characterized inference tasks from the human experimental literature. The Metusalem task examines how the addition of contextual discourse allows inferences about events to be made. In the behavioral procedure, the experimental variable measured was an implicit brain response to particular target words that were related to the discourse (or not) via inference. Thus, there was no explicit inference task. After each trail the subject was asked a simple Yes/No comprehension question to ensure that participants read each scenario for comprehension.

The McKoon task examines how the reorganization of the same words in different sentences can change the event context to allow inferences about events. In contrast to the Metusalem task, in the McKoon task, the experimental variable measured was an explicit behavior (response time) to a question that required inference on the event described in the sentence. These explicit questions encourage the subjects to make more strategic and deeper encodings of the inferred event.

In the classic context of inferencing, as characterized in the cited works of Metusalem and McKoon, at least some forms of inference are assumed to rely on rather elaborate cognitive processes that include the construction of situation models that represent agents, actions, causal roles, etc. [START_REF] Zwaan | Embodiment and language comprehension: reframing the discussion[END_REF][START_REF] Zwaan | The construction of situation models in narrative comprehension: An event-indexing model[END_REF]. However, [START_REF] Mckoon | Inference during reading[END_REF] argue for a minimalist hypothesis of inference, where automatic inference during reading relies only on easily available information, while more elaborate representations are constructed by dissociable processes, as required for specific, strategic needs. We can interpret our current results in this context, where the Metusalem inferencing task relies preferentially on this minimalist implicit processing, which can be furnished by Static LMs. In contrast, the McKoon inference task requires representations associated with more explicit strategic processing, and that are more associated with the Contextual LMs.

That is, the McKoon inference task is sensitive to how the contextual organization of words in a sentence can change its relation to a single target word. The target sentence evokes the test word by inference. The control sentence uses as many of the same words as possible, but organized in a different order, and does not evoke the test word. This requirement for disambiguation is consistent with contextual LMs. On the other hand, the Metusalem inference task is sensitive to how the simple accumulation of word information in a text can disambiguate a related word by inference, but it is not specifically sensitive to the contextual organization of words within a sentence or discourse. This can be achieved using aggregation of word embeddings from a static LM. In summary, static LMs are able to simulate minimalist inference processing as required by the Metusalem task, whereas contextual LMs are required to simulate the more strategic processing as required for the McKoon task. It is noteworthy that we specifically chose not to use "left-to-right" autogregressive models/transformers (e.g. . This is because we wanted to take an Occam's razor approach, using the simplest models that allow to reveal the distinction between implicit and explicit inference processing. This said, auto-regressive models can be seen as interesting from the psycholinguistics point of view, and testing such models would make sense in a future avenue of research.

We have identified two forms of inference processing (Metusalem and McKoon) that are realized by human neural mechanisms that have not yet been fully characterized. For these inference processes, we defined linking hypotheses between the observed human behavior or neurophysiology, and the response of the language model. In this context we have demonstrated that these two forms of inference rely on dissociable computational processes corresponding to static and contextual language models. This allows us to propose that these two forms of inference should also rely on dissociable neural mechanisms.

That is, this functional distinction should now translate into patterns in language processing in the human brain that we would predict based on our results. In this context, Schrimpf et al [START_REF] Schrimpf | The neural architecture of language: Integrative modeling converges on predictive processing[END_REF] characterized the brain score of language models as a function of their ability to predict human neural data recorded while subjects perform language comprehension (typically self-paced reading). They observed that brain score is correlated with the ability to predict the next word in text, but not necessarily with the ability to perform other language tasks, including inferencing (evaluated as a subset of language tasks in [START_REF] Wang | GLUE: A multitask benchmark and analysis platform for natural language understanding[END_REF]). Interestingly, the brain activity used in these analyses was recorded while subjects were reading, but not directly performing other language tasks. This leads us to propose the hypothesis that if neural activity is recorded while subjects are performing tasks that require inference processing, then brain scores on such a dataset will correlate with the performance on the corresponding inference tasks. Indeed, it is highly likely that behaviorally dissociable forms of language processing will be associated with distinct neural computations, and distinct spatiotemporal patterns of brain activation.

More specifically, we predict that when human subjects are exposed to comprehension tasks that have been computationally dissociated, as in the case of the Metusalem and McKoon inference tasks, this processing will be accompanied by dissociable spatiotemporal patterns of brain activity.

We can consider that there are minimalist and more extended forms of inference as demonstrated by [START_REF] Mckoon | Inference during reading[END_REF]. Metusalem inference can rely on a shallow form of inference that is modeled by the accumulation of static LM embeddings. In contrast,

McKoon inference is more appropriately modeled by contextual LMs. We thus predict that when humans are submitted to inference tasks, we should see neurophysiological evidence for these dissociable forms of inference. Specifically, performing a systematic search should identify ROIs or networks whose activation is correlated with the shallow embedding accumulation as solicited by the Metusalem task, and another set of ROIs whose activation is correlated with the deeper transformer based representations as solicited by the McKoon task.

It will be of particular interest to identify the brain networks that are solicited in inference tasks that require more explicit strategic processing, and to compare these with networks that have been associated with automatic semantic processing. Binder and colleagues [START_REF] Binder | The neurobiology of semantic memory[END_REF][START_REF] Binder | Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies[END_REF] have performed extensive meta-analysis of studies that involve semantic memory. They conclude that in addition to modality specific sensory-motor areas, semantic memory recruits high-level convergence zones in the inferior parietal, lateral temporal, and ventral temporal cortex. They note that these regions are far from primary sensory and motor cortices and appear to be involved in processing general rather than modality-specific semantic information [START_REF] Binder | The neurobiology of semantic memory[END_REF]. A related study likewise observed a corresponding distributed system for comprehension of pictures and sentences depicting human events [START_REF] Jouen | Beyond the word and image: characteristics of a common meaning system for language and vision revealed by functional and structural imaging[END_REF].

Interestingly, the task in this experiment was simply to remain vigilant, and respond to occasional questions that asked about what was shown in the previous stimulus. Despite the low requirements for explicit strategic processing, activation of an extensive semantic network was observed.

Future research should examine how task requirements, such as the need to perform strategic inference as required for the KcKoon task, will recruit brain networks not seen for more automatic processing. The specification of computationally dissociable instances of the same class of tasks, as we have done here for Metusalem and McKoon inference, provides the basis for discovering whether these computational dissociations are reflected in distinct neural computations in the human brain. It is likely that this progress will inform and be informed by the cognitive processing of events (e.g. [START_REF] Radvansky | Event boundaries in memory and cognition[END_REF]). Indeed, mounting neuroscientific evidence shows how event structure is processed in the brain in terms of the spatial and temporal distributions of narrative event structure [START_REF] Baldassano | Discovering event structure in continuous narrative perception and memory[END_REF][START_REF] Baldassano | Representation of real-world event schemas during narrative perception[END_REF], and recurrent models of cortical processing are beginning to provide neurocomputational explanations for these event driven phenomenon [START_REF] Dominey | Narrative event segmentation in the cortical reservoir[END_REF]. What remains to further explore is the link between the computational process such as those we explore here, and the spatiotemporal distribution of neural activity observed in the brain during narrative comprehension that relies on inference. The debutante owed a letter to her mother, so she picked up some paper and sat down.

Slower response

Control Sentence: Like her mother, the debutante lettered in sports and often had her name in the paper.

Faster response Figures:

Figure 1. Model evaluation pipeline. For static models, separate embeddings are collected for each word in the discourse, and integrated in as an average vector, to generate the discourse vector. In related studies this integration is performed by a recurrent reservoir model. For contextual LMs, the language model generates the contextual discourse vector directly based on the input discourse. In parallel, the embedding is retrieved for the target word. The embeddings for the discourse and the target word are compared by the cosine of the angle between them, with 1 as identical and 0 as orthogonal. References:

  scores for Context(Related -Unrelated) vs Sentence(Unexpected-Related) for the 72 experimental trials provided by the original article. The t-value of the comparison is the measure of performance. An evaluation of the Static model using the w2v100 corpus on the 72 trials is presented in Figure 2. There we see indeed, that the discourse context rescues the similarity for the Related word. That is, the advantage of Unexpected-Related over Unexpected-Unrelated is revealed in the Discourse context. The comparison in Eqn. 3 yields t statistic= 4.14, p=9.3e-05. *** Figure 2 About Here *** 2.3.4 McKoon inference task:Downloaded from http://direct.mit.edu/nol/article-pdf/doi/10.1162/nol_a_00090/2057535/nol_a_00090.pdf by guest on 23 November 2022

  , we tested 22 Static LMs and 23 Contextual LMs on 2 semantic tasks, and 2 inference tasks. Model details are provided in the Supplementary Material. For the semantic tasks, all models successfully performed the Chwilla semantic task, and all but one Static model (Static LM 11, Supplementary Material Appendix) and one Contextual model (Contextual LM 10) successfully performed the Metusalem semantic task. For inference, all but three of the Static models (Static LMs 11, 18, 19) successfully performed Metusalem, and all but 3 Static models (Static LMs 2, 12, 18) successfully performed the McKoon inference task. Successful performance was characterized by p < 0.05 on the t-statistics described for

  Figure 4 illustrates the result of comparison between Metusalem semantics and Metusalem/McKoon inference for Static and Contextual LMs. Similar to Experiment 1, for the Static models, we continue to observe a clear positive relation between semantics and inference for Metusalem, but not for McKoon. Interestingly, with the Contextual models the semantics-inference relation is rescued for McKoon inference. That is, we see that with the

  relative to their reduced performance on McKoon inference. In contrast, Contextual models display good performance both for Metusalem and McKoon inference, and outperform the Static models on both inference tasks. *** Figure 5 about here *** These observations are confirmed by a 2x2 ANOVA on Inference Task(Metusalem, McKoon) x Model (Static, Contextual). There is a significant main effect for Inference Task, with Metusalem having higher scores than McKoon (Main effect for Task F(1,21) = 18, p < 0.001). There is a significant main effect for Model, as Contextual models perform better than the Static models (main effect for Model F(1,21) = 55, p < 0.001). While Contextual models perform equally well for both tasks (Scheffe post-hoc p>0.1), the Static models perform significantly better on Metusalem than McKoon inference (Scheffe post-hoc p<0.001). This task related differences for the models is revealed in the significant Task x Model interaction (F(1,21) = 5.5, p < 0.05). These performance differences can be considered in terms of the nature of the tasks and the models. Both inference tasks involve processing of multiple word sentences, and so there may be an overall advantage for the Contextual models. The Metusalem task involves unambiguous interpretation of multiple target words, while the McKoon task involves multiple interpretations of the same word in different contexts. By construction, the requirement in McKoon inference for contextual sensitivity favors Contextual models, and can help us to understand the observed Task x Model interaction.

Figure 2 .

 2 Figure 2. Example results of Metusalem inference for a single static model using the enwiki_20180420_100d (w2v100) corpus. S-Sentence, D-Discourse context. E -Expected, R -Unexpected Related, U -Unexpected Unrelated. The measure of inference is indicated in the solid and dotted horizontal boxes. In the Sentence condition, there is a small difference between S-R and S-U (solid box). When discourse context is provided, the relative similarity for the related words (D-R) is rescued (dotted box). Inference is measured by the difference between discourse vs. sentence (DR-DU) -(SR-SU), as specified in Eqn. 3. The red arrows indicate the "rescue" as the semantic relatedness advantage for related vs. unrelated words is increased in the discourse context.

Figure 3 :

 3 Figure 3: Experiment 1. Relation between Chwilla semantics and Metusalem/McKoon inference by Static and Contextual LMs. Both classes of models display significant correlation between semantic and inference for the Metusalem task (red and orange lines). Interestingly this is not the case for the McKoon task, where both fail to demonstrate such effective correlation between semantic and inference performance (blue and green lines).Each data point in the scatter plot refers to a specific LM, and the number corresponds to the numbered LMs in the Supplementary Material for Static and Contextual LMs.

Figure 4 .

 4 Figure 4. Experiment 2. Relation between Metusalem semantics and Metusalem/McKoon inference with Static and Contextual LMs. Contextual models now display significant correlations between semantic and inference processing for both inference tasks. Only the Static models with the McKoon task fail to display clear effective correlation with semantics. Each data point in the scatter plot refers to a specific LM, and the number corresponds to the numbered LMs in the Supplementary Material for Static and Contextual LMs.

Figure 5 .

 5 Figure 5. Mean performance for the Static and Contextual models on the Metusalem and McKoon inference tasks performance (as characterized by the t-scores on comparisons). The Contextual models perform well on both tasks, while the Static models perform better on Metusalem vs. McKoon. Whiskers denote 0.95 confidence intervals.

Figure 6 .

 6 Figure 6. Experiment 4. Comparison between performance (as characterized by the t-scores on comparisons) on the two semantic tasks for the Static and Contextual LMs. While Contextual models dominated for inference processing (Fig 5), the best scores for the Chwilla semantic task are attained by the Static models.

  Downloaded from http://direct.mit.edu/nol/article-pdf/doi/10.1162/nol_a_00090/2057535/nol_a_00090.pdf by guest on 23 November 2022 examine the variability across the different model groups. For example, for Metusalem inference the best two contextual models, 9 and 19 both correspond to versions that employ the Mpnet training method (Song, Tan, Qin, Lu, & Liu, 2020) (see Supplementary Material).

Table 1 .

 1 Example of stimuli and results in the Metusalem (2012) inference task.

	Tables:
			Expected Unexpected-	Unexpected-
				Related	Unrelated
			CRASH POLICEMAN CONDUCTOR
				N400 Size
	Sentence	A moment later, she heard a	Small	Large	Large
	Alone	terrible
	Discourse	Elizabeth was standing at the	Small	Small-Medium	Large
	Context +	intersection waiting for the		(N400
	Sentence	light to change. All of a		"rescued" by
		sudden she saw a car barrel		discourse)
		through the red light. A
		moment later, she heard a
		terrible
				Target word
				WRITE
	Context Sentence:

Table 2 .

 2 Example of stimuli and results in the McKoon & Ratcliff (1986) inference task.

  Downloaded from http://direct.mit.edu/nol/article-pdf/doi/10.1162/nol_a_00090/2057535/nol_a_00090.pdf by guest on 23 November 2022

Downloaded from http://direct.mit.edu/nol/article-pdf/doi/10.1162/nol_a_00090/2057535/nol_a_00090.pdf by guest on 23 November 2022

Acknowledgements: 

Research supported by Region Bourgogne Franche-Comté (AAP 2019 RobotSelf), and JST Moonshot R&D Grant Number JPMJMS2011, and JSPS KAKENHI Grant Number 19H05693 and 22K17949.

Research supported by Region Bourgogne Franche-Comté (AAP 2019 RobotSelf), and JST Moonshot R&D Grant Number JPMJMS2011, and JSPS KAKENHI Grant Number 19H05693 and 22K17949. ORCID https://orcid.org/0000-0002-9318-179X Peter Ford Dominey https://orcid.org/0000-0002-0608-0297 Nicolas Lair


Stefan Kuhn
email: stefan.kuhn@ut.ee

Jean-Marc Nuzillard
email: jm.nuzillard@univ-reims.fr

J.-M Nuzillard

nuclear magnetic resonance spectroscopy data

Easy structural dereplication of natural products by means of predicted carbon-13

Introduction

The concept of dereplication is invoked each time a way is found to avoid repeating a task that was already performed. In the field of the organic chemistry of natural products, a compound produced by different living organisms may have been purified many times by different research teams resulting in repeated structure elucidation works. [1] Purification dereplication avoids the repetitive search for an appropriate compound isolation method. Structural dereplication avoids the recording of detailed spectroscopic data and their interpretation for compounds already reported in the scientific literature.

Mixture analysis as reported by J. Hubert and collaborators combined purification and structural dereplication in a single workflow called CaraMel. [2] With this procedure, the isolation of all the constituent of a mixture to a high purity level is not required for compound identification. Structural dereplication is achieved within the CaraMel procedure by looking into a locally developed and enriched database for compounds that match with a list of experimental 13 C nuclear magnetic resonance (NMR) chemical shift values. The database contains structures that were accumulated through bibliographic searches driven by the taxonomic classification of the studied organisms. Each compound in this database is linked to a list of 13 C NMR predicted chemical shifts. The structural and spectroscopic data management is ensured by a software acquired from Advanced Chemistry Development, Inc. (hereafter, ACD/Labs, https: //www.acdlabs.com/) called ACD/C+H Predictors and DB, hereafter referred to as the ACD/Labs software, even though the ACD/Labs company provides many other software products.

Supplementing a collection of small molecule structures with predicted NMR data by means of the ACD/Labs software is a tedious operation because there is by design no simple way to deal with batches of structures. Prediction quality is high but operations on large amounts of structures is unpractical. More precisely, prediction in ACD/Labs software may be carried out by two different procedures. The accurate and slow one relies on the search of molecular fragments within an internal database. Besides, a predictor is dedicated to the verification of experimental NMR chemical shifts values assigned by user to molecular structures. The latter predictor (hereafter called the validation algorithm) operates in an unsupervised way on collections of molecular structures for which a previously user-supplied NMR assignment exists. The output of the validation algorithm can be reused by a dedicated software in order to update the initially given atom to chemical shift assignments so that the predicted ones appear as if they were experimental. This way of supplementing a collection of structures with chemical shift values by means of the ACD/Labs validation algorithm was already reported in the context of the construction of small-sized, taxonomically focused databases. [3] The present article reports the creation and usage of a general natural product database for structural dereplication. This database, acd_lotusv7, derives from the opensource LOTUS natural products database [4] and can be queried through the open-source web interface and search engine of nmrshiftdb2. [5] Natural product chemists can thus access published knowledge easierly than ever. The use of 13 C NMR spectroscopy as primary data source for compound identification and the advantages of resorting on predicted chemical shift values instead of experimental ones have already been discussed in previous works. [2,6] The overall number of identified natural products is not known with high accuracy as it increases every day but is evaluated as a few hundred thousands. [7] The LOTUS database is not presently comprehensive but is steadily enriched with new structures, making of it and of acd_lotus evolving tools for structural dereplication of natural products.

Materials and Methods

All calculations were carried out on a DELL Precision 3530 laptop computer with 16 GB of RAM memory and an Intel ® Core™ i5-8400H CPU @ 2.5 GHz running Windows 10 Education version 20H2. The ACD/C+H NMR Predictors and DB 2021.1.0 software was purchased from ACD/Labs (Toronto, Ontario, Canada). The library of cheminformatics functions RDKit version 2021.03.2 was run in the conda environment (https://www.anaconda.com) with python version 3.6.13 as programming language (https://www. python.org/). Additional chemical structure operations were carried out by functions implemented in the Knap-sackSearch GitHub repository (https://github.com/nuz illard/KnapsackSearch). [8] The LOTUS database version 7 in file 220525_frozen_metadata.csv.gz, was downloaded from the zenodo data repository, https://zenodo.org/rec ord/6582124.

The unpacking of 220525_frozen_metadata.csv.gz provided the text file 220525_frozen_metadata.csv from which each line was scanned to constitute triplets made of the line number, of a wikidata identifier (QID, see https: //www.wikidata.org) and of a SMILES chain. [9] There may be more than one line related to the same QID and SMILES; in this case only the first line was considered for future use. The text produced by the python script csv2smi.py from the triplets was redirected to file lotusv7.smi; it contained two columns, one for the SMILES chain and the other one for a compound character string formed by the QID and the line number joined by an underscore character, such as in Q43656_2, and used as compound name. Wikidata compound Q43656 is cholesterol and it appeared in file z20525_ frozen_metadata.csv at line 2. The file lotusv7.smi contained a minimal description for 201022 compounds.

The script smi2ACD.py applied to lotusv7.smi resulted in file fake_acd_lotusv7.sdf in which a fake chemical shift value was assigned to each carbon atom (99.99). The resulting chemical shift lists, one per compound, were formatted to be understood by the ACD/Labs software as if they were experimental chemical shifts. File fake_acd_lotusv7.sdf contained 201022 compounds. Action of tautomer.py and rdcharge.py scripts achieved in-place tautomer correction and charge/valence correction. The necessity of using these scripts was reported in a preceding work. [3] Operations on structure files make use of the RDKit cheminformatics function library for SMILES chain interpretation, 2D structure diagram generation, tautomer correction and SDF structure file handling.

The splitter.py script applied to file fake_acd_ lotusv7.sdf created 21 .sdf files stored in a dedicated subdirectory. These pieces were named fake_acd_lotusv7_ xx.sdf with xx ranging from 00 to 20, the 20 first ones contained 10,000 compounds each and the last one contained the remaining structures. Chemical shift prediction by ACD/Labs software using the validation algorithm was carried out on these sub-files. Each sub-file was imported in an ACD/Labs database, validated for chemical shifts and exported as fake_acd_lotusv7_xx_exported.sdf. Action of script CNMR_predict.py on the latter produced the files true_acd_lotusv7_xx.sdf in which initial fake chemical shift values were replaced by predicted ones. The content of these files was concatenated into a single file, acd_ lotusv7_tmp.sdf, containing 201,010 compound descriptions. The supplement.py script collected chemical taxonomy data present in file 220525_frozen_metadata.csv and appended them to compound metadata in file acd_lotusv7_ tmp.sdf in order to produce file acd_lotusv7.sdf. Chemical taxonomy data were obtained by the authors of LOTUS from NPclassifier and Classyfire. [10,11] The chemical identifiers SMILES, InChI and InChIKey [12] were recalculated from the structures in acd_lotusv7.sdf and the resulting InChIKeys compared to those provided by 220525_frozen_ metadata.csv. File acd_lotusv7.sdf was finally imported in the ACD database file acd_lotusv7.NMRUDB and in nmr-shiftdb2 for structural dereplication works.

In addition, the prediction of the 13 C chemical shifts for the structures in acd_lotusv7.sdf was also carried out using the HOSE code approach. This was achieved by means of nmrshiftdb2 data from August 1 st , 2019 and of the extended HOSE code search method described in [13] . The resulting predictions were also imported in nmrshiftdb2 to probe the influence of the predictor on dereplication.

Three natural product structures and their corresponding list of experimental chemical shift values were selected from published literature in order to illustrate the use of the acd_lotusv7 database for structural dereplication from 13 C NMR data. The used search engine was the one of nmr-shiftdb2. The nmrshiftdb2 software supports the handling of experimental and predicted NMR spectra which can originate from various sources identified by category tags. The new category acd_lotusv7 was introduced at the time of acd_lotusv7.sdf file importation. Another new category, hose_lotusv7 was created to tag predictions produced by the HOSE code approach. The results of searches carried out with different selection criteria may be combined in nmr-shiftdb2, including those by category tag, chemical shift list, and molecular formula. The nmrshiftdb2 spectrum search algorithm determines the similarity of two spectra by calculating a distance value. Therefore, the search results do not depend on a distance threshold or on any other parameter but are presented to the user after having been sorted in the order of decreasing similarity.

Results and Discussion

All the structures reported in LOTUSv7 file 220525_ frozen_metadata.csv but 12 were present in acd_ lotusv7.sdf, thus constituting a very high success rate due to the careful design of the LOTUS database. However, compounds Q105187773 and Q105007277 were manually discarded as they erroneously contained divalent helium atoms. The other discarded compounds were left out by the ACD/Labs software at the time of fake_acd_ lotusv7_xx.sdf file importation. Discrepancies between initial InChIKey descriptors and recalculated ones arose in 4167 compounds and originated from wrong writing or reading of configuration markers in 2D structure drawings.

The overall chemical shift prediction by the ACD/Labs sofware took about 30 hours, corresponding to an average processing rate of 2 compounds per second. The size of the final acd_lotusv7.sdf file was 1,256 MB. It can be read by any cheminformatics toolkit and its compressed version, of size 219 MB, was stored in a public zenodo repository at https://zenodo.org/record/6621129.

In the following, we illustrate the dereplication process by the retrieval of the structure of three natural compounds of low, medium, and high complexity from published lists of carbon-13 NMR chemical shifts. Compound 1 is characterized by a list of 15 13 C NMR chemical shifts: 36.4, 22.3, 29.7, 33.5, 44.9, 31.8, 161.4, 78.7, 41, 75, 120.5, 176, 8.3, 14.7, and 16 ppm. [14] Searching in the acd_lotusv7 category of nmrshiftdb2 was carried out with these values as chemical shift targets. The structure ranked first in the result set did fit will all constraints on chemical shift values and was identical to the published structure. The analyzed compound was an eremophilane sesquiterpene lactone isolated from Hertia cheirifolia, an Asteraceae collected in the North of Africa. [14] Searching in the whole of nmrshiftdb2 gave the same structure as the top hit, but showed some additional relatively good hits not found within the acd_lotusv7 category. We also combined the search with a chemical formula search for C15H22O3, the chemical formula of the top hit. This gave the same structure as the top hit, as expected, but excluded some hits from the top 10, which had a different elemental composition. This aspect of the corresponding search interface is shown in Figure 1. It should be noted that some of the top 10 are stereoisomers of the top hit. Since the top hit was the expected stereoisomer, the search has identified the correct compound. Wikidata links were added to the records in nmrshiftdb2 (visible on the Additional Data tab), so following the link for the top hit, Q105172965, lead to a list of three plants in which it was found. Selecting Hertia cheirifolia led to the reference to the original publi-cation, referenced Q57818186 in Wikidata, and then to its DOI: 10.1016/0031-9422(90)83039-4, thus providing an incredibly quick way of establishing a relationship between a list of chemical shifts and an article published in a phytochemistry journal in 1990.

The same search was attempted using HOSE code prediction. The best match was the same as with the ACD/Labs prediction, but its similarity score was significantly lower (see Figure 2), mainly due to three shift values being almost 4 ppm away from the ACD/Labs prediction. The other top hits were also the same as with the ACD/Labs prediction as they all presented similar shift differences so that hit ranking was preserved.

Compound 2 was reported with 43 [15] Searching acd_lotusv7 under the same conditions as those used for compound 1, produced stereoisomers and compounds with various formulae in the list of the ten best hits. Restricting the search to molecular formula C43H52N4O7 associated the correct structure to the list of chemical shift values in the original publication. The molecular formula of the second best hit was C43H54N2O8 and corresponded to the best hit compound but with a water molecule added, resulting from the opening of an epoxide ring. The best hit compound was isolated from Ervatamia peduncularis whose QID is Q96375411. Searching this QID in Wikidata revealed that this compound was named conofoline while it was published in 1995 as pedunculin by one of the authors. [16] Conofiline was isolated from Tabernaemontana divaricata and published shortly before pedunculin. It appeared that Ervatamia peduncularis and Tabernaemontana peduncularis are synonyms, suggesting that the two research groups studied two plants of the same genus, differing by their species name, but producing the same highly complex dimeric indolomonoterpenic alkaloids.

The similarity score of conofoline was only 35.09% with HOSE code prediction, whereas it reached 76.4% with the one by the ACD/Labs software. As for compound 1, the lower score arose from a few predicted values being significantly off. The fifth hit from the ACD/Labs prediction, QID Q104944410, has a similarity score of 43.17% using the HOSE code prediction. Structure Q104944410 would therefore come first in a search using HOSE code predictions only.

Results in nmrshiftdb2 searches can potentially be improved by associating chemical shifts with carbon multiplicity information. The search tool of the ACD/Labs software does not offer this possibility. Multiplicity characterises methine, methylene, methyl, and quaternary carbon atoms and is reported in queries by one-letter symbols S(inglet), D(oublet), T(riplet), and Q(uadruplet), respectively. This information is experimentally provided either by 1D 13 C DEPT spectra or by multiplicity-edited 2D proposals, but the similarity scores for the HOSE code predicted spectra changed: 30.7% for conofoline, close to what is was without the multiplicty information (35.09%), whereas it decreased significantly from 43.17% to 33.93% for structure Q104944410. Even though introducing multiplicity did not produce the expected result when using the chemical shift prediction based on HOSE codes, it made the score of an incorrect best hit a lot less convincing than chemical shifts alone, thus enforcing the idea according to which more information helps to take better decisions.

Compound 3 was reported with 42 [17] Searching the acd_lotusv7 category in nmrshiftdb2 for this spectrum, a compound with 42 carbons was ranked first. The best hit, referenced Q105155240 in Wikidata, was associated with "liquorice" in this database and with the chemical study of Glycyrrhiza uralensis. The publication from which the 13 C NMR data were taken referred to Glycyrrhiza glabra, or sweet liquorice, and to Glycyrrhiza uralensis. [17] The corresponding compound is called glycyrrhizic acid, also known as glycyrrhizin. Its representation as a "flat" molecule is due to lack of configuration data in the original document that was imported in LOTUS. The best hit which had the same formula when searching for the spectrum without a formula restriction belonged to the class of triterpene saponins, as indicated by Classyfire data. This is the same class as for the expected structure.

A few other compounds were tested for dereplication from their 13 C NMR data but their number is certainly too small yet in order to suggest that acd_lotusv7 could be the definitive solution (if any exists) to the problem of the quick identification of already known natural products. However, the first trials were highly encouraging, mainly due to the richness of the content of the LOTUS database and to the quality of chemical shift prediction. No attempt were made to date to identify compounds from partial lists of chemical shifts by means of the acd_lotusv7 database. Reducing the completeness of input data results in less accurate and more diverse structure proposals, as already noticed during the course of many previous dereplication studies based on the CaraMel workflow. [1]

Conclusion

This article reports the building process of the acd_lotusv7 database using the natural product structures contained in the LOTUS database, version 7, and using 13 C NMR chemical shift predictions provided by the ACD/Labs predictor in the validation mode. Database users can carry out structural dereplication either through the search tool of the ACD/Labs software or through the freely accessible interface of the nmrshiftdb2 web site. The latter offers as supplementary feature the possibility of associating the highly useful multiplicity information to chemical shift values in order to retain only the most NMR relevant candidate structures. [6] Database usage is illustrated by examples of various structural complexity. We also show that the quality of the prediction plays a significant role in the success of dereplication. Future works will involve the creation of a protocol intended to minimize the amount of calculation required by the updating for the acd_lotus database subsequently to the publication of new LOTUS versions.

Scheme 1 .

 1 Scheme 1. Structure of compounds 1, 2, and 3.

Figure 1 .

 1 Figure 1. A search for compound 1, restricted to category acd_lotusv7 and formula C 15 H 22 O 3 .

Figure 2 .

 2 Figure 2. The ACD/Labs prediction (column Prediction) with the searched experimental shifts (column Input Shift) and the HOSE code prediction (column Prediction hose_lotusv7. The difference in predicted values for atoms 4, 7, and 11 was almost 4 ppm.

  13 C NMR chemical shifts: 165.1, 58.2, 47.7, 42.1, 54.5, 135.5, 104.0, 143.7, 138.6, 136.7, 128.2, 54.3, 56.5, 90.4, 23.5, 7.3, 26.8, 36.5, 61.6, 168.6, 60.8, 60.3, 50.8, 73.2, 52.7, 53.5, 40.8, 51.2, 136.5, 122.8, 122.4, 128.0, 105.9, 149.7, 52.6, 57.0, 19.7, 23.6, 7.4, 27.9, 34.5, 67.2, and 31.4 ppm.

  13 C NMR chemical shifts: 39.6, 26.7, 89.3, 40.0, 55.6, 17.7, 33.1, 45.6, 62.2, 37.4, 199.4, 128.8, 169.4, 43.6, 26.9, 26.7, 32.2, 48.8, 41.8, 44.1, 31.7, 38.5, 28.2, 16.7, 16.9, 18.9, 23.6, 28.7, 28.8, 179.1, 105.1, 84.5, 77.8, 73.0, 77.3, 172.3, 106.9, 76.8, 77.7, 73.3, 78.4, and 172.0 ppm.

Acknowledgements

JMN thanks the Centre National de la Recherche Scientifique for personal financial support.

Conflict of Interest

The authors have no conflicts of interest to declare.

Entry for the Table of Contents

The quick identification of known low molecular weight organic compounds of natural origin by means of 13 C NMR data is made easy by the querying of the acd_lotusv7 database that includes structures from the LOTUS database and ACD/Labs chemical shift predictions. The open source nmrshiftdb2 web interface at https://nmrshiftdb.nmr.uni-koeln.de provides a simple and free way to retrieve compounds from acd_lotusv7 and to analyze search results.