Maxime Méloux
email: maxime.meloux@protonmail.com

Christophe Cerisara
email: christophe.cerisara@loria.fr

Novel-WD: Exploring acquisition of Novel World Knowledge in LLMs Using Prefix-Tuning

Teaching new information to pre-trained large language models (PLM) is a crucial but challenging task. Model adaptation techniques, such as fine-tuning and parameter-efficient training, are often prone to catastrophic forgetting, and most existing benchmarks focus on task adaptation rather than acquiring new information. This work studies and quantifies how PLM may learn and remember new world knowledge facts that do not occur in their pre-training corpus, which only contains world knowledge up to a certain date. To that purpose, we first propose NOVEL-WD, a new dataset consisting of sentences containing novel facts extracted from recent Wikidata updates, along with two evaluation tasks in the form of causal language modeling and multiple choice questions (MCQ). We make this dataset freely available to the community, and beyond the dataset itself, we release a procedure to build again later on new versions of similar datasets with up-to-date information. In a second part, we explore the use of prefix-tuning for novel information learning, and analyze how much information can be stored within a given prefix. We show that a single fact can reliably be encoded within a single prefix, and that the capacity of the prefix increases with its length and with the base model size.

Introduction

Since the introduction of the Transformers architecture in [START_REF] Vaswani | Attention is All you Need[END_REF], pre-trained language models (PLMs) have become the de facto standard for most natural language processing tasks [START_REF] Chiang | Recent Advances in Pre-trained Language Models: Why Do They Work and How Do They Work[END_REF]. Since those models are typically trained in a semi-or selfsupervised setting, adaptations such as fine-tuning are required to adapt them to downstream tasks [START_REF] Dai | Semi-supervised Sequence Learning[END_REF]Howard and Ruder 2018;[START_REF] Radford | Language Models are Unsupervised Multitask Learners[END_REF].

In addition to requiring task-specific adaptation, large language models are usually unknowing of recent events or novel world knowledge which is not contained in their training set [START_REF] Alivanistos | Prompting as Probing: Using Language Models for Knowledge Base Construction[END_REF][START_REF] Kucharavy | Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense[END_REF]). As a result, applications that rely on up-to-date and accurate knowledge require further updates or adaptation of PLMs after their initial training.

More generally, the question arises as to how models can be taught novel factual knowledge, and how to evaluate the effectiveness of the adaptation.

Fine-tuning has been one of the main proposed approaches to adapt pre-trained models to new tasks and domains. However, full model fine-tuning can lead to catastrophic forgetting [START_REF] Elhage | A Mathematical Framework for Transformer Circuits[END_REF][START_REF] Kirkpatrick | Overcoming catastrophic forgetting in neural networks[END_REF], and can be costly when performed on large models [START_REF] Strubell | Energy and Policy Considerations for Modern Deep Learning Research[END_REF]. Furthermore, [START_REF] Wei | Simple synthetic data reduces sycophancy in large language models[END_REF] showed that when fine-tuning a model on a small corpus with new information, the model instead learns to generate previously unseen information (hallucinations), and that once learned, this behavior is then repeated for other information. Fine-tuning may therefore not enable a model to learn new information that was not seen during training.

Parameter-efficient fine-tuning (PEFT) methods have emerged as an lightweight alternative to full model finetuning, in which only a fraction of the parameters of the original model are modified. PEFT allows for efficiently modifying a small fraction of model parameters using methods such as prefix-tuning [START_REF] Li | Prefix-Tuning: Optimizing Continuous Prompts for Generation[END_REF], adapter-tuning [START_REF] He | Towards a Unified View of Parameter-Efficient Transfer Learning[END_REF] or LoRA [START_REF] Hu | LoRA: Low-Rank Adaptation of Large Language Models[END_REF]. Information can be stored using in-context learning [START_REF] Cs. Logan | Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models[END_REF], prompting (Liu et al. 2023b) or prompt-tuning [START_REF] Lester | The Power of Scale for Parameter-Efficient Prompt Tuning[END_REF] amongst others. PEFT methods have recently experienced a surge of popularity 1 .

In this study, we focus on prefix-tuning [START_REF] Li | Prefix-Tuning: Optimizing Continuous Prompts for Generation[END_REF], a parameter-efficient fine-tuning method in which the pre-trained model parameters are kept frozen, but a small, continuous vector called the prefix is optimized. Based on the idea that context can steer a language model without changing its parameters, prefix-tuning optimizes the model's context as one or several continuous vectors corresponding to either embeddings or to key-query pairs in attention layers, whose effects will be propagated to all activation layers and subsequent tokens. [START_REF] Wang | Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding[END_REF] and Liu et al. (2022a) showed that novel knowledge can efficiently be contextually fed into large language models through prompting. However, the size of a prompt in a given model is limited by the context size of that model. In this paper, we view prefix-tuning as a generalized form of prompting taking continuous values, and having controllable depth and length, and as such, we hypothesize that this method can reliably store significant amounts of factual information. This is backed by the findings of [START_REF] Kossen | In-Context Learning in Large Language Models Learns Label Relationships but Is Not Conventional Learning[END_REF], which argues that incontext learning enables a model to learn information. Our goal is therefore to investigate this question in the case of prefix-tuning, and more specifically how much knowledge can be compressed into the prefix. In addition, by using prefix-tuning rather than LoRA, fine-tuning or adapters, we hope to avoid the hallucination problem mentioned in [START_REF] Wei | Simple synthetic data reduces sycophancy in large language models[END_REF] by working with (generalized) prompts without modifying the existing model weights.

The main contributions of this paper are as follows:

• We introduce NOVEL-WD, a framework for generating new, curated datasets and benchmarks of novel/rare information extracted from Wikidata in order to evaluate the learning of new facts in PLMs. • We evaluate the performance of prefix-tuning for novel knowledge acquisition, and measure the extent to which prefix-tuning can efficiently compress information in different situations.

Related work

Adapting models to new tasks is a relatively old problem. [START_REF] Yoon | Lifelong Learning with Dynamically Expandable Networks[END_REF] showed that dynamically expandable networks can obtain good performance in this setting by slowly increasing model capacity. Lin et al. (2022) explored the task of improving accuracy of Transformer models on outof-data streams using continual model refinement (CMR) to maximize the diversity of training samples in a nonstationary distribution. [START_REF] Razdaibiedina | Progressive Prompts: Continual Learning for Language Models[END_REF] showed that using a collection of progressively growing prompts alleviates catastrophic forgetting and increases model generalization capacities across tasks.

Many studies have explored how information storage functions within the Transformer architecture. [START_REF] Elhage | A Mathematical Framework for Transformer Circuits[END_REF] gave a comprehensive overview of the Transformers architecture under the lens of mechanistic intepretability. [START_REF] Geva | Transformer Feed-Forward Layers Are Key-Value Memories[END_REF] showed that the feedforward layers of Transformers models act similarly to key-value memories in the context of information retrieval systems. Based on that work, [START_REF] Mitchell | Crosslingual Generalization through Multitask Finetuning[END_REF] introduced MEND, a framework that leverages a group of small networks to successfully perform local factual edits within the feedforward layers of a large Transformers model. Meng et al. (2022b,a) expanded on this idea by using causal inference to locate the attention feedforward layer containing a given fact and editing the corresponding matrix as a constrained optimization problem.

In contrast, several approaches for storing new information within a language model have been proposed. One such approach is the use of flexible, external memories, as exemplified in [START_REF] Wu | Memorizing Transformers[END_REF][START_REF] Wu | An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks[END_REF]. Another, dynamic method is that of retrieval systems, which can leverage external knowledge bases, sometimes including the Web, to that purpose. Examples of such works include [START_REF] Guu | REALM: retrieval-augmented language model pretraining[END_REF], [START_REF] Lewis | Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks[END_REF], [START_REF] Borgeaud | Improving language models by retrieving from trillions of tokens[END_REF] and Liu et al. (2023a). Finally, new information can be stored in the shortterm through methods such as prompt-tuning [START_REF] Liu | GPT Understands, Too[END_REF](Liu et al. , 2022b)).

In terms of evaluation, [START_REF] Petroni | Language Models as Knowledge Bases?[END_REF] is an early attempt at measuring relational and factual knowledge within PLMs. [START_REF] Zhu | Modifying Memories in Transformer Models[END_REF] proposed new, information-theory based evaluation metrics for factual knowledge. [START_REF] Kadavath | Language Models (Mostly) Know What They Know[END_REF] and [START_REF] Lin | Teaching Models to Express Their Uncertainty in Words[END_REF] focused on measuring model uncertainty as a way to distinguish properly known facts from hallucinated ones. [START_REF] Jang | Towards Continual Knowledge Learning of Language Models[END_REF][START_REF] Jang | TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models[END_REF] introduced the framework TEMPORALWIKI, which includes a process to generate datasets and benchmarks from information extracted from Wikipedia and aligned with Wikidata triples, with the goal of evaluating the performance of models on new factual knowledge.. [START_REF] Yu | KoLA: Carefully Benchmarking World Knowledge of Large Language Models[END_REF] detailed the creation of a large and refined benchmark, specifically tailored to measure world knowledge within PLMs. [START_REF] Kasai | RealTime QA: What's the Answer Right Now[END_REF] proposed a continual MCQ benchmark for world knowledge, updated every week with new questions about recent events extracted from news websites. [START_REF] Yang | On Robust Prefix-Tuning for Text Classification[END_REF] successfully used prefix-tuning to adapt a PLM to the new task of text classification, while [START_REF] Ma | CPT: Cross-Modal Prefix-Tuning for Speech-To-Text Translation[END_REF] used the same method for speech-to-text translation. Prefix-tuning was also shown to obtain good performance in natural language understanding [START_REF] Lester | The Power of Scale for Parameter-Efficient Prompt Tuning[END_REF], summarization [START_REF] Chen | Incorporating Knowledge into Document Summarization: an Application of Prefix-Tuning on GPT-2[END_REF] and sentiment analysis [START_REF] Balakrishnan | Exploring Robustness of Prefix Tuning in Noisy Data: A Case Study in Financial Sentiment Analysis[END_REF] inter alia. [START_REF] Zhao | Domain-Oriented Prefix-Tuning: Towards Efficient and Generalizable Fine-tuning for Zero-Shot Dialogue Summarization[END_REF] showed that prefix-tuning may also be used for efficient domain adaptation.

The main difference between NOVEL-WD and the datasets found in TEMPORALWIKI lies in the scope and intended use of our dataset. NOVEL-WD is constructed from Wikidata alone, and uses synthetic data for the training and evaluation tasks, therefore limiting the need to download and process large Wikipedia dump files. We intend our approach to be particularly useful to generate a moderate amount of data at a high frequency. Furthermore, we include two evaluation tasks in the form of sentence completion and MCQ. Finally, our training set contains minimal training sentences for each fact of our dataset rather than a larger snippet of text, in order to easily compare the learning capabilities of different models and adaptation techniques.

To our knowledge, no previous studies have been conducted on the use of prefix-tuning to learn novel information, nor on quantifying information storage inside the prefix.

Methodology Research question

In this study, we would like to investigate the following questions:

• Can a simple prefix (i.e. a prefix with a number of virtual tokens of n = 1 and a depth of d = 1) learn a single fact? Does this learning generalize to reformulations of this fact? • Can a larger prefix (n > 1) learn multiple facts? What effect does prefix size have on learning and generalization? In-context learning suggests that the answer to this question and the previous one are positive. • In the existing literature, the prefix is usually spread across all layers of the model. However, recent work [START_REF] Simoulin | How Many Layers and Why? An Analysis of the Model Depth in Transformers[END_REF] suggests that the deeper layers in Transformer models are associated with abstract and high-level capabilities, while factual information is stored in the lower layers. Does restricting the prefix depth d therefore affect the learning and generalization capacities of the model? • Can training metrics be used to indicate overfitting and forgetting in a prefix-tuned model at training time? • Do the answers to the previous questions remain true when model scales?

The assumption is that a smaller prefix may be required when the model is bigger thanks to the extra information the model already has.

Information learning

We model a fact as a semantic triple of the form (subject, predicate, object), in which the subject and object are typically noun phrases, and the predicate a verb phrase. Given a baseline language model L, a modified language model L ′ and a triple T = (s, p, o), we consider the following aspects, largely adapted from Meng et al. (2022a):

• Learning: Given a sentence or question S containing s and p and expecting the continuation o, we consider that

L has learned T if L(S) ̸ = o and L ′ (S) = o.
• Generalization: Given a sentence or question Ŝ ≈ S, we consider that

L ′ can generalize if L ′ ( Ŝ) = o. • Specificity: Given a second triple T ′ = (s ′ , p ′ , o ′ ) such that o ̸ = o ′ and a sentence or question S ′ expecting the continuation o ′ , we consider that L ′ is specific if L ′ (S) = o and L ′ (S ′ ) ̸ = o. • Non-forgetting: We consider that L ′ has retained the fact contained in T if L(S) = o and L ′ (S) = o.

Evaluation

Given a baseline model L and a list T = T 1 , ..., T p of facts encoded as triples of the form T i = (s i , p i , o i ), we evaluate the efficiency of an adaptation technique by applying the following approach:

• We apply the adaptation technique on L, using as training set a dataset D containing a list of simple sentences previously generated from the triples of T . • We evaluate learning in the resulting model L ′ , by measuring its perplexity in a causal language modeling setting on the sentences of D, and comparing it to that of L. • We evaluate generalization by measuring the perplexity of L ′ in a causal language modeling setting on complex, creative sentences created by reformulating the sentences of D. • We measure specificity and non-forgetting by evaluating L and L ′ on existing MCQ benchmarks.

Dataset

In this section, we describe the steps used to create NOVEL-WD and give an overview of the resulting dataset. A sample output of each step of the full process is given in

Triple extraction

We begin by extracting RDF triples that were newly added to Wikidata. To do so, we retrieve new triples from a daily incremental database dump. We restrict ourselves to items and exclude lexemes, which represent lexicographical data. We also do not take into account complex triples, in which the subject or object is a Wikimedia template, as well as triples in which the subject is a numerical identifier, a filename or a URI. We then resolve eventual internal Wikidata links in the subject, predicate or object by replacing them with the English name of the associated item. Finally, when multiple triples share the same subject and predicate, we randomly select one such triple and discard the other ones, so as to limit the risk of models trying to learn multiple conflicting facts.

Training set

To generate a training set, we convert each triple into a simple sentence. In order to do so, we query a 8-bit quantized version of VICUNA-13B [START_REF] Chiang | Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality[END_REF]) with a twoshot prompt. For each triple, we generate one such sentence.

Test sets

To evaluate the performance of models on our dataset, we provide two different evaluation tasks.

The first task is a causal language modeling task: For each triple, we used 8-bit VICUNA-13B in a two-shot setting to generate five sentences in which the object of the triple is missing. In order to test for generalization capabilities and to avoid repeating the training sentence, we specifically prompted the model for "creative sentences". Manual editing was then applied to the output sentences in the infrequent situation (occurring for fewer than 10 facts) where full sentences were generated rather than an incomplete one.

The second task is a question answering task in the multiple choice question setting (MCQ). For each triple, a twoshot 8-bit VICUNA-13B prompt was first applied to generate a question asking for the object of the triple. Then, a similar prompt was applied to generate four "likely answers" to the question. Among the four generated answers, we remove the ground-truth one if it is present, and select the three first remaining ones as distractors. After manually checking and editing the generated answers in rare cases (3 occurrences) where they semantically overlap, we then add in the correct answer. We therefore obtain a question with four possible choices, exactly one of which being correct.

Final dataset

After all the steps above have been applied, NOVEL-WD consists of 338 distinct triples, and each triple contains one associated training sentence, five incomplete validation sentences, one question and three distractors.

Experimental setup

The baseline model chosen for our experiments is the 7.1billion parameter version of BLOOMZ (Muennighoff et al. 2023), BLOOMZ-7B1. The training was ran for up to 450 epochs using the AdamW optimizer with a weight decay of 0.1 and an initial learning rate of 3 * 10 -2 , decreasing by a factor of 10 after 10 epochs of non-decreasing training loss. We did not project the prefix through an intermediate MLP as mentioned in [START_REF] Li | Prefix-Tuning: Optimizing Continuous Prompts for Generation[END_REF], as we found that it did not increase training stability and generally resulted in lower performance.

For all of our models, prefix-tuning was implemented by learning the value of the previous key and value vectors in attention layers, resulting in two vectors per layer and per virtual token being learned, for a total of 2 * d * n vectors.

For each macro-experiment and number of facts k, we divided the D = 338 facts of NOVEL-WD into non-fully overlapping subsets of length k, and trained one copy of the baseline model on each subset. For a given k, the number of subsets was computed as max(5, ⌊D/k⌋). The resulting number of experiments can be found in Table 2. For example, for k = 3, we sampled 112 subsets of 3 facts, and trained a separate copy of BLOOMZ-7B1 on each of those 112 subsets. Due to the restricted dataset size, there exists significant overlap between the training sets of experiments when k ≥ 50. Table 2: The number of subsets for each number of facts.

In each macro-experiment, one model was trained on each split.

For consistency, during the evaluation phase, we only evaluated a model using the sentences and questions of NOVEL-WD corresponding to the k triples it had been trained on.

Evaluation

To evaluate our models in the text prediction setting, we prompt them with each of the five incomplete sentences associated with each fact from the training set, and generate the following ten tokens without sampling and with a temperature of 1. We only count an answer as correct if the model's output contains the exact answer's text, capitalization excepted, and we report the accuracy over every sentence of the test set for a given model. We also measure the proportion of learning models for a given k, by selecting only facts of the test set for which the baseline model does not output any correct prediction, and counting the proportions of the prefix-tuned models trained on those questions for which the test set accuracy is non-zero. In other words, learning models are models which are able to correctly predict at least one sentence completion for facts that were not known by the baseline.

To perform regression tests, we selected the SciQ (Welbl, Liu, and Gardner 2017) and MMLU (Hendrycks et al. 2020a;Hendrycks et al. 2020b) datasets. For SciQ, we measure the accuracy of the baseline and prefix-tuned models in the MCQ setting, by using the same prompt as for NOVEL-WD, and selecting the lowest per-token perplexity choice. We apply this method on all 1,000 questions of the test set. For MMLU, we append each of the possible four completions to each sentence, and then select the one with the lowest per-token perplexity as the model's answer. This is applied to the test sets from each of the 57 categories found in the dataset. Due to computational costs, regression tests were ran on a random sample of 5 prefix-tuned models for each value of k.

Results and analysis

Base setup

Our initial experiment focuses on a single prefix (n = 1, d = 1), corresponding to 8,192 trainable parameters, or 0.000116% of the baseline model's parameters.

The proportion of prefix-tuned models with increased accuracy in the prediction setting is given in Figure 1 For 1 ≤ k ≤ 3, between 54.1% and 55.4% of the models are successfully able to learn at least one information over the baseline. This amount stays relatively consistent for k ≤ 10, with the proportion of learning models ranging from 40.5% to 55.4%. For k = 20, this proportion drops to 18.8%, and none of the models trained for k > 20 achieved any accuracy gains over the baseline.

The baseline model obtains a consistent accuracy ranging from 3.0% to 6.3%, suggesting that a small number of facts found in the dataset are either already known or easily deducible by the model. In contrast, the prefix-tuned models obtain a mean accuracy peaking at 29.1% for k = 3, and gradually decreasing for k > 3 until k = 50, for which the results are no longer significantly better than the baseline.

This initial result suggests that during training, the prefix is usually able to select and remember 1 to 3 facts well, and up to 20 with decreasing accuracy. Furthermore, this learning is conditional on having a low enough number of facts present in the training data; having more than 10 facts seems to hamper the model's ability to learn even a single fact.

Error analysis In the case of k = 1, close to half of the facts found in NOVEL-WD were not successfully learned by a single prefix. While we could not identify any meaningful semantic or content differences between the types of facts that were learned and those that were not, we report in Table 3 quantitative statistics between those two categories. For each reported statistic, the NL value was found to be significantly larger than the L one, as measured using a one-sided Welch's t-test (p = 0.05).

This suggests that the facts that were not successfully learned are typically longer and are farther from the baseline model's distribution, both in their sentence form and in the text completion setting, which might result in an inability for prefix-tuning to sufficiently steer the model towards learning them.

Detecting overfitting and forgetting

We report in Figure 3 We observe that for k = 1, almost all experiments end with a training loss approaching zero, with the exceptions of a few outliers for which the loss remains high. This confirms our previous finding that the prefix is almost always able to learn a single fact, but may not be able to generalize in the prediction setting. When increasing k, the losses increase linearly up to k = 10 (median value: L train = 0.38). For n ≥ 20, the loss increases sharply and quickly approaches the baseline model's loss of 4.38. We interpret this inflection as consistent with our previous observations, suggesting that a change of learning mode occurs in the vicinity of k = 15: For lower values, the model is efficiently able to learn and generalize novel information, while for higher values, we hypothesize that the model is no longer able to store all facts and instead unsuccessfully attempts to learn a combined representation of the training set. These findings are also consistent with the evolution of the prefix norm given: For 1 ≤ n ≤ 3, we observe a linear increase in prefix norm, which may indicate that the model does not make full use of the available prefix capacity. For 3 ≤ n ≤ 10, the prefix norm is nearly constant and may signal increasing compression within the prefix. Finally, for n ≥ 10, the prefix norm decreases rapidly.

Finally, we report in Table 4 the results of the evaluation over SciQ and MMLU, which shows that the prefixtuned models do not seem to forget facts learned during pretraining or incur any loss of reasoning capabilities, for any value of k. Surprisingly, our prefix-tuned models even perform consistently and significantly better than the baseline for all values of k. We did not have time to study this difference in detail, and leave this question open for future work. For MMLU, we report the score obtained by the lowest and highest accuracy as well as the average across categories. nearly all values of k when increasing the prefix size from 1 to 20, as well as significant gains in the proportion of learning models for k ∈ {1, 4, 20, 100}. Similar results are obtained when further increasing the prefix size from 1 to 100. However, none of the variation in accuracy or proportion of learning models between n = 20 and n = 100 are statistically significant.

Effect of prefix size

We interpret those results as follows: Increasing the prefix size only modestly increases the chances for a model to be able to learn at least one fact. However, such an increase has a strong impact on the prediction capabilities of the model, which suggests that the model is able to learn more facts and to generalize better.

We hypothesize that the former may stem from the varying complexity of the facts in our dataset: for some facts, the base model may already contain information about the the subject and predicate, and prefix-tuning might only be needed to learn the value of the object. A typical example of this situation can be found in facts of the type "[historical figure] was born on [date]". On the contrary, there exist more complex facts for which the subject and predicate themselves might be novel, and for which the base model might not contain information. We also note that increasing the prefix size past 20 brings no further improvement to the learning and generalization capacities of our model, which may indicate that prefixes are inherently limited in terms of information capacity.

Effect of prefix depth

We report in Table 6 the results obtained by increasing the number of layers spanned by the prefix in our initial setup from d = 1 (minimal depth) to d = 30 (full-depth prefix).

We observe that increasing the prefix depth as a significant effect on both the accuracy and the proportion of learning models. For all values of k, the average accuracy is increased by 8 to 31%, with the highest increase reached for k = 10. which once more suggests that up to three facts can be efficiently stored within a prefix, but performance stays comparable up to k = 10.

The second main observation is the fact that the proportion of learning models significantly increases for all values of k except k = 1, with gains of up to 80% for k = 100. Generally, we hypothesize that increasing the prefix depth allows for much more complex information to be learned, and enables the model to learn at least one information for all but the highest amount of facts in the training set.

Increasing the value of d from 1 to 30 effectively multiplies the number of trainable parameters by 30, but far surpasses the results obtained by increasing the prefix length by a factor of 100. We therefore remark that prefix depth seems to have a much stronger effect on model performance than prefix length.

Effect of base model

To investigate the effect that the type and size of the base model may have on prefix-tuning, we repeat our initial experiments on two additional models: BLOOMZ-1B7, the 1.7 billion parameter version of BLOOMZ, was chosen for scale comparisons.

We report in Table 7 the prediction accuracy obtained on the entire dataset with no prefix-tuning, and in Table 8 ter being significantly larger than the former.

In terms of scaling, we first note that there are no significant improvements in terms of the proportion of learning models between BLOOMZ-1B7 and BLOOMZ-7B1. This strengthens the intuition that this may be due to the inherent complexity of some facts in the dataset, and to the fact that the ability to learn a fact is already present in smaller models. However, increasing the model size has a noticeable effect on the prediction accuracy, which increases by several percentage points for k ∈ {4, 5, 10, 20, 50}. We believe that this is partially due to the scaling generalization capabilities of the models. However, as the number of trainable parameters almost doubles between BLOOMZ-1B7 and BLOOMZ-7B1, these improvements may also be explained by an increase in prefix capacity.

Conclusion

In this study, we have developed a dataset for novel fact learning in pre-trained language models. We have shown that prefix-tuning can be used to learn new facts, and investigated the effect of various factors on prefix-tuning performance. Our main recommendation is to use full-depth prefixes when training, but to limit the prefix length to a maximum of 20 virtual tokens.

We see several major avenues for future research based on this work. While we measured the effect of different factors independently, their combined effect might be different. In particular, it is hard to predict how prefix length and depth may interact together. Another research direction is the use of different and more recent baseline architectures such as Falcon [START_REF] Almazrouei | Falcon-40B: an open large language model with state-of-the-art performance[END_REF]. Finally, a long-term goal could be to scale our approach to larger datasets, for example by using a mixture of prefixes at capacity along with a routing module. This could allow the use of a small, regular stream of new information to continually update a model.

The entirety of the code used to create NOVEL-WD and perform our experiments can be found on GitHub.

  , and Figure 2 contains the mean accuracy obtained in the prediction setting for different numbers of facts.

Figure 1 :

 1 Figure 1: Percentage of prefix-tuned models obtaining increased accuracy over the baseline in the prediction setting, with error bars spanning 95% confidence intervals.

Figure 2 :

 2 Figure 2: Mean accuracy of prefix-tuned (PT) models and of the baseline in the prediction setting, with error bars spanning 95% confidence intervals.

  the final training loss of each experiment, and in Figure 4 the norm of the two vectors contained in the prefix at the end of the training phase of each experiment.

Figure 3 :

 3 Figure 3: Final training loss of each experiment in the basic setup.

Figure 4 :

 4 Figure 4: Frobenius norm of the key and value vectors of the prefix at the end of each experiment in the basic setup.

Table 1 .

 1 Element Value Triple (Frances Allen, spouse, Jacob Schwartz) Train sent. Frances Allen is married to Jacob Schwartz. Test sent. 1 Frances Allen's spouse is Test sent. 2 The spouse of Frances Allen was Test sent. 3 Frances Allen was married to Test sent. 4 Frances Allen has been married to Test sent. 5 The name of Frances Allen's spouse is

	Question	Who was Frances Allen's spouse?
	Distractor 1 Charles Householder
	Distractor 2 David Padua
	Distractor 3 John Cocke

Table 1 :

 1 A sample of the dataset for a single triple.

Table 3 :

 3 Quantitative comparison of the facts of NOVEL-WD that were successfully learned (L) and those which were not (NL) within a single prefix. Reported values are averaged per category.

	Train set	Test set

Table 4 :

 4 Accuracy of the models on the MMLU and SciQ datasets, averaged over 5 random models for each value of k.

		SciQ acc.		MMLU acc.
	k		Min	Max	Avg
	Baseline	0.757	0.130 0.463 0.307
	1	0.833	0.184 0.512 0.343
	2	0.864	0.189 0.517 0.341
	3	0.840	0.189 0.517 0.340
	4	0.838	0.184 0.517 0.339
	5	0.827	0.191 0.509 0.339
	8	0.833	0.184 0.509 0.341
	10	0.834	0.193 0.509 0.341
	20	0.808	0.185 0.515 0.328
	50	0.835	0.190 0.518 0.335
	100	0.826	0.192 0.512 0.340
	200	0.828	0.189 0.524 0.342

Table 5 :

 5 Proportion of learning models (pLM) and mean prediction accuracy for different number of virtual tokens n in the prefix. Bold values denote statistically significant improvements over n = 1, using a one-sided z-test for proportions for pLM and a one-sided t-test for the accuracy (p = 0.05).

	Table 5 contains the results obtained when prefix-tuning in-
	stances of BLOOMZ-7B1 while varying the number of vir-
	tual tokens n contained in the prefix.
	We observe significant improvement in accuracy for

Table 6 :

 6 The highest average accuracy is obtained for k = 3, Proportion of learning models (pLM) and mean prediction accuracy for different prefix depths d in the prefix. Bold values denote statistically significant improvements over d = 1, using a one-sided z-test for proportions for pLM and a one-sided t-test for the accuracy (p = 0.05).

		d=1	d=30
	k	Acc	pLM	Acc	pLM
	1	0.274 0.541 0.354 0.590
	2	0.279 0.548 0.441 0.667
	3	0.291 0.554 0.520 0.768
	4	0.247 0.464 0.467 0.690
	5	0.227 0.493 0.470 0.731
	8	0.177 0.405 0.487 0.690
	10 0.159 0.485 0.476 0.789
	20 0.123 0.188 0.401 0.813
	50 0.076	0	0.275 0.333
	100 0.053	0	0.130 0.800
	200 0.055	0	0.101 0.000

Table 8 :

 8 the results obtained after prefix-tuning. Proportion of learning models (pLM) and mean prediction accuracy for different number of virtual tokens n in the prefix. Bold values denote statistically significant improvements over the previous column, using a one-sided z-test for proportions for pLM and a one-sided t-test for the accuracy (p = 0.05).

	Model	BLOOMZ-1B7 BLOOMZ-7B1
	Acc	0.044	0.050
	Params/prefix	4,096	8,192
	Table 7: Comparison of the baseline models through their
	accuracy in the prediction setting over the entirety of
	NOVEL-WD, and the number of parameters contained
	within a single prefix (n = 1, d = 1).
	We first observe that BLOOMZ-1B7 and BLOOMZ-
	7B1 share a similarly low baseline accuracy, despite the lat-

As of August 12,

2023, the peft Python package has been downloaded over 430,000 times during the last month.

Acknowledgments

The authors would like to thank the ORION program for its contribution to the funding of MM's research internship. This work has benefited from a French government grant managed by the Agence Nationale de la Recherche with the reference ANR-20-SFRI-0009. Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr). This work was granted access to the HPC resources of IDRIS under the allocation 2023-AD011011668R3 made by GENCI.


Nicolas Poirier

Carine Calastrenc

Girousse Laurent

de mission. Acquisition de données radar. Mission du 05/07/22 -Abbaye de Belleperche, Cordes-Tolosannes (82)

come

Compte-rendu de mission Acquisition de donne es radar

Mission du 05/07/22 -Abbaye de Belleperche, N. Poirier, C. Calastrenc, L. Girousse

Objectif et contexte

La mission d'acquisition de données radar du 05/07/2022 a été réalisée sur la commune de Cordes-Tolosannes, autour de l'abbaye de Belleperche, dans le cadre d'un projet d'intervention archéologique présenté par P. Georges-Zimmermann (INRAP).

Conditions d'intervention

Implantation des grilles

Les grilles de prospection radar ont été implantées de manière à couvrir les plus grandes surfaces possibles tout évitant les différents obstacles : arbres, puit, etc.

Au total, 4 grilles de prospection ont pu être réalisées :

-BELP001 : 17,50 m X 39 m -BELP002 : 12,50 m X 39 m -BELP003 : 18 m X 28 m -BELP004 : 38 m X 28 m

Matériel utilisé

Le matériel utilisé comprend le système SIR4000 de GSSI couplé à une antenne de 400 MHz.

Les données ont été acquises en bandes parallèles espacées de 0.5 m avec l'antenne de 400 MHz et en bades parallèles espacées d' 1 m avec l'antenne de 200 MHz.

Les grilles ont été implantées en utilisant des décamètres ruban et les points origines des grilles localisés avec ces mêmes décamètres par triangulation sur des points remarquables (angles bâtiments, de clôtures, etc..

Résultats

La pénétration du signal dans le sol ne permet pas une lecture satisfaisante au-delà de 2,50m à 3m où le rapport signal/bruit est trop important.

La majorité des anomalies identifiables sont repérées entre 80 cm et 150 cm de profondeur. Nous produisons donc pour chaque grille une carte à la profondeur de présence maximale de ces anomalies :

-53 cm, 80 cm et 160 cm pour BELP001

-36 cm, 108 cm et 127 cm pour BELP002

-100 cm et 110 cm pour BELP003 -52 cm, 80 cm, 107 cm, 150 cm et 232 cm pour BELP004

Ces 13 cartes ont été géoréférencées au sein du logiciel QGIS.

Interprétation

A 50 cm de profondeur, seules les grilles BELP001 et BELP002 livre des anomalies interprétables comme des indices de vestiges archéologiques. A l'est de ces 2 grilles, lles deux anomalies linéaires qui se croisent peuvent correspondre à des canalisations (l'une de direction nord-sud semble conduite au puit) ou des cables électriques.

A 100 cm de profondeur, on repère des anomalies linéaires orthogonales à l'est de BELP001 (en blanc) et des anomalies massives (en bleu) à l'ouest des grilles BELP001 et BELP002. Ces anomalies pourraient correspondre aux vestiges de l'aile des moines et du petit cloître, ou encore de la résidence abbatiale de la Renaissance (cf. Fig. 2 supra). La grille BELP003 livre un plan de maçonnerie quadrangulaire pouvant correspondre au chevet plat de l'église abbatiale.

A 150 cm de profondeur, on lit toujours l'empreinte des vestiges probables de l'aile des moines en BELP001 et BELP002, tandis q'un plan plus complet de l'église abbatiale se dessine en BELP003 et BELP004, avec des anomalies linéaires qui semblent dessiner le plan du chevet et de probables chapelles latérales (?), un mur ou délimitation de collatéral nord, et des anomalies plus ponctuelles mais massives au sud (bases de piliers?).

Nous laissons le soin aux responsables thématiques du projet, mieux au fait qu enous de l'histoire du site, le soin d'interrpréter plus finement ces résultats. Nous concluons en soulignant la bonne qualité des données acquises et la bonne pénétration du signal dans le sol qui permet une lecture fine des anomalies jusqu'à plus de 2 m de profondeur.

L

  'objectif était d'évaluer le potentiel archéologique des espaces extérieurs de l'abbaye à l'emplacement d'un épais remblais en ayant recours à la technologie GPR (Ground Penetrating Radar). La zone prospectée est susceptible de correspondre à l'emplacement de l'église abbatiale (75m x 20m d'après les sources écrites, démolie au XIXe s.), d'un petit cloître et d'une résidence abbatiale de la Renaissance.

Figure 1 :

 1 Figure 1 : Cordes-Tolosannes (82) -Mission du 05/07/2022 -Localisation de l'abbaye de Bellperche

Figure 3 :

 3 Figure 3 : Cordes-Tolosannes (82) -Mission du 05/07/2022 -Localisation des zones prospectées

Figure 4 :

 4 Figure 4 : Cordes-Tolosannes (82) -Mission du 05/07/2022 -Mise en oeuvre du géoradar

Figure 5 :

 5 Figure 5 : Cordes-Tolosannes (82) -Mission du 05/07/2022 -Exemple radargramme avec limite signal/bruit

Figure 6 :

 6 Figure 6 : Cordes-Tolosannes (82) -Mission du 05/07/2022 -Représentation 2D à 50 cm de profondeur


Les acquisitions

Cette acquisition avait pour but l'obtention d'un modèle 3D du site et de son environnement, mais aussi la réalisation d'un Modèle Numérique de Terrain sans la végétation. Au total, 24 stations ont été réalisées. Ce qui a permis de couvrir une surface d'environ 3000 m², dont 500 m², correspondant aux zones de fouilles (au nord-est) , le reste des acquisitions ont concernées la partie sud et est du site. Chaque scan a donc, en moyenne, permis de couvrir 208 m².

Matériel utilisé : Faro Focus3d 120S

Ce levé topographique a été réalisé avec un scanner Faro Focus3d 120s. Cet appareil produit un faisceau laser invisible d'une longueur d'onde de 905 nm (infrarouge). Il s'agit d'un ensemble compact laser+scanner qui permet une acquisition de nuages de points 3D avec une grande précision (incertitude de mesure de 2mm à 10m). Un appareil photographique intégré permet également d'obtenir des clichés à 360 degrés des zones scannées Les photographies produites sont utilisées pour affecter la valeur de la couleur aux points et ainsi obtenir un nuage de point colorisé.

Les paramètres d'acquisition sont les suivant :

-Résolution : 1/5 ou 1/4 et qualité 4x, -Vitesse d'acquisition : 122 000 points par secondes. Ce qui représente entre 29 et 44 millions de points par scan.

Ces paramètres favorisent une acquisition rapide (chaque scan prend environ 10 minutes), rendue obligatoire par le temps disponible sur site (une journée).

Pour les acquisitions de la zone basse correspondant aux zones du chantier de fouilles, l'acquisition s'est faite en couleur. Dans un premier temps, l'assemblage des nuages a été réalisé avec la première méthode (la méthode automatique). Il a permis d'extraire un Modèle numérique de terrain avec une résolution spatiale de 5cm et une erreur moyenne d'un demi cm ce qui représente déjà une bonne qualité du positionnement.

Les scans

Dans un second temps, l'assemblage suivant la deuxième méthode sera réalisée afin de préciser le modèle.

Traitements des nuages de points

Une fois les nuages alignés, les différents scans ont été groupés en un seul nuage qui a été exporté dans le logiciel CloudCompare pour les traitements de filtrage, de ré-échantillonnage et de segmentation Pour des questions de temps de traitement, il a été nécessaire de ré-échantilloner le nuage de point. Dans un premier temps, il a été choisi de le filtrer pour éliminer le "bruit". Cela consiste à supprimer les points isolés créés par rebond du signal sur un élément situé au premier plan de type feuille ou branche ou par mauvaise appréciation d'une bordure d'un objet ou d'un relief. Cette méthode de filtrage consiste à définir une limite autour du point et de calculer la distance qu'il a avec chacun de ses voisins dans la limite définie ainsi que la moyenne des distances et l'écart-type.

Filtrage du bruit du nuage de points brut et ré-échantillonnage

A partir de ces valeurs, si un point est éloigné de plus de la distance moyenne + X écart-type (X est une valeur à donner), le point est considéré comme bruit et supprimé (Fig. 4).

Segmentation du nuage de points

La segmentation, ou classification du nuage de points, a été réalisée de différentes façon. Par manque de temps, elle n'a pu être réalisé que sur la zone basse de la zone de fouille. L'objectif était de pouvoir comparer et combiner les résultats.

Les 3 outils utilisés sont les algorithmes suivants :

-CSF (Zhang W, Qi J, Wan P, Wang H, Xie D, Wang X, Yan G.

Canupo

Le principe de Canupo est de définir une thématique par des ensembles de points. Ainsi dans un premier temps, on crée deux nuages de points qui contiennent des extraits sélectionnés de portions de sol ou de sursol. C'est à partir de ces échantillons que l'outil va définir ce qui est du sol ou pas. Le résultat est mitigé. Avec cet outil aussi le sol est très bien défini. Si l'abri sous roche n'a pas disparu et si des rochers ont bien été détectés comme faisant partie du sol, il manque des morceaux de l'abri sous roche et des entités flottent au-dessus du sol (Figure 7). Le travail d'échantillonnage doit être affiné pour avoir un très bon résultat. Si le nuage final du sol peut être très proche de la réalité, le frein de cet outil est le temps très long de calcul qui limite les essais.

Label Connected Components (LCC).

Cet outil segmente le nuage de points en morceaux séparés par une distance minimale. Chaque morceau correspond à un élément connecté, c'est à dire composé de points proches.

Les paramètres à renseigner sont l'échelle de segmentation et le nombre minimum de points par composant. La valeur donnée à l'échelle influe directement sur le nombre de composants possibles : plus la valeur est grande, plus l'espace est divisible et plus le nombre de composants séparés peut être trouvé. Le nombre minimal de points permet de supprimer les composants dont le nombre de points est inférieur à cette valeur. Il permet aussi de décomposer les entités en petites parties. Si le paramétrage est très simple, il influe directement sur le temps de calcul qui peut être très important pour une valeur d'échelle grande (> 12). Cet outil ne réalise pas une classification sol-sursol, mais il découpe l'espace en composants. Chacun peut être sélectionné pour composer le nuage final du sol. Avec un paramétrage suffisamment fin, le nuage entier est segmenté au prix d'un temps de traitement très long. Cela n'est pas nécessaire car le sol est un ensemble continue de points et donc un "unique" composant. En augmentant le nombre minimum de points par composant, on élimine les petits morceaux qui ne représentent pas le sol. L'avantage de cet outil est qu'il ne supprime pas de morceaux du nuage de points comme peuvent le faire Canupo ou CSF, il créé des groupes, des ensembles de point que l'on peut ou pas supprimer. Son paramétrage est très simple, toutefois le temps de traitement et de sélection des composants est (très) long.

La Figure 8 montre le résultat de la segmentation de la zone basse de la zone de fouille avec une résolution distance mini de 1.5 cm et un nombre minimum de 100 points par composant. Après traitement, cela représente 10783 composants. Si cet outil détecte très bien le sol comme grande et unique entité, il élimine également une grande partie du sursol. Cependant, les éléments qui non pas de rupture de continuité avec le sol sont aussi considérés comme parties intégrantes du so ; les enlever demande soit d'utiliser un autre algorthime, soit de les supprimer à main.

Conclusions

La finalité du nettoyage du nuage de points est de pouvoir faire des mesures exactes mais aussi de produire des données dérivées : coupe, plan et MNT. Le nuage de point ayant été géoréférencé, il se trouve donc dans un repère métrique. Il est alors tout à fait possible de réaliser toutes les mesures nécessaires (Figure 12) comme si l'opérateur se trouvait sur site. Le travail réalisé sur le nuage de point du site de Basi permet de produire un de la zone basse avec un résolution de 2cm (Figure 14). Afin de renforcer le relief, ce dernier a été ombré Le levé topographique du site de Basi, qui a durée 9 heures,a permis, à partir de 24 stations, de modéliser en 3D la zone d'étude sous forme d'un nuage de points. Après XXXX heures de traitement de filtrage et de segmentation des données, un nuage de points au plus près du sol a été obtenus C'est sur ce dernier qu'il est maintenant possible de s'appuyer pour réaliser de nombreux produits dérivés comme des coupes, des MNT ou plus simplement de faire des mesures en 3D.

Les données (données de base et produit fini)ont été sauvegardées pour une utilisation pérenne et peuvent être mobilisées à tout moment pour des compléments d'étude.

  Figure 2). Il s'organise à l'intérieur d'une zone de chaos rocheux prit à l'intérieur du maquis. La végétation, très dense, est constituée de d'arbre épars sous lequel se développe une végétation ligneuse basse importante et de liseron épineux. L'ensemble forme un couvert végétal très important.

Figure 1 :Figure 2 :

 12 Figure 1 : carte de situation du site de Basi. Sources Actualitix et IGN

Figure 3 :

 3 Figure 3 : Positions des scans sur le chantier. La partie "basse" du chantier se situe dans la moitié nord de l'image

Figure 4 :

 4 Figure 4 : Exemple de filtrage du bruit par CloudCompare. Source CloudCompare

Figure 5 :

 5 Figure 5 : Extrait du nuage de points qui représente une partie du chantier

Figure 6 :

 6 Figure 6 : Classification CSF. Le sol est très bien détecté, mais l'abri disparait ainsi que des parties verticales des rochers (en rouge). Par contre, de l'outillage reste non détecté ainsi qu'une souche (en orange)

Figure 7 :

 7 Figure 7 : : Résultat de la classe "sol" de l'outil Canupo. Des parties de l'abri sous roche ont disparu et des morceaux d'entités n'ont pas été nettoyées (entourés en rouge).

Figure 8 :

 8 Figure 8 : Segmentation par LCC. Chaque composant est sélectionnable pour être conservé ou non du nuage de points final.

Figure 9 :Figure 10 :

 910 Figure 9 : Segmentation LCC. Des composants n'ont pas été détectés

Figure 11 :

 11 Figure 11 : Segmentation LCC. En couleur le sursol traité manuellement qui ne sera pas pris en compte pour les profils et le MNT

Figure 2 :

 2 Figure 2 : mesure d'une longueur dans l'abri sous roche du niveau supérieur du site

Figure 3 :

 3 Figure 3 : Extraction d'un profil de coupe sur la partie basse de la zone

Figure 4 :

 4 Figure 4 : MNT de la zone basse

  Ainsi pour chaque point a été affecté une teinte. Cette teinte est une information supplémentaire qui ne se substitue pas aux valeurs "d'intensité" mesurées par le laser et qui sont stockés en nuance de gris. Cette valeur d'intensité correspond au signal retour du laser analysé par le scanner. Ce signal est influencé par de nombreux paramètres : distance, réflectivité des matériaux, angle d'incidence, conditions atmosphériques et bien sûr l'électronique du scanner et les paramètres du laser.

Le nuage de points visualisé en nuances de gris représente très bien l'environnement de l'acquisition avec notamment un très bon rendu du relief. Si l'acquisition en couleur donne de bons résultats en pleine journée avec un fort éclairage, cela n'est pas le cas quand la luminosité diminue. Les dernières acquisitions ont été réalisé à la fin du jour, voir en pleine nuit, et donc n'est visible seulement qu'en nuance de gris. C'est avec les scans bruts en intensité "nuance de gris", non biaisée par l'éclairage, que nous avons réalisé l'assemblage des nuages de points.

. Assemblage des scans L

  'assemblage des 24 nuages de point s'est faits avec le logiciel Faro SCENEIl a été choisi de réaliser un assemblage par cible qui allie rapidité et précision. L'un des avantages de ce logiciel est de pouvoir mesurer l'erreur de positionnement de chaque scan et ainsi pouvoir jauger de la qualité du modèle 3D réalisé. Ainsi, pour le modèle du site de Basi, les valeurs moyennes de l'erreur de positionnement des cibles est de 4.8 mm. Le meilleur positionnement est de 0.8 mm et le moins bon est de 9.7 mm. Le nombre moyen de connexion entre les scans, est de 9. Ainsi, sur les 24 scans réalisés, en moyenne, chaque scan est connecté à 9 autres. Cela est valable pour l'ensemble des scans réalisés, hormis pour le dernier qui n'est relié qu'à deux scans. Le scan 15 (Fig.3) est celui qui est interconnecté avec le plus grand nombre de scan. Ainsi il « interagit » avec 17 autres scans. Ce premier alignement automatique pourrait être amélioré, c'est-à-dire que l'erreur de positionnement des scans pourrait être réduire, en réalisant un assemblage avec le logiciel CloudCompare. Pour ce faire, il faut sélectionner manuellement les correspondances entre les différents repères et ajuster l'alignement en utilisant l'algorithme ICP (Iterative Closest Points).

	4
	assemblage se
	faisant en post traitement, une erreur de positionnement compromet l'assemblage des scans.
	Les sphères ne sont utilisées que pour l'assemblage des scans. Les Damiers servent à la fois à
	l'assemblage et « au géoréférencement » du référentiel "scanner" dans le repère utilisé sur le
	chantier. A cette fin, nous avons utilisé une station totale Trimble M3 pour réaliser le levé des
	damiers dans le référentiel topographique du chantier. Le nombre de scans (24 stations) réalisés
	est imposé par plusieurs facteurs : la distance de reconnaissance des sphères, qui doit être
	inférieure à 20m, la nécessité de réduire au maximum les angles morts de façon à effectuer une
	description la plus fidèle possible de la zone et de la topographie du terrain. Toutefois,
	l'expérience montre qu'il reste toujours des zones non documentées, à cause d'angles morts et/ou
	d'obstacles sur la trajectoire du laser. Un des moyens pour limiter ces zones d'ombre est de
	multiplier les angles de relevé et ainsi le nombre de stations.
	Pour ces 24 stations, le temps de relevé terrain a été de 9:30 heures. Il a débuté à 10:30 h le matin
	et s'est fini à 20 heure. Durant cette période de temps, les repères ont été positionnés, leurs
	coordonnées ont été relevées à la station totale et les scans ont été réalisés. Certaines zones ont
	dû être élaguées afin de supprimer la végétation trop envahissante et ainsi pouvoir scanner des
	zones entières en limitant le nombre de scan..

L'assemblage des nuages se prépare sur terrain. Des repères artificiels, des cibles à Damiers et des sphères, ont été positionnés de façon à ce qu'au moins 5 repères soient visibles sur chaque scan et que qu'au moins 3 sphères soient communes entre deux scans. Le positionnement de ces repères est primordial car il permet d'une part l'assemblage des scans et d'autre part leurs « géoréférencement ». Ce positionnement doit être minutieusement réalisé car l'C'est algorithme permet d'améliorer l'erreur de positionnement en faisant effectuer des rotations et des translations successives d'un nuage de points par rapport à un autre (que l'on indique comme étant le scan de référence). Ces rotations et translations sont effectuées jusqu'à atteindre une stabilisation de l'erreur. Cette deuxième méthode d'assemblage réclame beaucoup plus de temps de traitement

  Une fois le nuage de point groupé réalisé, il est nécessaire d'effectuer un filtrage du bruit et un rééchantillonnage.. Le nuage de points brut comporte 360 millions de points et représente un poids de 15.8 Go au

format ASCII (x, y, z, R, G, B) et de 4.8 Go au format binaire CloudCompare. Pour des questions d'archivage pérenne, le nuage de point au format ASCII a été conservé : dans quelques années, il est certain que les futurs logiciels sauront encore ouvrir un fichier stocké sous ce format.