Thomas Blaise
email: thomas.blaise@universite-paris-saclay.fr

Sid Ahmed

Ali Khoudja

Cédric Carpentier

Benjamin Brigaud

Missenard

Xavier Mangenot

Philippe Boulvais

Philippe Landrein

Jean Cochard

Far-field brittle deformation record in the

Keywords: Calcite U-Pb geochronology, Jurassic limestones, Paris Basin, tension gash, far-field 16 deformation

Jurassic carbonate strata in the eastern Paris Basin exhibit several generations of 19 faults, tension gashes and stylolites. Although their relative chronology can sometimes 20 be determined according to crosscutting relationships, the duration of major deformation 21 phases and their influence on fluid flow and carbonate cementation are still uncertain.

22 This contribution aims to clarify the timing of brittle deformation and associated calcite 23 cementation. Tension gashes filled by calcite in Jurassic carbonates were sampled in 24 outcrops and boreholes and dated through U-Pb geochronology. Almost all the sampled 25 fractures were cemented during the Cenozoic period. Continuous deformation spreads

For oxygen and carbon stable isotope analysis, eleven calcite crystals from fracture in-267 fills were fragmented and collected with tweezers under the magnifying glass. The 268 crystals were subsequently grounded into a homogeneous powder in an agate mortar.

269

Oxygen and carbon isotopes were analysed in the Geosciences Laboratory of Rennes 270 University (France) following the methodology described in Malfilatre, 2012. All isotopic 271 values are reported to the V-PDB standard (Table 2).

272

Five samples consisting of large in-fillings were selected for clumped isotope analysis. Paleocene transition (Fig. 7). This age cannot be confidently interpreted and attributed 379 to a known deformation phase. It may either be linked to the so-called "sub-hercynian" 380 inversion phase defined by Ziegler (1987) or to an early stage of the Pyrenean 381 compression, as evidenced in the south-eastern Pyrenees (Cruset et al., 2020) (André et al., 2010;Bergerat et al., 2007, this study). Indeed, the ages of tension 387 gashes and hydraulic breccias documented here span from 58.9 ± 1.8 Ma to 18.7 ± 1.0 388 Ma, i.e., from the Thanetian to the Burdigalian periods (Fig. 7).

389

Our data show that tectonic quiescence then prevailed during the Paleocene and Early 390 Ypresian, a result in agreement with the slow rate of N-S convergence between the 391 Iberian and Eurasian plates during this period (Mouthereau et al., 2021;Grool et al., 392 2018;Macchiavelli et al., 2017). A single tension gash oriented N50° is dated at 58.9 ± 393 1.8 Ma (Paleocene) in the Sarazinière quarry. This fracture may be related to an early 394 stage of the Pyrenean compression. Alternatively, this event may be linked to the so-395 called "Laramide" phase (Ziegler, 1987), which corresponds to a major inversion period 396 with an uplift and erosion quantified to about 600 m in the Morvan Massif, south of the 397 Paris Basin, and to 300 m in the study area (Barbarand et al., 2013, Blaise et al., 2014).

398

This event does not seem to be restricted to the eastern border of the Paris Basin, since

425

A few fractures are dated between 40 and 35 Ma, while another main fracturing event is 426 identified at around 33 Ma (Fig. 7, Table 1). Ten of the 32 measured U-Pb ages et al., 2018;Sun et al., 2022). This is shown by the synchronous precipitation 558 of calcite in tension gashes and in the adjacent intergranular pore space. However, the 559 spatial extension of such "tectonic-induced cementation" is unknown and needs further

273

  30 mg of calcite was sampled using a micro-drilling tool. Δ 47 measurements were 274 performed at the California Institute of Technology (USA) in two analytical sessions 275 (April and May 2021) with an automated acid digestion and gas purification device 276 coupled to a dual inlet Thermo MAT253, as described in Passey et al., 2010. Samples 277 were weighed into silver capsules (~ 8 mg) and reacted in a common phosphoric acid 278 bath (~ 103 %) for 20 minutes at 90 °C under static vacuum. The resulting CO 2 was 279 passed through an ethanol/dry ice U-trap (~ -80 °C) before being collected on a liquid 280 nitrogen temperature (-196 °C) U-trap. Following the 20 minutes reaction period, the 281 collected CO 2 was thawed, entrained in helium, and carried through a Porapak Q 282 120/80 mesh gas column held at -20 °C using He as the carrier gas. The purified CO 2 283 was analyzed using a Thermo Scientific MAT 253 Mass Spectrometer set to collect 284 masses 44-49. Mass 48 was only monitored to detect any hydrocarbon contaminant.285δ 18 O and δ 13 C data was also acquired as part of each Δ 47 analysis and calculated using 286 the parameters reported relative to the PDB reference frame based on the calibrated 287 composition of the laboratory working gas and the correction scheme and constants 288 fromBrand et al., (2010). In order to account for the temperature dependence of oxygen CO 2 gas and carbonate resulting from the reaction with 290 phosphoric acid at 90 °C, a fractionation factor of 1.00811 was used for calcite 291(following Swart et al., 1991). The raw Δ 47 data was corrected for instrument non-292 linearity and scale compression (Dennis et al., 2011) using several heated (at 1000°) 293 and equilibrated gases (at 25°C) of various bulk isotopic compositions that were run 294 during each session. These gases were used to convert measurements into the inter-295 laboratory absolute reference frame (Dennis et al., 2011). To guarantee accuracy of the 296 Δ 47 data, we routinely analyzed two carbonate reference materials (Carrara marble and 297 TV04). One of these two carbonate standards was analyzed once for every five 298 analyses of the unknown samples in order to check for procedural analytical stability 299 and accuracy, and to determine the long-term external reproducibility of our 300 measurements. The Δ 47 values obtained for these carbonates over the course of this 301 study (May to July 2019) are: Δ 47-CDES25 = 0.409 ± 0.016‰ (1SD, n = 10) for Carrara; 302 Δ 47-CDES25 = 0.666 ± 0.011‰ (1SD, n = 8) for TV04, i.e., within accepted Δ 47 values for 303 TV04 (Δ 47-CDES25 = 0.655‰) and Carrara (Δ 47-CDES25 = 0.405‰). Finally, the corrected 304 Δ 47 values were converted into temperatures using the composite Δ 47 -T calibration of 305 Bonifacie et al. (2017), which has been shown to be appropriate for calcite and dolomite 306 between 0 and 300 °C, and which has been shown to be consistent with measurements 307 made at Caltech. The oxygen isotopic compositions of the water (δ 18 O water ) from which 308 the carbonates precipitated were calculated for each estimated TΔ 47 using the bulk 309 δ 18 O carb values and the calcite-water fractionation equation from O'Neil et al. (1969).

  same borehole was dated at 68.4 ± 4.0 Ma, i.e., the Cretaceous-378

  Pb age of 61.1 ± 2.5 was documented for calcite cements in the Middle Jurassic at 400 depth in the central ParisBasin (Mangenot et al., 2018).401A continuous deformation is then recorded in the age distribution of tension gashes from 402 the Late Ypresian to the Rupelian, although two age clusters can be evidenced between 403 40-50 Ma (9 samples) and 30-35 Ma (7 samples) (Fig.7). The first age cluster (40-50 404 Ma) is associated to N10° to N20° tension gashes (BAZX90a, MXV7, VCLD). The Late 405 Ypresian to Bartonian opening and filling of these gashes probably record the far-field 406 onset of the main Pyrenean compressional event at that time(Macchiavelli et al., 2017).

  phase, discrete tectonic pulses may have generated higher 408 rates of fracturing. A cluster of ages is recorded at ca. 48 Ma (samples A00607c, 409 SOR6ab, SOR6c, BAZX90a), synchronous to the emersion of the central Paris Basin 410(Briais et al., 2016) in response to the main Pyrenean compression period (e.g., Cruset 411 et al., 2020). Several tension gashes are then dated at ca. 43 Ma (Fig. 7), matching the 412 U-Pb ages reported by Pagel et al. (2018) along the Gondrecourt graben and the age of 413 a vertical tension gash in HTM102 borehole close to the Andra URL (Brigaud et al., 414 2020). 415 A reactivation of the variscan Vittel fault south of the studied area is evidenced by two 416 fractures sampled in the Sommerécourt quarry and dated at 42.5 ± 3.9 Ma and 40.9 ± 417 1.7 Ma (Fig. 7). At Removille, sample REM1 was collected along a fault delimiting a 418 small graben of similar orientation to the Gondrecourt and Joinville grabens. The age of 419 this sample, 42.1 ± 2.7 Ma confirms the synchronous formation of these structures. 420 Finally, sample OT1-2 from the Ottange quarry dated at 40.8 ± 2.2 Ma demonstrates 421 that this intraplate deformation extended far north of the study area (Fig. 1). They are 422 also recorded in the central Paris Basin, as documented by Mangenot et al. (2018).423 These deformations and cementations are all interpretated as a far-field consequence 424 of Pyrenean shortening.

  427 obtained are in the 35-30 Ma range, matching previous U-Pb ages around 33 Ma 428 reported for fractures along the Gondrecourt faults (Pagel et al., 2018) and vug-filling 429 calcites in Oxfordian limestones from the Andra URL borehole cores (Pisapia et al., 430 2018). Following Pagel et al. (2018) by analogy with the known tectonic agenda of the 431Rhine Graben, we suggest that these dated structures are related to extensional 432 deformation and the opening of the ECRIS(Dezès et al., 2004; Bergerat et al., 2007; -step evolution identical to the Gondrecourt graben, i.e., a strike-slip or 438 oblique displacement during the Lutetian, followed by extension during the Rupelian.439On the other hand, we cannot fully exclude a compressional origin for all of these ca. 33 440 Ma tension gashes as a consequence of further Pyrenean or Alpine far-field 441 deformations(Cruset et al., 2020; Parizot et al., 2021). Whatsoever, we show here that 442 deformation in the eastern Paris Basin occurred continuously from the Late Ypresian to 443 the Rupelian (Fig.7) and that the transition from a contractional regime prevailing during 444 the Eocene period to an extensional deformation during the Oligocene may have been 445 progressive. 446 After this 33 Ma ECRIS-related event, few younger gashes are dated around 30 Ma. 447 Tension gashes developed in the quarries of Dugny-sur-Meuse and Maxey-sur-Vaise 448 dated around 32-27 Ma are oriented N170° and N175° and may have opened during a 449 compressional phase documented by Bergerat (1987) and André et al. (2010) and 450 interpreted as a late Pyrenean horizontal stress. This age also replicates the ones 451 obtained by Parizot et al. (2021) in the north Pyrenean foreland basin, and those of were sampled in the vicinity of the NW-SE Poissons fault system 468 (Fig. 1) that reactivated in response to the compression during either the earliest stages 469 of the Alpine contractional stress or a Late Pyrenean event during 470 Aquitanian/Burdigalian (Parizot et al., 2021 ; Hoareau et al., 2021). 471 Finally, the absence of younger ages indicates that the growth of the nearby Jura fold-472 and-thrust belt formed roughly between 11 and 3 Ma (Homberg et al., 2002; Smeraglia 473 et al., 2021) had no or very limited impact on the deformation of the eastern Paris Basin. 474 To sum up, the deformation of the eastern Paris Basin is essentially related to far-field 475 propagation of compressional stress during the Pyrenean orogen growth. It has been 476 accommodated continuously by the opening of tension gashes and the development or 477 reactivation of strike-slip faults and stylolites between around 50 Ma (sample PPA1073) 478 and 27 Ma (sample MXV9a), with deformation pulses at ca. 48 Ma and 43 Ma, and only 479 interrupted shortly by the ECRIS-related extensional event at around 33 Ma. 480 On a larger scale, this tectonic agenda of the eastern Paris Basin is consistent with the 481 U-Pb ages of slickenfibres and fracture filling calcites in the north foreland basin of the 482 Pyrenees (Parizot et al., 2021), in the southeastern Pyrenean fold and thrust belt and 483 the south foreland Ebro basin (Cruset et al., 2020) and along the Cevennes fault system 484 (Parizot et al., 2022). South of the Paris Basin, fluorite mineralization occurred likely in 485 response of this contractional event (Lenoir et al., 2021). This event is also recorded in 486 the Jura fold-and-thrust belt, where a pre-orogenic NE-SW strike-slip fault was recently 487 evidenced through U-Pb geochronology of slickenfibre calcite yielding two ages similar 488 to ours at 48.4 ± 1.5 and 44.7 ± 2.6 Ma (Smeraglia et al., 2021).

  . Origin and Temperature of calcite-mineralizing fluids 490 The meteoric origin of waters from which calcite precipitated in tension gashes was 491 suggested by André et al. (2010) on the basis of the δ 18 O values measured in calcite.492Yet, the oxygen stable isotope composition of parent-waters was uncertain, since the 493 fractionation factor between H 2 O and calcite was estimated from only six two-phased 494 aqueous fluid inclusions that homogenized at temperatures ranging from 31 to 38 °C 495(Buschaert et al., 2004). Thanks to the Δ 47 clumped isotope composition of calcite, the

  evidence for a continuous fracturing of the Jurassic carbonates from 564 the Ypresian to the Rupelian in the eastern Paris Basin. This period coincides with the 565 main shortening related to the Pyrenean orogen growth, interrupted by extensional 566 deformation during Late Eocene -Early Oligocene. Three U-Pb age clusters are 567 identified, the two firsts at around 48 Ma and 43 Ma, related to the far-field propagation 568 of Pyrenean horizontal stress, and a third at 33 Ma corresponding to the ECRIS-related 569 extensional regime. Fluids of meteoric origin percolated through the fracture networks at 570 rather low temperatures (from 27 to 53 °C), with a progressive decrease of temperature 571 though time. Fluid flow and associated calcite precipitation was not restricted within the 572 fractures, but also occurred in the intergranular pore space. des failles et des macles de la calcite. Bulletin de la Société Géologique .D., HORSTWOOD, M.S.A. & COTTLE, P. 2007. 2016. Advances in Isotope 857 Ratio Determination by LA-ICP-MS. Elements 12, 317-322. 858 ZIEGLER, P.A. 1987. Late Cretaceous and Cenozoic intra-plate compressional 859 deformations in the Alpine foreland-a geodynamic model. Tectonophysics 137, 860 389-420. 861 ZIEGLER, P.A. & DÈZES, P. 2007. Cenozoic uplift of Variscan Massifs in the Alpine 862 foreland: Timing and controlling mechanisms. Global and Planetary Change 58,

Figure 1 .Figure 2 .Figure 3 .Figure 4 .Figure 5 .Figure 6 .Figure 7 .Figure 8 .Figure 9 .

 123456789 Figure 1. Geological map of the eastern Paris Basin (a), with a detailed view of the major structures in the studied area(b). Outcrop and quarries sample locations are indicated by black squares.

  259 does not include uncertainty propagations. A second age uncertainty is given, by 260 propagating the systematic uncertainty of the age of primary reference material WC-1 261 (2.6%), and the 2 standard errors of the 207 Pb/ 206 Pb and 206 Pb/ 238 U of the analytical 262 session by quadratic addition. Sample ages and uncertainties are listed in Table 1 and 263 detailed metadata for LA-ICP-MS calcite U-Pb geochronology is available in

	264	supplementary materials.
	265	3.d. Oxygen and carbon stable isotope composition and clumped isotope thermometry
	266
		F o r F o r F o r F o r F o r
		P e P e P e P e P e
		e r e r e r e r e r
		R e R e R e R e R e
		v v v v v
		i e i e i e i e i e
		w w w w w

Table 2 .

 2 Oxygen and carbon stable isotope composition and clumped isotope

	Page 35 of 151
	870	thermometry.

Ring & Gerdes, 2016), an event that is strongly expressed in the eastern Paris Basin

Acknowledgements

574

This manuscript benefited from fruitful discussion with Christian Hibsch. We also thank

Cambridge University Press

Geological Magazine

Frédéric Haurine for his assistance during the LA-ICP-MS U-Pb analysis. This work is 576 the outcome of collaborative project between the University Paris-Saclay and Andra, the 577 French agency for radioactive waste management, funded in part by Andra. We thank 578 the two anonymous reviewers for their constructive comments that helped to improve 579 the manuscript.

Declaration of interest 581

The authors declare none


Liana Ermakova
email: liana.ermakova@univ-brest.fr

Eric Sanjuan

Jaap Kamps

Stéphane Huet

Irina Ovchinnikova

Diana Nurbakova

Sílvia Araújo

Radia Hannachi

Elise Mathurin

Patrice Bellot

Overview of the CLEF 2022 SimpleText Lab: Automatic Simplification of Scientific Texts

Keywords: Scientific text simplification, (Multi-document) summarization, Contextualization, Background knowledge, Scientific information distortion simpletext-project.com

published or not. The documents may come

Introduction

Scientific literacy is an important ability for people. It is one of the keys for critical thinking, objective decision-making and judgment of the validity and significance of findings and arguments, which allows discerning facts from fiction. Thus, having a basic scientific knowledge may also help maintain one's health, both physiological and mental. The COVID-19 pandemic provides a good example of such a matter. Understanding the issue itself, choosing to use or avoid particular treatment or prevention procedures can become crucial. However, the recent pandemic has also shown that simplification can be modulated by political needs and the scientific information can be distorted [START_REF] Ermakova | Covid or not Covid? Topic Shift in Information Cascades on Twitter[END_REF]. Thus, the evaluation of the alteration of scientific information during the simplification process is crucial but underrepresented in the state-of-the-art.

Digitization and open access have made scientific literature available to every citizen. While this is an important first step, there are several remaining barriers preventing laypersons to access the objective scientific knowledge in the literature. In particular, scientific texts are often hard to understand as they require solid background knowledge and use tricky terminology. Although there were some recent efforts on text simplification (e.g. [START_REF] Maddela | Controllable Text Simplification with Explicit Paraphrasing[END_REF]), removing such understanding barriers between scientific texts and general public in an automatic manner is still an open challenge. The CLEF 2022 Sim-pleText track 1 brings together researchers and practitioners working on the generation of simplified summaries of scientific texts. It is a new evaluation lab that follows up the SimpleText-2021 Workshop [START_REF] Ermakova | Text Simplification for Scientific Information Access: CLEF 2021 Simple-Text Workshop[END_REF]. All perspectives on automatic science popularisation are welcome, including but not limited to: Natural Language Processing (NLP), Information Retrieval (IR), Linguistics, Scientific Journalism, etc.

SimpleText provides data and benchmarks for discussion of challenges of automatic text simplification by bringing in the following interconnected tasks:

Task 1: What is in (or out)? Select passages to include in a simplified summary, given a query. Task 2: What is unclear? Given a passage and a query, rank terms/concepts that are required to be explained for understanding this passage (definitions, context, applications,..). Task 3: Rewrite this! Given a query, simplify passages from scientific abstracts.

Automatic scientific text simplification is a very ambitious problem which cannot be addressed by a simple solution, but we have isolated three clear challenges that need to be addressed to improve non-expert access to scientific literature. In order to simplify scientific texts, one has to [START_REF]Text Analysis Conference (TAC) 2014 Biomedical Summarization Track[END_REF] select the information to be included in a simplified summary, (2) decide whether the selected information is sufficient and comprehensible or provide some background knowledge if not, (3) improve the readability of the text [START_REF] Ermakova | Overview of SimpleText 2021 -CLEF Workshop on Text Simplification for Scientific Information Access[END_REF]. Our lab is organised around this pipeline. Our test data was built accordingly as we asked to rank difficult terms (Task 2) and simplify sentences (Task 3) retrieved for Task 1 and we evaluated the results with regard to the queries from Task 1.

In the CLEF 2022 edition of SimpleText, a total of 62 teams registered for the SimpleText track. A total of 40 users downloaded data from the server. A total of 9 distinct teams submitted 24 runs, of which 10 runs were updated. The details of statistics on runs submitted for shared tasks are presented in Table 1.

This introduction is followed by Section 2 presenting a brief overview of related evaluation initiatives, related tasks and related approaches. The bulk of this paper presents the tasks with the datasets and evaluation metrics used, as well as the results of the participants, in three self-contained sections: Section 3 on the first task about content selection, Section 4 on the second task about complexity spotting, and Section 5 on the third task about text simplification proper. We end with Section 6 discussing the results and findings, and lessons for the future.

Related work

This section presents a brief overview of related evaluation initiatives, related tasks and related approaches.

In parallel with the CLEF SimpleText track, which was accepted in 2020, there have been a range of related initiatives on scholarly document processing at NLP conference. In 2020, Scholarly Document Processing2 provided the shared tasks on -CL-SciSumm: Scientific Document Summarization; -CL-LaySumm:Lay Summary; -LongSumm: Generating Long Summaries for Scientific Documents. CL-SciSumm and LongSumm are focused on summarization task but no adaptation to general public is previewed. The CL-SciSumm'20 LaySummary [START_REF] Chandrasekaran | Overview and insights from the shared tasks at scholarly document processing 2020: Cl-scisumm, laysumm and longsumm[END_REF] subtask asked to produce a scientific paper summary without technical jargon. However, terms are not often replaceable due to the risk of information distortion and these complex concepts should be explained to a reader.

In 2022 the Third Workshop on Scholarly Document Processing3 hosted the following shared tasks which are related to our track although they don't tackle the simplification aspect:

-MSLR22: Multi-Document Summarization for Literature Reviews; -DAGPap22: Detecting automatically generated scientific papers; -LongSumm 2022: Generating Long Summaries for Scientific Documents; -SV-Ident 2022: Survey Variable Identification in Social Science Publications; -Scholarly Knowledge Graph Generation; -Multi Perspective Scientific Document Summarization.

As it turns out, the SimpleText tasks and SDProc tasks are complementary, and together build a larger community to work on this important problem.

Popular science articles are generally much shorter than scientific publications. Thus, summarization is a step to text simplification as it reduces the amount of information to be processed. However, information selection is understudied task in document simplification [START_REF] Zhong | Discourse Level Factors for Sentence Deletion in Text Simplification[END_REF] as existing works mainly focus on word/phrase-level [START_REF] Maddela | A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification[END_REF] or sentence-level simplifications [START_REF] Dong | EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing[END_REF]. However, the lack of background knowledge can become a barrier to reading comprehension and there is a knowledge threshold allowing reading comprehension [START_REF] O'reilly | How Much Knowledge Is Too Little? When a Lack of Knowledge Becomes a Barrier to Comprehension[END_REF]. Scientific text simplification presupposes the facilitation of readers' understanding of complex content by establishing links to basic lexicon while traditional methods of text simplification try to eliminate complex concepts and constructions [START_REF] Maddela | A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification[END_REF]. SimpleText is not limited to a "Split and Rephrase" task [START_REF] Narayan | Split and Rephrase[END_REF] but also aims to provide a sufficient context to a scientific text. Entity linking could mitigate the background knowledge problem, by providing definitions, illustrations, examples, and related entities, but the existing entity linking datasets are focused on people, places, and organisation [START_REF] Hoffart | Robust disambiguation of named entities in text[END_REF], while a non-expert reader of a scientific article needs assistance with new concepts and methods. INEX/CLEF'11-14 Tweet Contextualization [START_REF] Bellot | INEX tweet contextualization task: Evaluation, results and lesson learned[END_REF] and CLEF'16-17 Cultural Microblog Contextualization [START_REF] Ermakova | CLEF 2017 Microblog Cultural Contextualization Lab Overview[END_REF] tracks aim to provide lacking background knowledge to a tweet. Besides completely different nature of tweets and popular science, this use case differs from the text simplification as this lack of background knowledge is due to the tweet length. In contrast to the Background Linking task at TREC'20 News Track [START_REF] Anand Deshmukh | IR-BERT: Leveraging BERT for Semantic Search in Background Linking for News Articles[END_REF], SimpleText focuses on (1) scientific text; (2) selection of notions to be explained; (3) helpfulness of the provided information rather than its relevance.

Large pre-trained AI models, like Jurassic-1 [START_REF] Lieber | Jurassic-1: Technical Details and Evaluation[END_REF], Google's T5 [START_REF] Xue | mT5: A massively multilingual pre-trained text-to-text transformer[END_REF], BERT or GPT-3 [START_REF] Brown | Language Models are Few-Shot Learners[END_REF], outperformed other state-of-the-art models on several NLP tasks, including automatic summarization and text simplification [START_REF] Zhao | Integrating Transformer and Paraphrase Rules for Sentence Simplification[END_REF], but their serious issues are (1) consistency and coherency (coreference errors) [START_REF] Wang | Consistency and Coherency Enhanced Story Generation[END_REF] and (2) limitation to short texts (< 2k tokens) [START_REF] Yang | Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching p[END_REF]. Simple Wikipedia based datasets could be useful to train AI models but (1) they are not scientific publications; (2) there is no direct correspondence between Wikipedia and Simple Wikipedia articles [START_REF] Ermakova | Covid or not Covid? Topic Shift in Information Cascades on Twitter[END_REF]. Another dataset was introduced at TAC 2014 Biomedical Summarization Track [START_REF]Text Analysis Conference (TAC) 2014 Biomedical Summarization Track[END_REF] with a goal to retrieve important aspects of a paper from the perspective of the community.

Automatic evaluation metrics have been designed to measure the results of text simplification: SARI [START_REF] Xu | Problems in Current Text Simplification Research: New Data Can Help[END_REF] targets lexical complexity, while SAMSA estimates sentence structural complexity [START_REF] Sulem | Simple and Effective Text Simplification Using Semantic and Neural Methods[END_REF]. Standard evaluation measures (e.g. BLEU, ROUGE) are difficult to apply as one should consider the end user as well as source document content. Since traditional readability indices can be misleading [START_REF] Wubben | Sentence Simplification by Monolingual Machine Translation[END_REF], researchers proposed various approaches based on expert judgement [START_REF] Clercq | Using the crowd for readability prediction[END_REF], readability level [START_REF] Gala | Towards a french lexicon with difficulty measures: Nlp helping to bridge the gap between traditional dictionaries and specialized lexicons[END_REF], relevance judgement [START_REF] Cohan | Revisiting Summarization Evaluation for Scientific Articles[END_REF], crowd-sourcing [START_REF] Alva-Manchego | Asset: A dataset for tuning and evaluation of sentence simplification models with multiple rewriting transformations[END_REF], eye-tracking [START_REF] Grabar | Study of readability of health documents with eyetracking approaches[END_REF].

In contrast to that, we evaluate simplification in terms of lexical and syntax complexity combining with error analysis. As we demonstrated previously, scientific infor-mation is often distorted accidentally due to misunderstanding of terminology, omission of essential details, insertion of erroneous background etc. [START_REF] Ermakova | Covid or not Covid? Topic Shift in Information Cascades on Twitter[END_REF]. Information distortion analysis is close to scientific claim verification [START_REF] Nakov | Automated Fact-Checking for Assisting Human Fact-Checkers[END_REF][START_REF] Wadden | Fact or Fiction: Verifying Scientific Claims[END_REF] but fact checking is limited to search for relevant evidence and decide whether it supports the claim. Another close work is [START_REF] Pradeep | Scientific Claim Verification with VERT5ERINI[END_REF], where the TF-IDF cosine similarity between documents is computed on (1) a collection of abstracts of scientific papers from the Citation Network Dataset V1 AMINER [START_REF] Tang | ArnetMiner: extraction and mining of academic social networks[END_REF] and (2) a set of articles from Huffington Post. However, this approach is not robust to lexical changes, which are crucial for text simplification. To the best of our knowledge, no other automatic nor semi-automatic method for information distortion analysis exists.

3 Task 1: What is in (or out)?

In this section, we discuss the first task about content selection (and avoiding complexity) from a corpus of scientific abstracts, addressing the task:

Select passages to include in a simplified summary, given a query.

The task aims at finding references in computer science that could be inserted as citations in original press articles of general audience for illustration, fact checking or actualization. For each of the selected references, more relevant sentences need to be extracted. These passages can be complex and require further simplification to be carried out in Tasks 2 and 3. Task 1 focuses on content retrieval.

Evaluation framework

Corpus. As in 2021, we use the Citation Network Dataset: DBLP+Citation, ACM Citation network (12th version) [START_REF] Tang | ArnetMiner: extraction and mining of academic social networks[END_REF] as source of scientific documents that can be used as reference passages [START_REF] Ermakova | Overview of SimpleText 2021 -CLEF Workshop on Text Simplification for Scientific Information Access[END_REF]. It contains:

-4, 894, 083 bibliographic references published before 2020; -4, 232, 520 abstracts in English; -3, 058, 315 authors with their affiliations; -45, 565, 790 ACM citations.

Textual content together with authorship can be extracted from this corpus. Although we manually preselected abstracts for topics, participants also have access to an Elastic Search index; this index is adequate to passage retrieval using BM25.

Additional datasets have been extracted to generate Latent Dirichlet Allocation models for query expansion or train Graph Neural Networks for citation recommendation as carried out in StellarGraph 4 for example. The shared datasets provide: document abstract content for LDA (Latent Dirichlet Allocation) or Word Embedding (WE); document authors for coauthoring analysis; citation relationship between documents for co-citation analysis; citations by author for author impact factor analysis. These extra datasets are intended to be used to select passages by authors who are experts on the topic (highly cited by the community).

G12

Patient data from GP surgeries sold to US companies G12.1 patient data

G13

Baffled by digital marketing? Find your way out of the maze G13.1 digital marketing G13.2 advertising Topics. Topics are a selection of 40 press articles: 20 from The Guardian,5 a major international newspaper for a general audience with a tech section, and 20 from Tech Xplore6 a website taking part in the Science X Network to provide a comprehensive coverage of engineering and technology advances. Each article was selected in the computer science field to be in accordance with the provided corpus. URLs to original articles, the title and textual content of each topic were provided to participants. Articles were enriched with queries manually extracted from their content to provide an indication of the essential technical concepts covered. We manually checked that each query allows participants to retrieve from the corpus at least 5 relevant passages that could be inserted as citations in the press article. The use of these queries were optional.

Output formats. Results had to be provided in a TREC style tabulated format (with a ".csv" extension). The following columns were required (including the first line): run id Run ID starting with team ID, followed by "task1" and run name manual Whether the run is manual {0,1} topic id Topic ID query id Query ID used to retrieve the document (if one of the queries provided for the topic was used; 0 otherwise) doc id ID of the retrieved document (to be extracted from the JSON output) passage Text of the selected passage (abstract) For each topic, the maximum number of distinct DBLP references ( id json field) was 100 and the total length of passages was not to exceed 1,000 tokens. Table 3 shows an example of Task 1 output.

Evaluation metrics. All passages retrieved from DBLP by participants are expected to have some overlap (lexical or semantic) with the article content. Passage relevance were evaluated through:

1. Lexical and semantic overlap of extracted passages with topic article content, 2. Manual assessment of a pool of passages. To build a pooled test collection, we first extracted all the article IDs ranked by the number of participants who used the article to select passages. From this extraction, we only kept articles chosen by at least two participants and gave a relevance score on a scale of 0 to 5: 0 for irrelevant articles; 1 for marginally relevant articles; 2 when the abstract is relevant with the query; 3 when the abstract and keywords are relevant with the query; 4 when the abstract and keywords are relevant with the query and the topic (title of the original article); 5 when the abstract and keywords are relevant with the query and the extended topic (content of the original article).

In order to speed up the judgment process, for this edition we only evaluated relevance at the article level, and not at the sentence level. The abstract was considered as relevant as soon it has a sentence useful to explain the title or the original article. Among documents returned by at least three runs we found out:

-14 Guardian topics with lightly relevant documents; -11 Guardian topics with highly relevant documents; -10 Tech topics with lightly relevant documents; and -9 Tech topics with highly relevant documents.

For the documents returned by two runs, we had a high number of 1 and 2 scores for the Guardian topics. As regards the Tech Xplore topics, which have more technical queries since they deal with more technical and specific areas, queries were less ambivalent and more in keeping with the content of DBLP corpus. This has resulted in usually higher relevant scores, with many articles retrieved by two participants having a score of 3. Globally, whether the query comes from the Guardian or Tech Xplore, human evaluators found abstracts, among the articles retrieved by the participants from DBLP, that really explain the article or have matters which should have been addressed in the original article. Passages were often issued from publications that are more related to cognitive or information sciences than to technical fields, which shows that the DBLP corpus has expanded beyond computer science.

Results

A total of 3 teams submitted 6 runs: 4 automatic runs extracted 100 documents or abstracts per subquery, the CYUT automatic run extracted 5 sentences per subquery, and the manual extracted passages for a selection of subqueries.

We consider here the reduced pool of documents returned by at least two runs; there are 72 topics with judgments, with a mean of 6.7 and a median of 4 judged documents per topic. Since we have participants that focused on a short list of documents, we only report results computed at a depth of 5 returned documents. Table 4 shows cumulative (0-5) scores obtained by each run (Score), the number of returned documents with a score ≥ 1 (#Docs), the number of queries with at least one returned document (#Queries) and the average scores per document and query. We also provide NDCG@5 as the metrics used for official ranking on this task. These values show that the automatic run made by CYUT and the manual run significantly outperform other automatic runs in terms of selecting the abstracts with a high relevance.

4 Task 2: What is unclear?

In this section, we discuss the second task about complexity spotting in an extracted sentence from a scientific abstract, addressing the task:

Given a passage and a query, rank terms/concepts that are required to be explained for understanding this passage (definitions, context, applications etc.).

The goal of this task is to decide which terms (up to 5) require explanation and contextualization to help a reader to understand a complex scientific text -for example, with regard to a query, terms that need to be contextualized (with a definition, example and/or use-case). For each passage, participants should provide a ranked list of difficult terms with corresponding scores on the scale 1-3 (3 to be the most difficult terms, while the meaning of terms scored 1 can be derived or guessed) and on the scale 1-5 (5 to be the most difficult terms). Passages (sentences) are considered to be independent, i.e. difficult term repetition was allowed.

Evaluation framework

Train dataset. For this task, data is two-fold: Medicine and Computer Science, as these two domains are the most popular on forums like ELI5 [START_REF] Ermakova | Automatic Simplification of Sci-entific Texts: SimpleText Lab at CLEF-2022[END_REF][START_REF] Ovchinnikova | What science-related topics need to be popularized? A comparative study[END_REF]. As in 2021, for Computer Science, we use scientific abstracts from the Citation Network Dataset: DBLP+Citation, ACM Citation network (12th version) 7 [START_REF] Ermakova | Overview of SimpleText 2021 -CLEF Workshop on Text Simplification for Scientific Information Access[END_REF]. A master student in Technical Writing and Translation manually annotated each sentence by extracting difficult terms and attributing difficulty scores on a scale of 1-3 (3 to be the most difficult terms, while the meaning of terms scored 1 can be derived or guessed) and on a scale of 1-5 (5 to be the most difficult terms).

In 2022, we introduced new data based on Google Scholar and PubMed articles on muscle hypertrophy and health annotated by a master student in Technical Writing and Translation, specializing in these domains. The selected abstracts included the objectives of the study, the results and sometimes the methodology. The abstracts including only the topic of the study were excluded because of the lack of information. To avoid the curse of knowledge, another master student in Technical Writing and Translation not familiar with the domain was solicited for complexity spotting.

We provided 453 annotated examples in total.

Test dataset. To construct the test data, we retrieved 116,763 sentences from the DBLP abstracts according to the queries from Task 1. We then manually evaluated 592 distinct sentences for 11 queries. For the query Digital assistant we took the first 1,000 sentences retrieved by ElasticSearch. We pool terms submitted by all participants for all these queries, representing a number of 4,167 distinct pairs sentence-term in total. We ensured that for each evaluated source sentence the pool contained the results of all participants. Statistics of the number of evaluated sentences per query for Task 2 are given in Table 5.

Input and output formats. The input for the train and the test data was provided in JSON and CSV formats with the following fields:

snt id a unique passage (sentence) identifier. source snt passage text. doc id a unique source document identifier. query id a query ID. query text difficult terms should be extracted from sentences with regard to this query.

Input example (JSON format):

{"snt_id":"G06.2_2548923997_3", "source_snt":"These communication systems render self-driving vehicles vulnerable to many types of malicious attacks, such as Sybil attacks, Denial of Service (DoS), black hole, grey hole and wormhole attacks.", "doc_id":2548923997, "query_id":"G06. Participants had to submit a list of terms to be contextualized in a JSON format or a tabulated file TSV (for manual runs) with the following fields: run id Run ID starting with (team id) (task id) (name). manual Whether the run is manual {0, 1}. snt id a unique passage (sentence) identifier from the input file. term Term or other phrase to be explained. term rank snt term difficulty rank within the given sentence. score 5 term difficulty score on the scale from 1 to 5 (5 to be the most difficult terms). score 3 term difficulty score on the scale from 1 to 3 (3 to be the most difficult terms).

Output example (JSON format):

{"run_id":"NP_task_2_run1", "manual":1, "snt_id":"G06.2_2548923997_3", "term":"black hole attack", "term_rank_snt":1, "score_5":5, "score_3":3}, → → {"run_id":"NP_task_2_run1", "manual":1, "snt_id":"G06.2_2548923997_3", "term":"grey hole attack", "term_rank_snt":2, "score_5":5, "score_3":3}, → → {"run_id":"NP_task_2_run1", "manual":1, "snt_id":"G06.2_2548923997_3", "term":"Sybil attack", "term_rank_snt":3, "score_5":5, "score_3":3}, → → {"run_id":"NP_task_2_run1", "manual":1, "snt_id":"G06.2_2548923997_3", "term":"wormhole attack", "term_rank_snt":4, "score_5":5,"score_3":3}, → → {"run_id":"NP_task_2_run1", "manual":1, "snt_id":"G06.2_2548923997_3", "term":"Denial of service attack", "term_rank_snt":5, "score_5":4, "score_3":3} Evaluation metrics. We evaluated terms according to:

correctness of term limits; term difficulty score on the scale 1-3; term difficulty score on the scale 1-5.

For both scales of term difficulty, we used a converted scale 1-7. This scale 1-7 was chosen following the psycho-linguistic research of the perception and evaluation of lexical meanings performed by Osgood and his colleagues [START_REF] Osgood | Semantic Differential Technique in the Comparative Study of Cultures1[END_REF], in contrast to the psychometric Likert scale (1-5, Strongly disagree/Disagree/Neither agree nor disagree/Agree/Strongly agree), commonly used in the research that employs questionnaires [START_REF] Likert | A technique for the measurement of attitudes[END_REF].

In the classical version of the semantic differential technique, the scale shows the variety of the human perception of semantic nuances from negative (-3) to positive (+3) polarity where 0 marks the "norm" [START_REF] Osgood | Semantic Differential Technique in the Comparative Study of Cultures1[END_REF]. The scale 1-7 matches the Osgood's scale and seems more suitable to evaluate concepts and features avoiding associations with negative / positive assessment. Since the 1970s, the scale has been employed in various studies as an evaluation tool for qualitative features.

Table 6 provides examples of the used term difficulty scale. We separate the examples of abbreviations from non-abbreviated phrases / words.

We added 0 for terms that should not be explained at all and we converted the original scale 1-7 as presented in Table 7.

Table 8 provides some examples of the annotation for Task 2. TERM refers to the terms retrieved by participants, Correct limits is a binary category showing whether the retrieved terms is well limited, Corrected is an eventual correction of retrieved term limits, Difficulty is a term difficulty score in scale 1-7.

Results

A total of 4 teams submitted runs, of which 2 runs were updated. The results are given in Tables 9 and10. In both tables, we present results for correctly attributed scores regardless the correctness of term limits (Score 3 and Score 5) and the number of correctly limited terms with correctly attributed scores (+ Limits). Table 9 provides the results on all sentences we evaluated. However, to have comparable results for partial runs we also report scores on a subset 167 common sentences in Table 10, although we were constrained to exclude the run lea t5 due to a very low number of evaluated sentences.

Task 3: Rewrite this!

In this section, we discuss the third task about text simplification proper, rewriting an extracted sentence from a scientific abstract, addressing the task:

Given a query, simplify passages from scientific abstracts.

The goal of this task is to provide a simplified version of text passages (sentences) with regard to a query. Participants were provided with queries and abstracts of scientific papers. The abstracts could be split into sentences. The simplified passages were evaluated manually in terms of the produced errors as follows.

Evaluation framework

Train dataset As for Task 2: What is unclear?, we provided a parallel corpus of simplified sentences from two domains: Medicine and Computer Science (see Section 4.1). As previously, we use scientific abstracts from the DBLP Citation Network Dataset for Computer Science and Google Scholar and PubMed articles on muscle hypertrophy and health Medicine [START_REF] Ermakova | Overview of SimpleText 2021 -CLEF Workshop on Text Simplification for Scientific Information Access[END_REF][START_REF] Ermakova | Automatic Simplification of Sci-entific Texts: SimpleText Lab at CLEF-2022[END_REF].

Text passages issued from abstracts on computer science were simplified by either a master student in Technical Writing and Translation or a pair of experts: (1) a computer scientist and (2) a professional translator, English native speaker but not specialist in computer science [START_REF] Ermakova | Automatic Simplification of Sci-entific Texts: SimpleText Lab at CLEF-2022[END_REF]. Each passage was discussed and rewritten multiple times until it became clear for non-computer scientists. Medicine articles were annotated by a master student in Technical Writing and Translation specializing in this domain. Sentences were shortened, excluding every detail that was irrelevant or unnecessary to the comprehension of the study, and rephrased, using simpler vocabulary. If necessary, concepts were explained.

We provided 648 parallel sentences in total.

Test dataset We used the same 116,763 sentences retrieved by the ElasticSearch engine from the DBLP dataset according to the queries as for Task 2 (see Section 4.1). We manually evaluated 2,276 pairs of sentences for 11 queries. For the query Digital assistant we took the first 1,000 sentences retrieved by ElasticSearch. We pool source sentences coupled with their simplified versions submitted by all participants for all these queries. We ensured that for each evaluated source sentence the pool contained the results of all participants. The detailed statistics of the number of evaluated sentences per query for Task 3 are given in Table 11.

Input and output format

The input train and and the test data were provided in JSON and CSV formats with the following fields:

snt id a unique passage (sentence) identifier. source snt passage text. doc id a unique source document identifier. query id a query ID. query text simplification should be done with regard to this query.

Input example (JSON format):

{"snt_id":"G11.1_2892036907_2", "source_snt":"With the ever increasing number of unmanned aerial vehicles getting involved in activities in the civilian and commercial domain, there is an increased need for autonomy in these systems too.", "doc_id":2892036907, "query_id":"G11.1", "query_text":"drones"}

→ → → → →
Participants were asked to provide a list of terms to be contextualized in a JSON format or a tabulated file TSV (for manual runs) with the following fields: run id Run ID starting with (team id) (task 3) (name). manual Whether the run is manual {0, 1}. snt id a unique passage (sentence) identifier from the input file. simplified snt Text of the simplified passage .

Output example (JSON format):

{"run_id":"BTU_task_3_run1", "manual":1, "snt_id":"G11.1_2892036907_2", "simplified_snt":"Drones are increasingly used in the civilian and commercial domain and need to be autonomous."} → → → Evaluation metrics. We filtered out the simplified sentences identical to the source ones and the truncated simplified sentences by keeping only passages matching the regular expression (valid snippets) .+[?.!" ]* $'. Professional linguists manually annotated simplifications provided with regard to a query according to the following criteria. We evaluated binary errors:

-Incorrect syntax; -Unresolved anaphora due to simplification; -Unnecessary repetition/iteration (lexical overlap); -Spelling, typographic or punctuation errors.

The lexical and syntax complexity of the produced simplifications were assessed on an absolute scale, value 1 referring to a simple output sentence regardless of the complexity of the source one, 7 corresponding to a complex one. Lexical complexity is mostly identical to that presented in Section 4.1.

We consider syntax complexity based on syntactic dependencies, their length and depth. The dependency trees reveal latent complications for reading and understanding text; thus, psycholinguists consider the syntactic dependencies to be a relevant tool to evaluate text readability [START_REF] Futrell | The Natural Stories corpus: a reading-time corpus of English texts containing rare syntactic constructions[END_REF]. The depth and length of the syntactic chains we interpret according to [START_REF] Futrell | The Natural Stories corpus: a reading-time corpus of English texts containing rare syntactic constructions[END_REF].

We evaluate syntax complexity as follows:

1. Simple sentence (without negation / passive voice): Over Facebook, we find many interactions.

2. Simple sentence with negation / passive voice (e.g. Many interactions were found over Facebook) or Simple sentences with syntactic constructions that show chains of dependency and shallow embedding depth (e.g. Over Facebook, we find many interactions between public pages and both political wings.) 3. Simple sentences with long chains of dependency and shallow embedding depth, with syntactic constructions like complex object, gerund construction, etc. (e.g. Despite the enthusiastic rhetoric about the so-called collective intelligence, conspiracy theories have emerged.) or Short complex or compound sentence (e.g. We propose a novel approach that was used in terms of information theory.) 4. Simple sentences with long chains of dependency and deep embedding depth, with syntactic constructions like complex object, gerund construction, etc. (e.g. Over Facebook, we find many interactions between public pages for military and veterans, and both sides of the political spectrum) or Complex or compound sentence that contains long chains of dependency and deep embedding depth; 5. Simple sentences with long chains of dependency and deep embedding depth, with several syntactic constructions like complex object, gerund construction, etc. or & Complex or compound sentence that contains long chains of dependency and deep embedding depth; 6. Complex or compound sentences that contain long chains of dependency and deep embedding depth along with complex object, gerund construction, etc. or Simple sentence that contains modifications, topicalization, parenthetical constructions: Moreover, we measure the effect of 4709 evidently false information (satirical version of conspiracist stories) and 4502 debunking memes (information aiming at contrasting unsubstantiated rumors) on polarized users of conspiracy claims. 7. Long complex or compound sentences that contain several clauses of different types, long chains of dependency and deep embedding depth along with complex object, gerund construction, etc.

We evaluate the information quality of the simplified snippet based on its content and readability. Transformation of information from the source snippet brings in omission of details, insertion of basic terms to explain particular terminology and complex concepts, reference to resources. Due to necessary insertions and references, the simplified snippets often contain more words and syntactic constructions as compared to their source. Nevertheless, the goal is to reduce lexical and syntax complexity in the extended simplified snippets. In case the simplified snippet lacks information mentioned in the source, we evaluate the degree of the information loss. Irrelevant insertions, iterations and wordy statements in the extended simplified snippet we consider as a misrepresentation or distortion of source information when a reader experiences difficulties in processing source content due to wordiness of the loosely structured simplified snippet.

We assessed the information loss severity during the simplification with regard to a given query on the scale from 1 to 7, where 1 corresponds to unimportant information loss while 7 refers to the most severe information distortion. We consider the information loss as a kind of information damage even if the information in the simplified text contains the information of the source passage but has some insertions, which impedes perception of the content.

We distinguish the following 11 types of misrepresentation of source information. Our classification leans on the error typology in machine translation [START_REF] Lommel | Harmonised Metric[END_REF][START_REF] Ovchinnikova | Impact of new technologies on the types of translation errors[END_REF]. The sim-plified snippet often combines several types of distortion, e.g. omission and ambiguity. Nevertheless, we observed many instances of small distortions that severely diminish the quality of the simplification; therefore, we need to explain each type providing the clear and transparent instances. Our evaluation of the value of the information distortion leans on the calculation of the information loss and assessment of the diminished readability of the simplified snippet that generates difficulties in text semantic processing by readers.

We distinguish the following types of information distortion:

1. Style (distortion severity 1) Source snippet: In order to facilitate knowledge transfer between specialists and generalists and between experts and novices, and to promote interdisciplinary communication, there is a need to provide methods and tools for doing so.

Simplified snippet: There is a need to provide methods and tools for doing so.

In order to facilitate knowledge transfer between specialists and generalists and between experts and novices, we need to promote interdisciplinary communication.

We need to make it easier for people to share their knowledge with each other.

Comment: Deviations from the style norms do not lead to information loss; however, they diminish the quality of text structure and affect readers' assessment of the text and its content. 2. Insertion of unnecessary details with regard to a query(distortion severity 1)

Source snippet: In the clinical setting, availability of needed information can be crucial during the decision-making process.

Simplified snippet: availability of needed information can be crucial during the decision-making process. In the clinical setting, needed information is often difficult to come by. For confidential support call the Samaritans on 08457 90 90 90 or visit a local Samaritans branch, see www.samaritans.org for details Comment: The simplified snippet often contains more information than the source since the terminology is needed to be explained. An irrelevant insertion does not lead to the loss of information; however, it may bring in diminishing of the text readability and generate discomfort during text perception. The irrelevant reference to the support in the simplified snippet does not clarify the source. The source does not need any simplification.

Redundancy (without lexical overlap) (distortion severity 2)

Source snippet: The capability to get updated information and news is an important and decisive factor in business and finance.

Simplified snippet: The capability to get updated information and news is an important and decisive factor in business and finance. The ability to get updates on the latest news is also an important factor in the success of a business or finance company. For more information, visit CNN.com/News. Comment: Irrelevant insertions, iterations and wordy statements in the extended simplified snippet we consider as a misrepresentation or distortion of source information when a reader may misunderstand source content due to wordiness of the loosely structured simplified snippet. 4. Insertion of false or unsupported information (distortion severity 3)

Source snippet: The proposed method leads to not only faster running time but also efficient text localization.

Simplified snippet: The proposed method leads to not only faster running time but also efficient text localization. The proposed method is based on the idea that text should be written in a single block of text, rather than a series of blocks of text. The method is being developed by the University of California, San Diego Comment: False and unsupported information is inserted in the simplified snippet because of external links of the source snippet to the open Web resources. False information confuses the readership, thus it is inappropriate in simplified texts. 5. Omission of essential details with regard to a query (distortion severity 4)

Source snippet: In short, this thesis aims to repatriate young people's web practices from the sterile, positivist methods space of questionnaires and tests of digital literacy to social contexts of everyday life.

Simplified snippet: In short, this thesis aims to repatriate young people's web practices from the sterile, and tests of digital literacy.

Comment: Simplified texts regularly omit details and minor concepts since the objective of the simplification is to facilitate perception and processing of the main idea and novel information of the source. Therefore, distinguishing the essential details and concepts from the minor ones plays a crucial role in the text simplification. Omission of the essential details leads to the lack of information and novelty of the message. The simplified snippet lacks the essential information about the direction where the young people's web practice needs to be repatriated (to social contexts of everyday life). 6. Overgeneralization (distortion severity 5)

Source snippet: Online Social Networks explode with activity whenever a crisis event takes place. Simplified snippet: It explodes when a crisis event takes place.

Comment: In the simplified snippet the subject of the source snippet Online Social Networks is omitted being substituted by the pronoun It; also the essential detail with activity is omitted that brings in overgeneralization: the statement refers to all cases of the dysfunctions (server, browser, users' activity, moderators' restrictions, etc.) 7. Oversimplification (distortion severity 5)

Source snippet: If we accept the current orthodoxy and then blame the Web we offer a technological determinist explanation of reality: technology produces misinformed populations.

Simplified snippet: If we accept the current orthodoxy and then blame the Web we have an explanation of reality.

Comment: Oversimplification appears in the shortened simplified snippets when source utterance is transformed into a trivial statement or even a claim. The simplified snippet claims that we have an explanation of reality thanks to acceptance of the current orthodoxy and accusation of the Web. Meanwhile, the source snippet discusses the technological determinist explanation of reality. The omission of the essential details leads to the oversimplified statement that cannot explain the reality of the technological epoch. 8. Topic shift (distortion severity 5)

Source snippet: global warming induced by chemtrails or the link between vaccines and autism -find on the Web a natural medium for their dissemination.

Simplified snippet: The link between vaccines and autism -can be found on the Web a natural medium for changing.

Comment: Topic shift is revealed in a substitution of the source topic by omitting its part or selecting a wrong basic word to replace the peculiar term in the source. The source snippet lost the essential part of its topic (global warming induced by chemtrails) during the simplification process; moreover, the simplification resulted in the inappropriate syntactic structure of the snippet. 9. Contra sense / contradiction (distortion severity 6)

Source snippet: In this paper we discuss architectural design issues and trade-offs in connection with our experiences porting our agent-based platform, Opal, to the Sharp Zaurus personal digital assistant (PDA).

Simplified snippet: The Sharp Zaurus is a personal digital assistant (PDA) developed by Sharp. It is based on the Opal agent-based platform. We discuss architectural design issues and trade-offs in connection with our experiences porting Opal to the Zaurus PDA.

Comment: Contradictions in simplified snippets appear due to elimination of essential concepts or interrelations among concepts, omission of significant details, and transformation of the semantic structure of the source snippet. The simplified snippet mentions agent-based platform Opal as the basis for the Sharp Zaurus, but at the same time claims that Opal was ported to the Sharp Zaurus. The source snippet But the new phenomena, the non-agenda ownership, overcome any ideological influence, especially under the conditions of punishment mechanism applied to old politicians lost its semantic structure since the concepts ideological influence and punishment mechanism were eliminated in the process of its simplification. Thus, the simplified snippet But the new phenomena, the ownership of the non-agenda, had a lot of influence on old politicians lacks any explanation how the non-agenda ownership is related to old politicians and why they are influence by the new phenomena. 10. Ambiguity (distortion severity 6)

Source snippet: The experimental results show that 3D maps with texture on mobile phone display size, and 3D maps without texture on PDA display size are superior to 2D maps in search time and error rate. Simplified snippet: 3D maps with texture on mobile phone display size are superior to 2D maps in search time and error rate. The experimental results show that 3D maps without texture on PDA display size were superior to those with texture. The results were published in the journal 3D Maps.

Comment: Ambiguity presupposes that a statement has several equiprobable interpretations. The instance of the ambiguous simplified snippet in Table X lacks a key to understand whether the 3D maps without texture outperform those with texture or not. Ambiguity often appears due to syntactic simplification of the source. In the source, the clause changes in the strength of competition also reveal key asymmetrical differences is replaced by shorter clause but they do not have any biases that produces ambiguity: whether evidence corresponds to reality or not. The source clarifies the differences between two political parties: Though both Republicans and Democrats show evidence of implicit biases, changes in the strength of competition also reveal key asymmetrical differences however, the simplified snippet doubts the reliability of the evidence: Both Republicans and Democrats show evidence of biases, but they do not have any biases. Readers of the simplified snippet are unable to resolve the ambiguity. 11. Nonsense (distortion severity 7)

Source snippet: The large availability of user provided contents on online social media facilitates people aggregation around shared beliefs, interests, worldviews and narratives Simplified snippet: The large amount of user provided contents on online social media is called aggregation Comment: The source snippet was transformed into a simple sentence. The transformation brings in erroneous usage of the word aggregation that leads to the loss of meaning of the whole sentence. Instead of the original statement about accessibility of the social or public media on the Web, which facilitates dissemination of fake news and rumors, the simplified snippet claims that there is an opportunity to find a resource to read about fake news and rumors.

The final ranking for Task 3 was done by the average harmonic mean of normalized opposite values of Lexical Complexity (LC), Syntactic Complexity (SC) and Distortion Level (DL) as follows:

s i = 3 7 7-LC + 7 7-SC + 7 7-DL (1)
Score = ∑ i s i , if No Error 0, otherwise n (2)
In Equation 2, variable n refers to the total number of judged snippets and No Error means that the snippet i does not have any of Uncorrect syntax, Unresolved anaphora, nor Unnecessary repetition/iteration error.

Results

A total of 5 different teams submitted 14 runs (5 runs were updated). Absolute number of errors and average Lexical Complexity, Syntax Complexity and Information Loss are provided in Tables 12 and13. The final ranking for Task 3 is given in Table 14. We removed all runs with the 0 score. Very interesting partial runs were provided by the HULAT-UC3M team as the generated simplifications provided the explanations of difficult terms. However, HULAT-UC3M's 8 runs over 10 were not in the pool with selected topics. Thus, we provided only automatic evaluation results. The HULAT-UC3M's runs provide clear evidence of the interconnection of tasks 2 and 3.

Conclusion and future work

We introduced the CLEF 2022 SimpleText track, containing three interconnected shared tasks on scientific text simplification. We pipelined the passages retrieved for Task 1 in order to rank difficult terms (Task 2) and simplify sentences (Task 3). We evaluated term difficulty and simplifications with regard to the queries from Task 1.

For Task 1, we created a large corpus of scientific abstracts, a set of popular science requests with detailed relevance judgments on the level of relevance of scientific abstracts to the request and broader context of a newspaper article on this topic. The abstracts of scientific papers retrieved for these requests were used in the follow up tasks. For Task 2 and 3, we created a corpus of sentences extracted from the abstracts of scientific publications, with manual annotations of term complexity (Task 2). In contrast to previous work, we evaluate simplification in terms of lexical and syntax complexity combining with error analysis. We introduced a new classification of information distortion types for automatic simplification and we annotated the collected simplifications according to this error classification (Task 3). Recent pandemics have shown that simplification can be modulated by political needs and the scientific information can be distorted. Thus, in contrast to previous work, we evaluated the simplifications in terms of information distortion.

For next year, we plan continue the Task 1 setup, but also refine the relevance judgments to sentence level, and provide additional evaluation measures of readability levels. We will extend Task 2 to provide a context to difficult terms and we will work on automatic metrics based on the insights we obtained this year. In particular, for Task 2, participants will be asked to provide context for difficult terms. This context should provide a definition and take into account ordinary readers' needs to associate their particular problems with the opportunities that science provides them to solve the problems [START_REF] Ovchinnikova | What science-related topics need to be popularized? A comparative study[END_REF]. This year, the HULAT-UC3M team submitted runs which combine tasks 2 and 3 which demonstrates strong interconnection of the tasks as often the terminology cannot be removed nor simplified but it needs to be explained to a reader. Finally, we plan to continue the Task 3 setup, continuing the detailed manual annotations of samples, but also working on automatic metrics that best reflect the insights of this year's analysis.

For details about this year's track and the approaches of individual teams we refer to the CLEF CEUR proceedings [START_REF]Proc. of the Working Notes of CLEF 2022: Conference and Labs of the Evaluation Forum[END_REF]. Further details about the lab can be found at the SimpleText website: http://simpletext-project.com. Please join us and help to make scientific results understandable! impact in the train data construction and Léa Talec-Bernard and Julien Boccou for their help in evaluation of participants' runs. We thank Josiane Mothe for reviewing papers. We also thank Alain Kerhervé, and the MaDICS (https:// www.madics.fr/ ateliers/ simpletext/ research group.

Table 1 .

 1 CLEF 2022 SimpleText official run submission statistic

	Team	Task 1	Task 2	Task 3	Total runs
	UAms	2	1		3
	NLP@IISERB	3 (3 updated)			3
	SimpleScientificText		1 (1 updated)		1
	aaac		1 (1 updated)		1
	LEA T5		1	1	2
	PortLinguE			1 (1 updated)	1
	CYUT Team2	1		1	2
	HULAT-UC3M			10 (4 updated)	10
	CLARA-HD			1	1
	Total runs	6	4	14	24

Table 2 .

 2 SimpleText Task 1: Examples of topics and queries

	Topic ID Query ID Title or Query

Table 3 .

 3 SimpleText Task 1: Examples of output

	Run	M/A Topic Query	Doc	Passage
	ST1 task1 1 0 G01 G01.1 1564531496 A CDA is a mobile user device, similar to a Per-
				sonal Digital Assistant (PDA). It supports the cit-
				izen when dealing with public authorities and
				proves his rights -if desired, even without reveal-
				ing his identity.
	ST1 task1 1 0 G01 G01.1 3000234933 People are becoming increasingly comfortable
				using Digital Assistants (DAs) to interact with
				services or connected objects
	ST1 task1 1 0 G01 G01.2 1448624402 As extensive experimental research has shown in-
				dividuals suffer from diverse biases in decision-
				making.

Table 4 .

 4 SimpleText Task 1: Evaluation scores of official runs

	Team	Score	#Docs	Doc Avg	#Queries	Query Avg	NDCG
	CYUT	125	44	0.53	77	1.62	0.3322
	UAMS-MF ⋆	163	54	0.87	99	1.65	0.2761
	UAMS	52	17	0.22	40	1.30	0.1048
	NLP@IISERB	26	7	0.35	13	2.00	0.0290

⋆ Manual run.

Table 5 .

 5 SimpleText Task 2: Statistics of the number of evaluated sentences per query

	2",

Table 6 .

 6 Examples of the term difficulty scale used for evaluation. Difficult terms are highlighted with the green color

	Grade	Non-abbreviated (ordinary) term	Abbreviation
	7 external qubit in "The qubit-qutrit pair acts as a	XCSFHP in "We compared
	closed system and one external qubit serve as the en-	XCSFHP to XCSF on several
	vironment for the pair."	problems."
	6 "This paper bring forward based on immune genetic	"XCS with computed prediction,
	algorithm to solve man on board automated storage	namely XCSF, extends XCS by
	and retrieval system optimized problem, immune ge-	replacing the classifier prediction
	netic algorithm remains the characteristic which is	with a parametrized prediction
	not ..."		function."
	"Tile coding is a well-known function approximator	"Side-channel attack (SCA) is a
	that has been successfully applied to many reinforce-	very efficient cryptanalysis tech-
	ment learning tasks."	nology to attack cryptographic
			devices."
	5 "Experiment simulation result express: the result of	"This paper presents a simple real-
	immune genetic algorithm is better than traditional	coded estimation of distribution al-
	genetic algorithm in the circumstance of the same	gorithm (EDA) design using x-
	clusters and the same evolution generation."	ary extended compact genetic algo-
			rithm (XECGA) and discretization
			methods."
	4 "Immune genetic algorithm can shorten storage or re-	"This paper presents a simple real-
	trieval distance in application, and enhance storage or	coded estimation of distribution al-
	retrieval efficiency."	gorithm (EDA) design using x-
	"Deep learning has become increasingly popular in	ary extended compact genetic algo-
	both academic and industrial areas in the past years."	rithm (XECGA) and discretization
			methods."
	3 "The XECGA is then used to build the probabilistic	"We evaluate each measure's per-
	model and to sample a new population based on the	formance by AUC which is usually
	probabilistic model."	used for evaluation of imbalanced
			data classification."
	2 "Experiment simulation result express: the result of	NIST (The National Institute of
	immune genetic algorithm is better than traditional	Standards and Technology) in "Re-
	genetic algorithm in the circumstance of the same	cently NIST has published the sec-
	clusters and the same evolution generation."	ond draft document of recommen-
	"Specifically, the real-valued decision variables are	dation for the entropy sources used
	mapped to discrete symbols of user-specified cardi-	for random bit generation."
	nality using discretization methods."
	1 "video labeling game is a crowsourcing tool to collect	2D (2-dimensional), 3D (3-
	user-generated metadata for video clips."	dimensional) maps as in "The 3D
	"On the other hand, a 3dimensional (3D) map, which	maps will give more intuitive infor-
	is one of major themes in machine vision research,	mation compared to conventional
	has been utilized as a simulation tool in city and land-	2-dimensional (2D) ones."
	scape planning, and other engineering fields."
	0 "This device has two work modes: "native" and "re-	et al. (from latin "et alii" meaning
	mote"."		"and others") in "However, Nam et
	"The proposed rECGA is simple, making it amenable	al. pointed out. . . "
	for further empirical and theoretical analysis."

→ →

Table 7 .

 7 SimpleText Task 2: Scale conversion rules

	Term difficulty scale	0	1	2	3	4	5	6	7
	7 point scale	0	1	2	3	4	5	6	7
	⇒ 5 point scale	0	1	2		3	4		5
	7 point scale	0	1	2	3	4	5	6	7
	⇒ 3 point scale	0	1			2		3

Table 8 .

 8 SimpleText Task 2: Examples of the annotation This device has two work modes: 'native' and 'remote'. work modes YES 0 This device has two work modes: 'native' and 'remote'. modes native NO work modes 0 This device has two work modes: 'native' and 'remote'. device work NO device 0 This device has two work modes: 'native' and 'remote'. native remote NO native 1

	Sentence	Term	Limits	Diffi-
			OK Corrected culty
	This device has two work modes: 'native' and 'remote'.	remote	YES	1

Table 9 .

 9 SimpleText Task 2: Results for the official runs

		Total	Evaluated	Score 3		Score 5
				+Limits	+Limits	+Limits
	aaac	581,285	2,951	1,388	702	318	415	175
	SimpleScientificText 63,027	298	262	48	44	47	42
	UAms	263,022	1,315	1,175	105	69	60	49
	lea t5	23,331	5	4	0	0	0	0

Table 10 .

 10 SimpleText Task 2: Results on a subset of 167 common sentences

		Total	Evaluated		Score 3		Score 5
			+Limits	+Limits	+Limits
	aaac	581,285	833	414	200	104	127	67
	UAms	263,022	574	514	46	28	25	21
	SimpleScientificText 63,027	208	188	33	32	32	29

Table 11 .

 11 SimpleText Task 3: Statistics of the number of evaluated sentences per query

	Query	# Distinct source sentences	# Distinct simplified sentences
	1 digital assistant	370	1,280
	2 conspiracy theories	195	398
	3 end to end encryption	55	102
	4 imbalanced data	55	87
	5 genetic algorithm	51	85
	6 quantum computing	51	85
	7 qbit	50	76
	8 quantum applications	42	73
	9 cyber-security	28	47
	10 fairness	18	22
	11 crowsourcing	14	21

Table 12 .

 12 SimpleText Task 3: General results of official runs

	Run	Total	Unchanged	Truncated	Valid	Longer	Length Ratio	Evaluated	Uncorrect Syntax	Unresolved Anaphora	Minors	Syntax Complexity	Lexical Complexity	Information Loss
	CLARA-HD	116,763	128 2,292 111,627 201 0.61 851 28 3 68 2.10 2.42 3.84
	CYUT Team2	116,763	549 101,104 111,818	49 0.81 126 1		32 2.25 2.30 2.26
	PortLinguE full 116,763 42,189	852 111,589 3,217 0.92 564 7		5 2.94 3.06 1.50
	PortLinguE run1	1,000	359	7	970	30 0.93 80 1			3.63 3.57 2.27
	lea task3 t5	23,360	52 23,201 22,062	24 0.35	. . . .	.	.	.
	HULAT-UC3M01 1,000	.	13	973 968 2.46 95 10 1 20 4.69 3.69 2.20
	HULAT-UC3M02 2,001	3	58 1,960 1,920 2.53 205 10 1 37 3.60 3.53 2.34
	HULAT-UC3M03 1,000	2	13	958 966 2.53	. . . .	.	.	.
	HULAT-UC3M04 2,000	.	33 1,827 1,957 37	. . . .	.	.	.
	HULAT-UC3M05 2,000	.	56 1,921 1,918 2.38	. . . .	.	.	.
	HULAT-UC3M06 2,000	.	47 1,976 1,921 2.45	. . . .	.	.	.
	HULAT-UC3M07 1,000	.	56	970 972 2.43	. . . .	.	.	.
	HULAT-UC3M08 2,000	.	62 1,964 1,919 2.59	. . . .	.	.	.
	HULAT-UC3M09 2,000	.	170 1,964 1,904 2.15	. . . .	.	.	.
	HULAT-UC3M10 2,000	.	215 1,963 1,910 2.13	. . . .	.	.	.

Table 13 .

 13 SimpleText Task 3: Information distortion in evaluated runs

	Run	Evaluated	Non-Sense	Contresens	Topic Shift	Wrong Synonym	Ambiguity	Omission Of Essential Details	Overgeneralization	Oversimplification	Unsupported Information	Unnecessary Details	Redundancy	Style
	CLARA-HD	851 162 68 37 20 80 314 59 203 26 10 29 13
	CYUT Team2	126	2	1	.	.	4	42	4	5	.	.	.	4
	PortLinguE full	564	9	3	4	3 19	94	9	13	2	2	5	1
	PortLinguE run1	80	.	.	1	.	.	27	5	2	.	.	.	.
	lea task3 t5	.	.	.	.	.	.	.	.	.	.	.	.	.
	HULAT-UC3M01	95	1	7	2	.	5	2	.	1	5 38 36	.
	HULAT-UC3M02 205	4	9	4	.	9	4	.	. 12 72 61	1

Table 14 .

 14 SimpleText Task 3: Ranking of official submissions on combined score

	Run	Score
	PortLinguE full 0.149
	CYUT Team2 0.122
	CLARA-HD	0.119

https://ornlcda.github.io/SDProc/sharedtasks.html

https://sdproc.org/2022/sharedtasks.html

https://stellargraph.readthedocs.io/

https://www.theguardian.com/science

https://techxplore.com/

Acknowledgments

We like to acknowledge the support of the Lab Chairs of CLEF 2022, Allan Hanbury and Martin Potthast, for their help and patience. Special thanks to the University Translation Office of the Université de Bretagne Occidentale, and to Nicolas Poinsu and Ludivine Grégoire for their major