La nouvelle ministre de l'Education nationale, Nicole Belloubet, dispose de trois qualités pour mettre en oeuvre une politique au service des élèves et de la Nation. D'une part, elle a été rectrice, expérience précieuse pour diriger ce ministère. D'autre part, elle ne manque pas de courage politique. En 2005, elle a démissionné de son poste pour indiquer son désaccord avec la politique de F. Fillon. Enfin, elle est universitaire. Sa formation pourrait l'amener à fonder sa politique sur les multiples connaissances scientifiques relatives aux questions éducatives. Elle n'a pas hésité d'ailleurs à y faire référence dans un article où elle a exposé ses conceptions de l'école française (Après-Demain, 2016). Susceptible d'orienter l'action, la connaissance scientifique est concurrencée par deux autres variables. La première est la démagogie au fondement de cette tentation, partagée par tous les politiques, de subordonner leurs décisions aux idées de leur électorat. La seconde tient à la force des idéologies, systèmes de croyances qui orientent les modes de pensée et d'action. Pour l'école, le poids des idéologies influence chacun sur toutes les questions essentielles : les pratiques pédagogiques, les programmes scolaires, la place de l'État, la mixité sociale… Nicole Belloubet, dont la feuille de route est de poursuivre les multiples chantiers ouverts par G. Attal et E. Macron, va devoir arbitrer entre ces trois déterminants de la politique éducative. Deux des chantiers d'Attal font l'objet d'une analyse.

Le port de l'uniforme : le souhait fantasmé de l'uniformité

Le choix de Gabriel Attal de favoriser, voire généraliser, le port de l'uniforme, a pour objectif de créer de l'égalité entre les élèves qui ne pourraient plus se distinguer par le choix de leurs vêtements. Un tel projet méconnaît la réalité de l'école. L'inégalité entre les élèves tient à un ensemble de qualificatifs relatifs à leur statut scolaire (« intellos » vs « nuls »), leur identité sexuelle (« la pédale »), ou leur apparence physique, notamment leur taille (« le petit » vs « l'asperge »), leur poids (« la grosse »), la couleur de leur peau (« le black », « le beur »…), etc. Dans le quotidien des écoles, tous ces qualificatifs constituent des stigmatisations envahissantes (Galland, 2006 ;Merle, 2012). Le port de l'uniforme ne changera rien à ces harcèlements aux conséquences parfois dramatiques.

Un autre argument présenté en faveur de l'expérimentation de l'uniforme est de considérer que celui-ci favoriserait l'intégration des élèves, le sentiment d'appartenance à leur établissement, voire même plus de discipline et d'ordre. De telles affirmations sont fantaisistes. Plusieurs recherches (Ansari, 2022[1] ; Brunsma, 2010) ont montré que le port de l'uniforme n'exerce pas d'effet sur les problèmes de comportement tels que l'anxiété, l'isolement, l'agressivité ou la destruction de biens. Le port de l'uniforme n'exerce pas davantage d'effets sur les apprentissages scolaires, le sentiment d'appartenance à l'établissement et les aptitudes sociales des élèves (empathie, respect d'autrui…). Plutôt que d'expérimenter l'uniforme, le ministère G. Attal n'aurait-il pas dû, d'abord, étudier la littérature scientifique ?

Outre qu'il n'apporterait pas les effets positifs attendus, l'uniforme comporte plusieurs inconvénients. D'abord, si l'uniforme était généralisé à l'ensemble des élèves, la dépense pour l'État et les collectivités territoriales serait de l'ordre de 360 millions d'euros. D'autres dépenses dont l'utilité est incontestable devraient être prioritaires (heures de soutien aux élèves en difficulté, renouvellement du matériel informatique…). Ensuite, par leur définition même et leur standardisation, les uniformes sont mal ajustés à la morphologie de chacun. A contrario, le vêtement, souvent soigneusement choisi, est un élément de l'identité, une façon de se cacher et de paraître plus branché, plus ceci ou plus cela. Le choix du vêtement est une liberté et une composante des droits des élèves déjà insuffisamment respectés (Merle, 2023). L'école est suffisamment contraignante pour ne pas ajouter des contraintes supplémentaires pour les élèves, pour les conseillers d'éducation qui devront contrôler le port de l'uniforme, et pour les parents, souvent les mères, qui devront en assurer l'entretien.

Alors même que les connaissances scientifiques ne montrent pas l'intérêt du port de l'uniforme, pourquoi une telle politique est-elle défendue par G. Attal ? La raison tient en partie à une idéologie qui associe l'uniforme, celui des militaires, gendarmes et policiers, à l'ordre et la discipline. Mais les élèves ne sont pas des fonctionnaires et les écoles ne sont pas des casernes ! L'histoire de l'école le montre suffisamment. Au XIX siècle, si l'uniforme réglementaire imposé par Napoléon dans les internats des lycées impériaux créés en 1802 a progressivement disparu, les lycéens, petite minorité de garçons appartenant aux catégories les plus aisées, portaient généralement des tenues distinctives, redingote et gilet à boutons. Ces tenues des plus classiques n'ont nullement empêché, de 1870 à 1888, plus d'une e centaine de révoltes dans la centaine de lycées existant à cette époque. La plus célèbre, celle de 1883, opposa 60 agents de police à 300 « insurgés », élèves du prestigieux lycée Louis le Grand (Lec F., Lelièvre C., 2007). Les dégâts matériels furent considérables ! Penser que l'uniforme à l'école apporterait égalité et discipline est une idée fausse, façonnée par l'idéologie et l'inculture historique.

Si l'ignorance de l'histoire scolaire et la croyance aux vertus sécuritaires de l'uniforme ont pu inspirer le projet du port de l'uniforme, le projet de G. Attal, soutenu par E. Macron, est aussi, et peut-être surtout, classiquement démagogique. F. Fillon et M. Le Pen étaient favorables à l'uniforme lors de la campagne présidentielle de 2017. Ils ne faisaient que reprendre une antienne familière aux partis politiques de droite (Lelièvre, 2023). Pour le pouvoir actuel, la tentation démagogique est d'autant plus grande qu'une majorité d'enquêtés sont actuellement favorables au port de l'uniforme, surtout les électeurs du Rassemblement National (83 % d'entre eux), des Républicains (78 %) et du camp présidentiel (76 %) (Sondage HuffPost, octobre 2023). Pour éviter la fuite de son électorat vers la droite, E. Macron et G. Attal sont tentés -les politiques tombent toujours du côté où ils penchent -de mettre en oeuvre les mesures proposées par leurs concurrents politiques les plus proches.

Cette démagogie politique, sur l'école comme sur d'autres sujets, est vouée à l'échec. Outre que les électeurs préfèrent toujours l'original à la copie, les sondages sur le port de l'uniforme relèvent, pour une grande part, de la construction de l'opinion. Si les questions posées avaient été, en reprenant la formulation du sondage HuffPost, « Dans quelle mesure êtes-vous favorable à plus ou moins de surveillants dans les établissements scolaires ? », ou bien « Dans quelle mesure êtes-vous favorable à plus ou moins d'élèves par classe ? », les enquêtés auraient probablement été favorables à ces deux mesures dont l'utilité dans le quotidien des établissements n'est nullement contestable. La manipulation opérée par le sondage est de créer l'illusion d'une opinion sur des questions qui n'intéressent par forcément, voire pas du tout, les enquêtés. Consultés par vote électronique, les élèves de la cité scolaire Touchard-Washington au Mans (Sarthe), directement concernés, ont largement participé et rejeté à 78 %, l'idée d'une tenue unique. L'expérimentation souhaitée par G. Attal semble déjà un échec faute d'établissements volontaires. L'évaluation de cette expérimentation sera de surcroît biaisée. Elle ne reposera pas sur des établissements tirés au sort mais sur des établissements volontaires, par définition non représentatifs.

Stratégie électorale, le débat sur le port de l'uniforme relève aussi d'une stratégie du buzz. L'obsession des politiques est d'occuper l'espace médiatique, façon directe d'éviter l'émergence de questions plus centrales. Pendant que les uns et les autres dissertent sur l'intérêt du port de l'uniforme, les médias ne s'intéressent plus à la crise du recrutement des professeurs, au nombre d'élèves par classe particulièrement supérieur en France par rapport à la moyenne européenne, à la faiblesse des salaires des professeurs, à la forte inégalité de la réussite selon l'origine sociale, etc. Cette stratégie du buzz est une façon de détourner l'attention et, mieux encore, d'accréditer l'idée que le port de l'uniforme apporterait des solutions aux problèmes de l'école. Une politique éducative digne de ce nom ne peut se réduire à une manipulation de l'opinion.

Les groupes de niveaux : une question scolaire centrale L'idée de « groupes de niveaux » a été lancée par G. Attal dans le cadre de son « choc des savoirs ». Le principe en est désormais connu. Dès la rentrée scolaire 2024, dans chaque classe de sixième et de cinquième, en mathématiques et en français, devraient être créés trois groupes de niveaux : faible, moyen et fort. Les groupes d'élèves faibles devraient bénéficier d'un nombre d'élèves réduits à 15 afin de combler leurs lacunes. Pour ne pas figer les situations scolaires, la composition de ces groupes serait périodiquement revue afin de tenir compte des progrès des uns et des difficultés des autres. Ainsi présenté, le principe des groupes de niveaux peut paraître séduisant à une personne ignorante des réalités des élèves et des recherches scientifiques. Ces groupes peuvent faire l'objet de trois critiques incontournables. D'abord, il existe un consensus scientifique sur les effets délétères de l'étiquetage des élèves sur leur progression scolaire. L'appartenance à un groupe d'élèves faibles exerce un effet négatif sur l'image de soi scolaire, la motivation, les capacités attentionnelles et, in fine, le niveau des apprentissages. Cet étiquetage scolaire constitue une prophétie autoréalisatrice. A contrario, la progression des élèves faibles est favorisée par la présence de pairs aux compétences supérieures. Ces « locomotives » de la classe sont également utiles aux professeurs pour faire progresser l'ensemble des élèves. L'objectif louable de ne pas assigner en permanence un élève à tel ou tel groupe nécessitera des évaluations incessantes, source permanente de stress et de perte de temps au détriment des apprentissages. Mais surtout, ces changements de groupe sont pour une grande part illusoires. Au cours d'une année scolaire, la quasi-totalité des élèves progresse. La probabilité est forte pour que, malgré leurs progrès, une bonne part des élèves du groupe des faibles y demeure toute l'année. Peut-on imaginer une pratique pédagogique plus démotivante ?

Ensuite, en raison de la corrélation bien établie entre origine sociale et niveau scolaire, constituer des groupes de niveaux dès l'entrée en sixième implique nécessairement un tri social. Le groupe des forts sera majoritairement composé d'élèves d'origine favorisée et la situation inverse caractérisera le groupe des faibles. Il est en effet établi que dès l'âge de deux ans les inégalités sociolinguistiques (lexicale, syntaxique, grammaticale) sont particulièrement fortes selon l'origine sociale (INED, 2019). Il faut de longues années d'apprentissage aux enfants de milieux populaires pour s'imprégner de la culture scolaire grâce à la qualité de leurs professeurs et à leur socialisation avec des enfants des autres milieux sociaux. Avec des groupes de niveaux séparant dès onze ans les enfants de niveaux scolaires et sociaux différents, G. Attal propose une organisation scolaire qui, dans l'histoire de l'école, a constitué un frein puissant à l'égalité des chances. Inspirés par le passé, les groupes de niveaux ne peuvent que renforcer l'inégalité des chances scolaires.

Enfin, les comparaisons internationales montrent l'inefficacité des groupes de niveaux. Le système éducatif allemand est structuré, à la fin du primaire, par trois orientations principales. Le Gymnasium, équivalent du lycée français du XIX siècle, filière la plus réputée, prépare au baccalauréat (abitur) ; la Realschule, jusqu'à 16 ans, prépare aux formations professionnelles mais permet aussi d'accéder au lycée ; la Hauptschule, jusqu'à 15 ans, débouche sur l'apprentissage. Suite aux résultats décevants de l'école allemande à l'évaluation PISA 2000, une majorité des landers a décidé de suivre l'exemple des systèmes éducatifs plus performants, tels que la Finlande, et de développer activement un collège unique, la Gesamtschule, susceptible de scolariser tous les élèves à la fin du primaire (Merle, 2012).

Toutefois, dans la Gesamtschule, pour les mathématiques et leur langue maternelle, les élèves étaient séparés selon leur niveau scolaire avec, au minimum, un groupe d'élèves forts, susceptibles de réussir l'abitur. Le résultat de cette politique est connu. Dans les évaluations Pisa 2022, les élèves allemands ont un niveau proche des élèves français et les inégalités de compétences selon l'origine sociale sont quasi aussi importantes (DEPP, 2023). D'autres pays ont fait un choix exactement contraire, notamment la Pologne. En 2000, le système éducatif polonais se caractérisait par huit années d'école primaire, de 7 à 14 ans, et débouchait sur une orientation précoce. La réforme a consisté à réduire d'une année la durée du primaire et de créer, pour tous les élèves, l'équivalent d'un collège unique d'une durée de trois années. Le choix d'une orientation générale, technique ou professionnel a donc été reporté à l'âge de 16 ans. Suite à cette réforme, les résultats scolaires des élèves polonais aux évaluations Pisa ont sensiblement progressé de 2000à 2012(Le Donné, 2016). En 2022, le niveau moyen en mathématiques des élèves polonais de 15 ans est de 489 points, bien supérieur à celui des élèves français (474 points). L'école polonaise n'est pas seulement plus efficace, elle est aussi plus équitable : l'écart de performances des élèves selon leur origine sociale est, contrairement à l'école française, largement en dessous de la moyenne européenne (note d'information, n°48, 2023, figure 3). La comparaison entre les réformes des systèmes éducatifs allemand et polonais est édifiante. Une scolarité commune à tous les élèves favorise l'efficacité et l'équité. Souhaités par G. Attal, les groupes de niveaux provoquerons l'effet contraire.

Finalement, l'effet délétère des groupes de niveaux est triplement montré par les recherches scientifiques, l'analyse historique et les comparaisons internationales. A ces trois raisons qui devrait déboucher sur l'abandon de la réforme, il faut ajouter trois remarques complémentaires. e D'une part, les ressources supplémentaires en heures de cours nécessaires à la mise en place des groupes de niveaux vont entraîner la suppression d'une grande part des systèmes d'aide déjà existants (soutien personnalisé, travail en demi-groupe, etc.). D'autre part, le texte proposant des groupes de niveaux a été unanimement rejeté par le Conseil supérieur de l'éducation. Mieux vaut éviter le conflit dans une institution scolaire déjà caractérisée par la crise des vocations et les démissions.

Enfin, l'instauration de groupes de niveaux nécessite des ressources humaines considérables qui seront réparties de façon égalitaire dans tous les collèges alors même que leurs besoins sont radicalement différents. Pour réduire la forte inégalité de réussite selon l'origine sociale, les collèges au recrutement les plus populaires et dont les élèves sont les plus en difficulté doivent bénéficier de davantage de ressources afin, notamment, de réduire le nombre d'élèves par classe, politique particulièrement efficace (Piketty et Valdenaire 2006 ;Bourgen, Grenet, Gurgand, 2017). Une école plus efficace et plus équitable nécessite une différenciation des moyens, idée centrale ignorée par le projet Attal.

Pourquoi, alors même qu'il n'existe pas d'arguments en faveur de la création des groupes de niveaux, G. Attal en souhaite-t-il, même devenu premier ministre, si résolument sa mise en oeuvre ? Certes, le projet est globalement soutenu par la droite et l'extrême droite. Il demeure toutefois largement rejeté. Le projet d'Attal est moins démagogique qu'idéologique. Il ne fait que prolonger les idées de la majorité des élites politiques, économiques, intellectuels et religieuses des siècles passés, souhaitant toujours réduire la scolarité des enfants d'origine populaire ou les scolariser à part. En concevant les groupes de niveaux, G. Attal prolonge cette tradition du séparatisme scolaire et social. Cette orientation conservatrice est confirmée lorsqu'il déclare que l'hétérogénéité des classes « condamne certains à stagner et empêche d'autres de s'envoler ». Outre que son affirmation est une contrevérité scientifique, le propos révèle un idéal de l'entre soi, un projet d'une école ségrégationniste où chacun resterait à sa place. G. Attal est dans une logique de reproduction, un partisan plus ou moins conscient de l'apartheid scolaire (Felouzis, 2005).

Conclusion : centralisation, flou argumentaire et confusions conceptuelles

Le principe des groupes de niveaux se caractérise par un renforcement de la centralisation. Quels que soient les effectifs scolarisés, les innovations pédagogiques mises en place, le recrutement social de l'établissement, le niveau scolaire des élèves… les groupes de niveaux devraient devenir l'organisation scolaire unique, excluant toute adaptation locale. L'opposition unanime aux groupes de niveaux, outre leurs effets délétères incontestables, tient aussi à ce caporalisme ministériel, à une décision majeure de transformation structurelle de l'école française décidée par quelques « experts » soigneusement choisis et parfois complaisants. Il est pour le moins étonnant que la nouvelle ministre de l'éducation, qui affirmait en 2016 que « Le système éducatif français doit évoluer vers plus de décentralisation territoriale et fonctionnelle » (Après-Demain, 2016), se soit convertie à une conception de l'action publique strictement inverse à ses convictions antérieures.

Une autre caractéristique du projet Attal tient au flou de son argumentation. L'ex ministre de l'éducation a condamné « l'uniformité » des collèges et, dans le même temps, veut imposer celle-ci. Les groupes de niveaux ont pour objet de gérer « l'hétérogénéité » des classes alors même que celle-ci est déjà réduite dans chaque établissement en raison de la multiplication des options (Son et Riegert, 2016). La cohérence argumentaire de la réforme échappe. Tous les professeurs et parents savent parfaitement qu'il existe des collèges scolarisant massivement des enfants d'origine populaire au niveau scolaire faible et, inversement, des collèges au recrutement social particulièrement aisé et d'un bon niveau scolaire. Pourquoi faudrait-il impérativement réduire cette hétérogénéité déjà limitée? L'ex ministre de l'éducation nationale, faute de présenter une argumentation convaincante, demeure cantonné au registre de l'affirmation et de l'imposition : « Les groupes de niveau vont se mettre en place, les organisations syndicales sont opposées, mais j'assume de dire que cette mesure est nécessaire » (Les Echos, 9 février 2024). Faut-il rappeler au nouveau premier ministre que la posture du chef, le fameux « j'assume » et l'adjectif « nécessaire », ne sont pas des arguments ? Enfin, les débats relatifs aux groupes de niveaux favorisent les confusions conceptuelles. Les groupes de niveaux ont notamment pour objet de « gérer l'hétérogénéité » qu'il faudrait absolument réduire grâce à la création des groupes de niveaux. Toutefois, cette hétérogénéité n'est rien d'autre que la mixité sociale et scolaire. Pour cette raison, les groupes de niveaux sont contraires à l'objectif de mixité sociale affirmé par la loi du 24 août 2021 confortant les principes de la République : « L'autorité de l'État compétente en matière d'éducation veille (…) à l'amélioration de la mixité sociale au sein de ces établissements » (article 58). La ministre Belloubet n'échappe pas à cette confusion conceptuelle. Elle ne peut à la fois défendre les groupes de niveaux et affirmer que « le risque, celui que je ne laisserai pas advenir, c'est la sélection par l'échec, un refus de mixité scolaire et sociale dans les classes ». Elle doit faire un choix entre ségrégation et mixité sociale.

Il existe des solutions à l'impasse actuelle. Par exemple, sur les quatre heures hebdomadaires de cours, une heure par semaine, dans un groupe de 15 élèves, pourrait être consacrée à la compréhension de notions difficiles. Dans le cadre de travaux dirigés, les élèves pourraient travailler en autonomie partielle, dans des groupes de deux ou trois, dont la composition serait en partie variable à chaque séance. Ces groupes seraient constitués d'élèves de niveaux scolaires plus ou moins hétérogènes. Certains élèves sont tout à fait disposés à aider leurs camarades. Outre qu'il s'agit d'une action altruiste et valorisante, cette aide développe leurs compétences cognitives et socio-émotionnelles. Dans la devise de la République, la liberté, la solidarité et la fraternité sont considérés ensemble. Favoriser les pratiques coopératives entre élèves est une façon de créer des solidarités, des amitiés et des habitudes de travail insuffisamment présentes. Produit de l'ignorance des pratiques pédagogiques novatrices, de l'inculture scientifique, des réflexes démagogiques et du poids d'une idéologie conservatrice, le projet Attal des groupes de niveaux doit être résolument combattu.

  pierre-merle-quels-fondements-a-la-politique-educative-demagogie-ideologie-ouconnaissances-scientifiques/

Pierre Merle

Professeur émérite de sociologie

[1] https://www.tandfonline.com/doi/abs/10.1080/00220679809597575


Sadaf Abdul Rauf
email: sadaf.abdulrauf@gmail.com

François Yvon
email: yvon@isir.upmc.fr

Translating Scientific Abstracts in the Bio-Medical Domain with Structure-Aware Models

Keywords: Neural Machine Translation, Document-level Machine Translation, Bio-medical Natural Language Processing

Machine Translation (MT) technologies have improved in many ways and generate usable outputs for a growing number of domains and language pairs. Yet, most sentence based MT systems struggle with contextual dependencies, processing small chunks of texts, typically sentences, in isolation from their textual context. This is likely to cause systematic errors or inconsistencies when processing long documents. While various attempts are made to handle extended contexts in translation, the relevance of these contextual cues, especially those related to the structural organization, and the extent to which they affect translation quality remains an under explored area. In this work, we explore ways to take these structural aspects into account, by integrating document structure as an extra conditioning context. Our experiments on biomedical abstracts, which are usually structured in a rigid way, suggest that this type of structural information can be useful for MT and document structure prediction. We also present in detail the impact of structural information on MT output and assess the degree to which structural information can be learned from the data.

Introduction

Neural Machine Translation (NMT) systems have progressed remarkably well in the last decade with some claims of reaching human parity [START_REF] Hassan | Achieving human parity on automatic Chinese to English news translation[END_REF][START_REF] Popel | Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals[END_REF] but much remains to be studied about the underlying learning of NMT systems. While substantial developments have been made towards integrating a document-level context [START_REF] Voita | Context-aware neural machine translation learns anaphora resolution[END_REF][START_REF] Popescu-Belis | Context in neural machine translation: A review of models and evaluations[END_REF]5,[START_REF] Maruf | A survey on document-level neural machine translation: Methods and evaluation[END_REF], the exploration of these contextual cues and the extent to which they affect translation quality remains an under explored area [START_REF] Fernandes | When does translation require context? a data-driven, multilingual exploration[END_REF].

Translation depends upon multiple sources of dependencies between sentences in a document, that may be due to semantic, stylistic, discursive or pragmatic issues. Moreover, complex documents are also structured in parts, subparts and sections, and depending on their position within this global structure, sentences may convey varying pragmatic functions. Titles, introductory or conclusion statements, for instance, are typically thought to be playing a distinguished role and to contain the most important elements of content [START_REF] Hofmann | The impact of document structure on keyphrase extraction[END_REF].

Sections in a document often follow distinct writing styles and lexical preferences. For instance, an analysis of 50 most frequent words from the medical abstracts used in our experiments (see Section 4.1) showed that Titles had the most domain specific content words; the word "however" was in the most frequent words for Introduction and Conclusion only; Abstract, Results and Conclusion had the most overlap of lexical space. Grammatically, sentences in the Results and Materials and Methods sections were majorly in past tense, whereas Conclusion sentences were in present continuous as well as past tense.

Objectives are mostly written in present tense with a few sentences in past tense and so on. As sections in a document have their own grammatical and lexical blueprints, which must be reflected in their textual content, we hypothesize that informing MT systems about sentences' placement in a document may help improve the overall performance. For a translation system, the capacity to model the context may notably improve certain translation decisions, e.g. a better or most consistent lexical choice [START_REF] Kuang | Modeling coherence for neural machine translation with dynamic and topic caches[END_REF], or a better translation of anaphoric pronouns [START_REF] Voita | Context-aware neural machine translation learns anaphora resolution[END_REF]5]. The research efforts to augment NMT models with contextual or document-level information may be broadly categorized as data-vs model-centric approaches, where datacentric approaches focus on increasing the scope of the input to include more source and/or target texts [START_REF] Tiedemann | Neural machine translation with extended context[END_REF][START_REF] Zhang | Improving the Transformer translation model with document-level context[END_REF][START_REF] Tu | Learning to remember translation history with a continuous cache[END_REF][START_REF] Junczys-Dowmunt | Towards largescale document-level neural machine translation[END_REF], while model-centric approaches aim to adjust the training architecture and objective [START_REF] Guo | Star-transformer[END_REF][START_REF] Lopes | Documentlevel neural MT: A systematic comparison[END_REF][START_REF] Ma | A simple and effective unified encoder for document-level machine translation[END_REF]. Recent surveys of this subdomain of NMT are in [START_REF] Maruf | A survey on document-level neural machine translation: Methods and evaluation[END_REF][START_REF] Abdul Rauf | Document level contexts for neural machine translation[END_REF].

In this work, we study whether also incorporating structural information can help improve the MT choices and thus positively impact translation quality.

Note that the structural information considered here is related to the pragmatic organization of the documents, where sectioning helps to organize the discourse into coherent subparts, each aiming to fulfill a specific goal. This is in contrast with other attempts to translate structured documents [START_REF] Hashimoto | A high-quality multilingual dataset for structured documentation translation[END_REF][START_REF] Hanneman | How should markup tags be translated?[END_REF] which focus on integrating syntactic markers of the structure or [START_REF] Dobreva | Document sub-structure in neural machine translation[END_REF], where structural organization identifies thematic changes. In our data-centric approach, contextual and structural information are injected for each sentence based on a hierarchy of special tokens and does not require any change in the model. While simple, this technique allows us to study whether learning coherent, section-specific, translation styles can contribute to improving the overall performance.

As tags can be used in multiple ways, we further explore the differences in tagging the source or the target side, which also allows us to evaluate how reliably the structure information can be learned automatically from the data. We systematically study the impact of each tag on metric score and the translation output in conjunction with human analysis of section wise lexical preferences versus the MT models section prediction capabilities.

Our experiments focus on abstracts of scientific texts in the bio-medical domain [START_REF] Haddow | Machine translation in healthcare[END_REF][START_REF] Zappatore | Adopting machine translation in the healthcare sector: A methodological multi-criteria review[END_REF], where document structure information is often available. For this target genre, texts are expected to follow a standardized structure comprising typical subsections of one to five lines, known as the IMRaD structure (for Introduction, Method, Results, and, Discussion) [START_REF] Sollaci | The introduction, methods, results, and discussion (IMRAD) structure: a fifty-year survey[END_REF]. As we discuss in Section 3.1, preparing the data for these experiments has been challenging and a significant outcome of our work will be a new dataset annotated with document structure, that we will release to the community 1 .

In summary, the main research questions that we address in this work are:

1. [RQ1] Does incorporating document structure information in text help improve machine translation performance?

[RQ2]

What is the most effective way to incorporate this auxiliary information?

3.

[RQ3] When the structure is not observed, how well can it be automatically predicted? What will be the effect of prediction errors? This paper is organized as follows: we start with a brief overview of Neural Machine Translation models (section 2), followed by an elaboration of our approach and architecture in sections 3 and 4. Results based on automatic metric scores are detailed in sections 5 and 6, whereas the analysis of a human evaluation is presented in 6.3. The paper concludes with an overview of related works (Section 7) and a general discussion in Section 8.

Neural Machine Translation

Probabilistic encoder-decoder models

In this section, we briefly introduce the main concepts of neural machine translation (NMT), with an emphasis on concepts whose understanding is important for accurately describing methods that handle context. A much more complete presentation of NMT is in [START_REF] Koehn | Neural Machine Translation[END_REF][START_REF] Stahlberg | Neural machine translation: A review[END_REF].

The general principle of all MT architectures is to generate the best possible target translation e= e 1 . . . e T of the input source sentence f =f 1 . . . f S , according to a probabilistic decision rule:

e * = argmax e p θ (e|f ), (1)
where the model parameters (θ) are estimated from sentence-aligned parallel corpora. Learning such a distribution is unrealistic if one considers complete sentences as the random variable in the model. However, neural implementations of this probabilistic model enable to directly factorize this distribution as:

p θ (e|f ) = T t=1 p θ (e t |e <t , f ), (2)
without having to resort to conditional independence hypotheses (eg. Markovian dependencies).

Equation ( 2) simply states that the probability of each target word is generated conditioned on the current prefix of the target sentence (e <t = e 1 . . . e t-1 )

and on the entire source sentence (f ). The manipulation of such distributions is made possible by transforming discrete contexts (e <t , f ) and words e t in the vocabulary into continuous representation spaces. This means that each word f t (and accordingly each context or each substring) is associated with a large numerical vector representing the useful information about that word or context.

Factorization [START_REF] Popel | Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals[END_REF] suggests that the program (1) can be solved approximately by generating words from left to right according to the following greedy procedure:

p θ (e t |f ) = argmax T t=1 p θ (e t |e <t , f ). (3)
More effective search procedures can also be entertained (see [START_REF] Wiher | On decoding strategies for neural text generators[END_REF] for a review).

The estimation of θ is performed by maximizing the conditional log-likelihood (or cross-entropy) T t=1 log p θ (e t |e <t , f ) accumulated over a large set of sentences, resulting in a complex optimization program that is typically solved in an approximate manner by generic numerical optimization algorithms.

The main neural architectures are mainly distinguished according to the way the conditioning context is encoded. The Transformer architecture [START_REF] Vaswani | Attention is all you need[END_REF] replaces the recurrent component (in the source and target) introduced in [ Decoding must still be carried out from left to right, since each word already generated conditions the generation of future words.

Monolingual and Bilingual Attention Modules

In a nutshell, the Transformer architecture transforms a structured context (a sequence of tokens, but this can also be a tree or a graph) into a single numerical vector. The core operation in this transformation is the iterative computation of each individual token's representation based on their similarity to other tokens in the context. Denoting H l-1 = [h l 1 , . . . , h l T ] the (T × d) matrix representing a context of length T at the input of layer l, the representation hkl i for token i is computed by attention head kl as:

hkl i = softmax( h l-1 i Q kl [H l-1 K kl ] T √ d k )H l-1 V kl , (4)
, with Q kl , K kl , V kl the parameter matrices associated to this head and layer, d the model dimension, and d k the size of each of the K heads (d k = d/K). In this model, H 0 contains the lexical and positional embeddings. The output of these k computations are then concatenated and passed through a feed-forward (FFN) layer with ReLU activation; each of these steps also includes a summation with H l-1 and a layer normalization. In equations:

Hkl = Attn(H l-1 Q, H l-1 K kl , H l-1 V kl ) H l = LayerNorm(H l-1 + [ H1l , . . . , HKl ]) H l = LayerNorm(H l + FFN(H l )),
where Attn denotes the computation expressed by equation ( 4).

When used for MT, the self-attention mechanism is computed on the source side on the full observed sentence. On the target side, the self-attention is restricted to the previous words; an additional cross-attention module combines at each layer the decoder-side representations with the encoder representations output by the top layer H K : this way, information may flow from source to target, but not in the reverse direction.

Methods: Translating with Document Structure

We propose to introduce document structure information at the level of each sentence through a system of hierarchical tags. Tags have been widely used in NMT to incorporate additional discrete conditioning factors in (1), often for issues quite distinct from those caused by long-range dependency, e.g. to control the output language, the domain, or the level of formality and politeness [START_REF] Wu | Google's neural machine translation system: Bridging the gap between human and machine translation[END_REF][START_REF] Kobus | Domain control for neural machine translation[END_REF][START_REF] Madaan | Politeness transfer: A tag and generate approach[END_REF]. Tags also provide us with a simple and effective way to take structure into account.

3.1. Tagging sections within documents

The IMRaD scheme

Our starting point is that scientific documents in our target domain often follow a rigid structure, known as the IMRaD structure [START_REF] Sollaci | The introduction, methods, results, and discussion (IMRAD) structure: a fifty-year survey[END_REF]. This means that most abstracts will comprise one or several lines for the following sections:

Introduction, Method, Results, and, Discussion (see example in Figure 1), where we also consider the title line to be part of the structure.

Note that this structure can be overt and introduced by appropriate headings, or remain covert. In the latter case, the document structure cannot be immediately recovered from the text. Also note that variations in the organization and in the heading labels can be observed depending on the subdomains and publication venues.

Our main hypothesis is that each of these sections can be characterized with specific terms and phraseological patterns: in our experiments, we thus explore how to use this information in NMT and measure the correlated impact. We notably expect that by informing the system with sub-document information, it will better translate the typical style and phraseology of sentences occurring in each part.

For this purpose, we identified in our data all the abstracts that were conforming to this basic structure and worked to make this structure as explicit and standardized as possible. This notably implied to normalize the main headings, Title:

Comparison of two azithromycin distribution strategies for controlling trachoma in Nepal.

OBJECTIVE:

The study compares the effectiveness of two strategies for distributing azithromycin in an area with mild-to-moderate active trachoma in Nepal.

METHODS:

The two strategies investigated were the use of azithromycin for 1) mass treatment of all children, or 2) targeted treatment of only those children who were found to be clinically active, as well as all members of their household.

FINDINGS:

Mass treatment of children was slightly more effective in terms of decreasing the prevalence of clinically active trachoma (estimated by clinical examination) and of chlamydial infection (estimated by DNA amplification tests), although neither result was statistically significant.

CONCLUSION:

Both strategies appeared to be effective in reducing the prevalence of clinically active trachoma and infection six months after the treatment.

Antibiotic treatment reduced the prevalence of chlamydial infection more than it did the level of clinically active trachoma. as there was a large degree of variance in subheadings and structures across corpora (see § 4.1.3). To incorporate the standard IMRaD format, we thus mapped each subheading to the corresponding IMRaD label using a system of tags displayed in Table 1 The identification and standardization of the subheading information was a tedious process, involving a lot of rule-based processing and human intervention to take the variability of sub-headings into account. In order to reconstruct the fully parallel versions with subheadings, we also had to reinsert explicit headings in the source or target files when they were missing. Also, note that this information was not available for all biomedical abstracts.

In this case, we tagged every line but the title with a generic tag (<ABS>).

A hierarchy of tags

Building state-of-the-art NMT systems for abstracts in the biomedical domain requires to consider multiple sources of parallel data, opportunistically collecting texts from a variety of genres and domains [START_REF] Peng | Huawei's NMT systems for the WMT 2019 biomedical translation task[END_REF]. Our own training data, presented in Section 4.1, is accordingly very diverse, comprising in-domain parallel and outof-domain parallel corpora, texts automatically retrieved parallel collections, as well in-domain monolingual data that is automatically back-translated. Some are made of lists of isolated sentences, while others retain the document information.

Even within the in-domain data, some texts precisely match the genre of the test set (scientific abstracts), whereas others can be quite remote (eg. regulatory documents from the EMA 3 ). This means that for most of our training data, the document structure will often be either unknown, or will not correspond to the scientific genre and will not obey the IMRaD pattern.

In order to reflect this diversity and provide tags for all parallel sentences in our corpus, we extended our tagging scheme with two additional levels of tagging.

We then use hierarchical tags of the form (<1> <2> <3>), to incorporate a

3 European Medicines Agency, formerly known as EMEA. multi-level information about the domain, corpus and section for each sentence.

The first level of tags distinguishes between out-of-domain data (<G>), and in-domain data (<M>). The second level of tag aims to distinguish between data sources, hence the use of one dedicated tag for each corpus, except for data collected with Information Retrieval (IR) techniques and monolingual data, which are respectively tagged with <IR> and <BT> [START_REF] Caswell | Tagged back-translation[END_REF][START_REF] Marie | Tagged back-translation revisited: Why does it really work?[END_REF]. Finally, the third tag either corresponds to the structural information as described above, when it is available; otherwise all remaining sentences from other corpora were simply tagged with an "unspecified subheading" label (<US>).

Fully tagged sentences sampled from the Scielo test document displayed on

Using tags in MT

Having defined a tagging scheme, the next question concerns its use in machine translation. A first alternative is to use tags as words and insert them as prefix on the source side. This assumes that tags are always known prior to translation, and will have the effect to differentiate the contextual representations of source words, enabling the system to disambiguate the translation of polysemous words, whenever this is beneficial for the training loss. The same effect will be observed on the target side, thanks to the cross-lingual attention mechanism alluded to above. It is further possible to explicitly amplify this effect by injecting the tag information into the input representation of every token (eg. [START_REF] Kobus | Domain control for neural machine translation[END_REF][START_REF] Chu | An empirical comparison of domain adaptation methods for neural machine translation[END_REF] for domain information, or [START_REF] Conneau | Cross-lingual language model pretraining[END_REF] for language information).

The alternative is to inject the tags as a prefix of the target sentence. If we assume that tags are always observed, the expected effect is roughly similar to having tags on the source side. The main difference is that source token representations no longer depend on the tag value. We have not studied this approach, but have instead resorted to another way to use target side tags: to process them as if they were regular target words. The system will then be trained to predict their correct values using only the source side information (and the previous tag(s)). In this setting, source side representations will be adjusted to correctly generate tags and to extract features that are useful for this prediction [START_REF] Britz | Effective domain mixing for neural machine translation[END_REF]. With forced decoding [START_REF] Wuebker | Models and inference for prefix-constrained machine translation[END_REF], this approach can still be used when tags are observed. In addition, it can also be used to generate tagdependent translations, for unstructured abstracts, with unobserved tags. These two schemes are considered in our experiments and help to better analyse the usefulness of the structural tags. Similar analyses, for a variety of tags, are in [START_REF] Takeno | Controlling target features in neural machine translation via prefix constraints[END_REF][START_REF] Pham | Revisiting multi-domain machine translation[END_REF][START_REF] Wu | Language tags matter for zero-shot neural machine translation[END_REF]. They suggest that when tags are observed, having them on the source side yields better results. Note that these studies only consider one level of tagging, where we consider three.

To sum up, three scenarios will be contrasted below: (i) observed source side tags, (ii) observed target side tags (forced decoding), (iii) predicted target side tags: (i) and (ii)-(iii) differ in the training scheme, while (ii) and (iii) only differ in the way inference is performed.

Experimental Setup

Corpora and Pre-processing

We train our baseline systems on a collection of in-domain biomedical texts as well as out-of-domain parallel corpus, and use standard benchmarks from past studies for development and tests. Details are in Table 2.

Parallel corpora

We gathered parallel and monolingual corpora available for English-French in the biomedical domain. The former included the biomedical texts provided by the WMT'20 organizers: Edp, Medline abstracts and titles [START_REF] Jimeno Yepes | Findings of the WMT 2017 biomedical translation shared task[END_REF], Scielo [START_REF] Neves | The Scielo Corpus: a parallel corpus of scientific publications for biomedicine[END_REF] and the Ufal Medical corpus 4 consisting of Cesta, Ecdc, Emea (OpenSubtitles), PatTR Medical and Subtitles. Other sources of in-domain data are the Cochrane bilingual parallel corpus [START_REF] Ive | Diagnosing high-quality statistical machine translation using traces of post-edition operations[END_REF] 5 , the Taus Corona Crisis corpus 6 and the Mlia Covid corpus 7 . We also included additional in-domain data selected using Information Retrieval (IR) techniques from general domain corpora including News-Commentary, Books and Wikipedia corpus obtained from the Open Parallel Corpus (OPUS) [START_REF] Lison | Extracting large parallel corpora from movie and TV subtitles[END_REF]. These were selected using the data selection scheme described in [START_REF] Abdul-Rauf | On the use of comparable corpora to improve SMT performance[END_REF]. Medline titles were used as queries to find relevant sentences. We included the 2-best sentences returned from the IR pipeline into our additional corpus [START_REF] Naz | FJWU participation for the WMT20 biomedical translation task[END_REF].

Our out-of-domain corpora include the parallel data provided by the WMT'14 campaign for French-English: Gigafr-en, Common Crawl, Europarl, News Commentary and the UN corpora.

For development purposes, we used Medline testsets for WMT'18 [START_REF] Neves | Findings of the WMT 2018 biomedical translation shared task: Evaluation on Medline test sets[END_REF] and 19 [START_REF] Bawden | Findings of the WMT 2019 biomedical translation shared task: Evaluation for MEDLINE abstracts and biomedical terminologies[END_REF] bio-medical MT task, while Medline 20, Scielo (WMT '16) and Edp (WMT '17) were used as internal test data. 8 Test sets for both directions were combined to get larger test sets and are used to evaluate systems in both directions. We only used the parts with document structure, i.e. sentences with <ABS> were not included. The combined test set for Scielo and Medline had 8, 805 and 1315 sentences each, after removing the abstract sentences we get the test sets with 6475 and 997 sentences for Scielo and Medline20 respectively.

Monolingual sources

The backtranslation of monolingual sources has often been effectively used to cater for parallel corpus shortage in the bio-medical domain [START_REF] Stojanovski | The LMU munich unsupervised machine translation system for WMT19[END_REF][START_REF] Peng | Huawei's NMT systems for the WMT 2019 biomedical translation task[END_REF]. We also augment our training data with backtranslations using both French and English in-domain monolingual corpora.

Supplementary French data from three monolingual sources were collected from public archives: 1) abstracts of medical papers published by Elsevier from the Lissa portal; 9 2) abstracts from the national ISTEX archive;10 3) a collection of research articles collected from various sources11 [START_REF] Maniez | L'adjectif dénominal en langue de spécialité: étude du domaine de la médecine[END_REF] henceforth referred to as Med_Fr. These documents were automatically translated into English with an NMT system trained only on bio-medical corpora, with a reported BLEU score of 33.6 on Medline20 testset.

The English side of the Medline German and Spanish corpora distributed for WMT/Biomed 2021 12 is used as supplementary English data for backtranslation.

Duplicate documents were removed based on the document id information. For these abstracts, the internal structure of documents is often available and has been tagged as for the parallel data. These texts were then split into sentences 13and translated into French using an NMT system trained on all bio-medical corpora with a BLEU score of 36.4 on Medline20 testset.

Structured sources

The document structure is only available for a subset of the training data corresponding to the following sources: For the test and development corpora, when the structure was available, a manual alignment was performed based on the sentence alignment file provided by WMT organizers. 15 Alignment operations involved inserting an empty line after each subsection, deleting unaligned sentences and merging sentences as per alignment information. Statistics regarding the numbers of structured documents in the train, test and development corpora are in Table 2.

Translation systems

Our translation systems mostly use the basic Transformer architecture [START_REF] Vaswani | Attention is all you need[END_REF].

They all rely on Meta's seq-2-seq library (fairseq) [START_REF] Ott | fairseq: A fast, extensible toolkit for sequence modeling[END_REF] with parameter settings borrowed from the transformer_wmt_de_en recipe. 16 The ReLU activation function was used in all encoder and decoder layers. We optimize with Adam [START_REF] Kingma | A method for stochastic optimization[END_REF], set up with a maximum learning rate of 0.0005 and an inverse square root decay schedule, as well as 4000 warmup updates. All corpora are segmented into subword units using Sentence Piece [START_REF] Kudo | SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing[END_REF] with a vocabulary of 32K units. These units were learned on the union of all in-domain corpora. We share the decoder input and output embedding matrices. automatic evaluation. Evaluation was performed using SacreBleu [START_REF] Post | A call for clarity in reporting BLEU scores[END_REF] using BLEU [START_REF] Papineni | BLEU: a method for automatic evaluation of machine translation[END_REF], TER [START_REF] Snover | A study of translation edit rate with targeted human annotation[END_REF] and chrF [START_REF] Popović | chrF: character n-gram F-score for automatic MT evaluation[END_REF] metrics.

For fine-tuned systems, the process starts with models converged systems.

Training then resumes using a selected portion of the train data, with the same meta parameters and objective as for the base systems. In our results, names of fine-tuned systems are post-fixed with *-ft.

Document Structure Information and MT performance

To evaluate the impact of document structure information on NMT, we built systems with tags either on the source or target sides and compared them with baseline NMT systems. Scores show that incorporating tags substantially improves machine translation performance; this improvement is more prominent when tags are used on the source side to initialize the encoder input stream.

These results are tabulated in Table 3 on two bio-medical test sets, Medline20

and Scielo for standard MT systems (All ), source-tagged systems (Srctag) and destination-tagged systems (Trgtag). Baseline systems use all general domain and biomedical corpora totalling 41.7M sentences for EN-FR and 40.9M

sentences for FR-EN. Each of these systems is then further fine-tuned on biomedical corpora (5.7M sentences for EN-FR, and 4.9M sentences for FR-EN).

We use three MT metrics to evaluate the systems in this section. These include BLEU which uses n-gram precision and Brevity penalty, TER which gives scores based on edit operations and chrF which uses F-score statistic for character n-gram matches that a hypothesis requires to match the reference translation. Since TER is an error rate, a decrease in score signifies improvement, while for BLEU and chrF increase denotes improvement. We see that on average the three metrics agree with each other in the general trends exhibited by the systems, but BLEU is more prominent, thus we will present further analysis based on BLEU scores.

A general trend is that systems with extra document information perform substantially better than vanilla NMT systems. We find that it seems better to introduce the tags on the source than on the target side, similar to [START_REF] Takeno | Controlling target features in neural machine translation via prefix constraints[END_REF][START_REF] Pham | Revisiting multi-domain machine translation[END_REF][START_REF] Wu | Language tags matter for zero-shot neural machine translation[END_REF].

We see an average improvement of 0.92 and 0.46 BLEU points on the source and target side tags respectively. Further, in Trgtag setting, using reference tags is consistently better than predicting them, with more than 1 BLEU point difference between these settings for Scielo. These observations carry over for fine-tuned systems. We also see that the effect of fine-tuning is generally positive, with the exception for Scielo systems with Trgtag for EN-FR.

Ablation Experiment

An ablation experiment was conducted in the Srctag setting for French-English direction to isolate the impact of document structure information. For this, we trained two models with one tag each: the first model with only domain tags, the second with only document structure (section) tags. This is in contrast to our experiments with the hierarchical tagging scheme where 3 dimensions of information are used to condition the translation output. The rationale behind this ablation experiment is to isolate the effect of other dimensions.

The section and domain tagged systems had a BLEU score of 36.7 and

-src

Compte tenu du manque de renseignements concernant ce modèle modifié, nous avons procédé à une étude pour en déterminer la survie et le rendement à court et à moyen terme.

<INT>

Given the lack of information about this modified model, we conducted a study to determine shortand medium-term survival and performance.

<M>

Given the lack of information on this modified model, we conducted a study to determine its survival and performance in the short and medium term.

2 -src S'il existe de nombreuses approches chiropratiques, deux types de chiropracteurs peuvent-être identifiés; ceux s'intéressant aux troubles musculo-squelettiques et ceux souhaitant prendre en charge aussi des troubles non musculo-squelettiques.

<INT>

While there are many chiropractor approaches, two types of chiropractors may be identified;

those interested in musculoskeletal disorders and those wishing to also manage non-mechanic disorders.

<M>

If there are many cheropractical approaches, two types of cheropractors may be identified; those interested in musculoskeletal disorders and those interested in the management of non-mechanic disorders as well.

-src

Il est préoccupant de constater que les étudiants qui adhèrent au modèle de la subluxation soient prêts à intégrer ces opinions dans leurs futures prises en charge; souhaitant proposer des ajustements chiropratiques aux patients asymptomatiques.

<CON>

Concerningly, students who adhere to the subluxe model are prepared to integrate these views into their future care; wishing to propose chiropractical adjustments to asymptomatic patients.

<M>

It is worrying to note that students who adhere to the subluxation model are prepared to incorporate these views into their future management; wishing to propose cheropractical adjustments to asymptomatic patients.

-src

Les symptômes courants sont des maux de tête, des troubles de la vue, des acouphènes pulsatiles et un oedème papillaire.

<INT>

Common symptoms are headaches, visual disorders, perfumant psychosis, and hematomary edema.

<M>

Common symptoms are headache, vision disorders, puffering drowsiness, and facial edema.

respectively (at 360M updates)

. Section tagging gave an improvement of 0.5 BLEU points. We observe the impact on output in Table 4 where sentences 1-2 display a common pattern: the difference in grammatical preference between models trained with only section information vs only domain information.

Sentences 3-4 highlight variance in lexical choice for the two models. The difference in average sentence length of the output of the two systems was negligible (0.26 words). From these results, we conclude that compounded effect of hierarchical tagging is more pronounced as compared to using only one level of tagging.

Joint Prediction and Translation

In this section, we evaluate Trgtag systems in detail. Recall that for each sentence the model has to successively predict three tags, corresponding respectively to the domain, corpus and the section, before generating the output translation. We thus systematically evaluate and contrast (i) the accuracy of the prediction of each tag, and (ii) the impact of having a correct vs. incorrect prediction on the BLEU score. To do so, we use the systems trained with target side tags as prefix of the reference translation. This setting allows us to study how well tags can be automatically predicted from the source sentence and the resulting impact on the final translation.

For this, we force decode each test sentence four times with four prefixes of increasing size: pre0 is the empty prefix condition, where all tags are automatically generated, while pre3 initializes the decoder with the three reference tags;

pre1 and pre2 denote the two intermediary settings, respectively with one and two correct tags. The former two conditions respectively correspond to decoding with predicted or reference tags in Table 3, the latter two are novel.

Using reference vs. predicted tags

In our results, we bin the sentences based on tag predictions. Bin 000 contains sentences for which that all three tags are wrongly predicted; similarly

20

bin 111 contains sentences with three correct predictions, 101 those with a correct prediction for the first and third tags and so on. To measure the translation quality, BLEU scores are computed for each subset of sentences, excluding as before tags in the score computation. Condition pre0 is when we predict all three tags, while pre3, pre2 and pre1 involve some reference tag(s). Table 5 reports the BLEU scores and percentage of sentences in each bin.

We can make the following observations that apply for the two language directions. First, it turns out that predicting the first tag (domain) is easy which also explains the small gap in BLEU scores between pre0 and pre1. Having the second tag (corpus) right is the most challenging part, with an error rate higher than 75% for the non fine-tuned systems. This is because some of our in domain corpora are very close (eg. Cochrane and Medline). It remains difficult even after fine-tuning, with an error rate way above 60%. The third tag (section headings), is much easier -assuming the other tags are correct, we achieve accuracy higher than 50% (without fine-tuning) and higher than 60% with fine-tuning. When we get the corpus tag wrong, we are likely to fail for section prediction (see 100 vs.

101), i.e. if the system errs on corpus it is likely to err also on the section tag.

The main reason for this is data imbalance with just 8.7% in-domain sentences having document structure information (see Table 2).

BLEU-wise, we see overall a general improvement when we move from predicted tags (pre0) to correct tags (pre3); when the corpus is known, section prediction is relatively easy explaining why pre2 and pre3 are always very close, and in some cases it even seems slightly better to let the system use predicted sections headings than use the correct ones.

Structure vs No structure tags. As pre2 and pre3 scores are always very close, to better evaluate the impact of using reference section tags, we design another contrast and force decode all the test-sets with a 'generic' section tag instead of the actual IMRaD code. Results are in Table 6 and again contrast the effect of using source vs. target tags. where we even observe that the generic tag yields a tiny improvement over the other condition. These small differences may be due to the training data distribution, as 75% and 48% sentences of Medline and Scielo respectively had <ABS> as the section tag. Overall, these results do hint at a slight improvement in BLEU when the more specific section is known, but given the small difference and the data imbalance, we abstain to make any conclusive statement.

Predicting Subsection Tags

For this analysis, we report results on the Scielo testset, which among all the biomedical testsets, has the largest number of documents annotated with structure information (cf. Section 4.1.3). Confusion matrices for EN-FR are shown in Figure 3, and the corresponding metric scores and FR-EN confusion matrices are given in Appendix A.

Section prediction results as shown in Table 7 resonate with our human analysis of lexical and grammar usage in different sections shown in Table 8. We analysed 50 most frequent words per section along with their POS tags (using nltk). Some words are shown in Table 8 to demonstrate that the word itself is an indicator of its' most probable section and their usage is enriched in particular sections, e.g. the words like however and may exhibit higher probability to be As seen earlier, section prediction is better when the first two tags are known, i.e pre2, however title prediction remains good in FR-EN direction even for pre0 and pre1.

Introduction remains the most difficult section to predict but pre2 systems learn to predict well. Accuracy for the baseline systems is 0.61, which increases for fine tuned systems to 0.75 and 0.77 for EN-FR and FR-EN respectively.

Qualitative Analysis: Tag conditioned translation variants

In this section, we study how tags affect the lexical and grammatical choice in automatic translations. For this analysis, we analysed the 1167 translations of all the titles (<H1> tags) in the Scielo corpus and compared two cases: (a)

when the correct tag <H1> is used in source (b) when the generic tag (<ABS>) is used instead. Titles are interesting because they tend to be more compact in their wording than regular sentences and also differ in their syntactic structure as they mosly consist in long nominal phrases devoid of verbs (some illustrations in Table 9). As discussed above, titles can be predicted with some confidence (in pre2 conditions).

Overall, we observed that the two outputs were very comparable and hardly distinguishable in their BLEU scores (40.6 vs 40.9). In the majority of cases (931 titles), the two output translations are entirely identical. Out of the remaining cases, the differences are always small, yet we can identify interesting differences which illustrate the effect of these subsection tags. The first is that when using the <H1> tag, target titles are actually slightly shorter than in the other condition:

the average length difference between the two outputs is 1.06 chars, with 77 cases of (a) being longer, and 131 cases where (b) is longer -these two situations are illustrated in the first two examples of Table 9. 17 are quite heterogeneous in nature, but notably encompass coreference issues, coherence issues, and discourse-level issues [START_REF] Popescu-Belis | Context in neural machine translation: A review of models and evaluations[END_REF]. Coreference issues are mostly related to the consistency of pronoun use across languages, where the choice of a morphological variant for a pronoun (e.g., in gender or number) may be conditioned by its referent from previous sentence(s). Coherence corresponds to longer range phenomena that ensure that a translated text can be read as a whole, implying a consistent choice of terms, tense, style, and references throughout the text. Finally, DLMT also includes the generation of texts that correctly reproduce the argumentative structure of the source input. In the realm of neural architectures, DLMT has been mostly approached as "MT with long-range dependencies", fostering multiple approaches to integrate large spans of text in the translation context. These approaches range from simple extensions of SLMT where multiple sentences are translated as one unit [START_REF] Scherrer | Analysing concatenation approaches to document-level NMT in two different domains[END_REF][START_REF] Ma | A comparison of approaches to documentlevel machine translation[END_REF][START_REF] Sun | Rethinking document-level neural machine translation[END_REF], to more sophisticated proposals combining a short-term context (at the sentence level) with a long-range context. This can be achieved with dual encoders [START_REF] Bawden | Evaluating discourse phenomena in neural machine translation[END_REF],

hierarchical architectures [START_REF] Miculicich | Document-level neural machine translation with hierarchical attention networks[END_REF][START_REF] Chen | Modeling discourse structure for document-level neural machine translation[END_REF][START_REF] Zheng | Towards making the most of context in neural machine translation[END_REF], or cache-based methods [START_REF] Tu | Learning to remember translation history with a continuous cache[END_REF]. Widening the notion of context, topic models or even domain adaptation techniques can also be used to represent large documents [START_REF] Kuang | Modeling coherence for neural machine translation with dynamic and topic caches[END_REF]. A conclusion of several comparative studies [START_REF] Kim | When and why is document-level context useful in neural machine translation?[END_REF][START_REF] Lopes | Documentlevel neural MT: A systematic comparison[END_REF][START_REF] Sun | Rethinking document-level neural machine translation[END_REF] is that simple techniques (no context, or basic concatenation) are difficult to outperform. A review of DLMT is in [START_REF] Maruf | A survey on document-level neural machine translation: Methods and evaluation[END_REF]; more recent approaches based on large language models are documented in [START_REF] Wang | Document-level machine translation with large language models[END_REF].

Tags. have been widely used in NMT to incorporate additional discrete conditioning factors in (1), and control the output language, the domain, or the level of formality and politeness [START_REF] Wu | Google's neural machine translation system: Bridging the gap between human and machine translation[END_REF][START_REF] Kobus | Domain control for neural machine translation[END_REF][START_REF] Madaan | Politeness transfer: A tag and generate approach[END_REF], mitigate gender bias issues [START_REF] Saunders | Neural machine translation doesn't translate gender coreference right unless you make it[END_REF], and provide us with a simple and effective way to take structure in account.

Notably, [START_REF] Kobus | Domain control for neural machine translation[END_REF][START_REF] Takeno | Controlling target features in neural machine translation via prefix constraints[END_REF] use tags in multi-domain systems, an approach extended in [START_REF] Stergiadis | Multi-domain adaptation in neural machine translation through multidimensional tagging[END_REF] with multi-level; tags in [START_REF] Sennrich | Linguistic input features improve neural machine translation[END_REF][START_REF] Niu | A study of style in machine translation: Controlling the formality of machine translation output[END_REF][START_REF] Niu | Multi-task neural models for translating between styles within and across languages[END_REF][START_REF] Madaan | Politeness transfer: A tag and generate approach[END_REF] inform systems about the level of formality or politeness of a sentence; finally, multilingual MT [START_REF] Firat | Multi-way, multilingual neural machine translation with a shared attention mechanism[END_REF][START_REF] Wu | Google's neural machine translation system: Bridging the gap between human and machine translation[END_REF][START_REF] Johnson | Google's multilingual neural machine translation system: Enabling zero-shot translation[END_REF] use tags to select the desired target language. In the same line of studies, [START_REF] Kikuchi | Controlling output length in neural encoder-decoders[END_REF] use numerical tag values corresponding to length constraints, while [START_REF] Takeno | Controlling target features in neural machine translation via prefix constraints[END_REF] use a tag to enforce the decoding direction of a bidirectional system, in [START_REF] Marie | Tagged back-translation revisited: Why does it really work?[END_REF][START_REF] Xu | LISN @ WMT[END_REF] tags inform back-translated data, [START_REF] Marchisio | Controlling the reading level of machine translation output[END_REF] use tags to control the readability level of NMT output, [START_REF] Wu | Language tags matter for zero-shot neural machine translation[END_REF][START_REF] Mao | Exploring the impact of layer normalization for zero-shot neural machine translation[END_REF] demonstrate zero-shot NMT capability using tags.

Tags have also proven useful for grammatical error correction using NMT [START_REF] Liang | BERT enhanced neural machine translation and sequence tagging model for Chinese grammatical error diagnosis[END_REF] where tags are used with each word to indicate the operation e.g. keep, delete, replace etc. [START_REF] Freitag | A natural diet: Towards improving naturalness of machine translation output[END_REF] use tags to enforce the decoder towards natural MT output, [START_REF] Jain | Entity projection via machine translation for cross-lingual NER[END_REF] used tags for named entity recognition, whereas tags have been used

in [START_REF] Berard | Naver labs Europe's systems for the WMT19 machine translation robustness task[END_REF] for entity projection for cross-lingual NER.

To our knowledge, there exists no prior work on using tags to represent the document structure in NMT other than [START_REF] Xu | LISN @ WMT[END_REF], which only use tags on the source side. Closer to our work, one-level document structure-tags have been used for document quality prediction by [START_REF] De Buy Wenniger | Structure-tags improve text classification for scholarly document quality prediction[END_REF], which reports a strong correlation with text classification. They used three tags (Title, Abstract and BodyText) and reported performance reduction when using smaller structure-tag set on all three domains that they worked on.

In the context of Multilingual NMT, [START_REF] Wu | Language tags matter for zero-shot neural machine translation[END_REF] shows language tags to significantly impact zero-shot translation quality. They report best scores by placing the target language tags on the encoder side which helps alleviate the off-target issue for such models. Similar analyses, for a variety of tags, are in [START_REF] Takeno | Controlling target features in neural machine translation via prefix constraints[END_REF][START_REF] Pham | Revisiting multi-domain machine translation[END_REF],

and suggest that when tags are observed, having them on the source side yields better results. Our results also show better results with tags on the source side.

It is further possible to explicitly amplify this effect by injecting the tag information into the input representation of every token (eg. [START_REF] Kobus | Domain control for neural machine translation[END_REF][START_REF] Chu | An empirical comparison of domain adaptation methods for neural machine translation[END_REF] for domain information, or [START_REF] Conneau | Cross-lingual language model pretraining[END_REF] for language information).

Conclusion and Discussion

Sentences in a document generally follow a typical style, for example the introductory sentences are clearly distinguishable from concluding sentences based on the lexical choice and sentence style etc. Whether or not can an MT system be trained to learn and use these sentence specific styles is the question that we have studied in this article.

We have tried to take advantage of the rigid structure of abstracts in the biomedical domain to assess the effect of structure on the translation quality.

For this, we have carefully annotated a large subset of our training data with structural information, and trained models that were able to take this structure into account thanks to a hierarchical tag system that introduces an extra non-local context during translation. In addition to the elaboration of a new annotated resource, that will be useful for further works, our study has shown that (a) predicting the structural labels was possible from the sole source text, at least for fine tune systems exposed to structured documents; (b) based on this complex of tags, it was actually possible to improve our automatic evaluation scores, even though the actual effect of the sole structural tags on BLEU scores was found to be fairly limited in our test conditions. This highlights one clear limitation of our results, which mostly rely on automatic metrics such as BLEU or TER. It is likely that such scores may not fully capture the fine-grained stylistic variations that are implied by the use of document structure information.

In our future work, we would like to generalize this approach to other structural labels that may be available for other documents and and domains, as well as to combine this information with other, more fine grain, additional contextual information that may be useful for document-level MT. Another perspective is to explore how structural information is handled in NMT systems based on large language models, which represent a competitive alternative to the encoder-decoder models used in this study.

Appendix B. Examples of variations incurred by tag differences

For this analysis, we generated translations of random documents using all possible tag combinations in source and target sides. A large set of these combinations were human analysed, some sample outputs for three tag combinations 1045 are in Table B.12 for Srctag vs Trgtag with the corresponding BLEU, TER and chrF scores.

We observe variation in lexical choice as well as grammatical sentence formation, due to using Srctag vs Trgtag. We see the preference of all Srctag combinations for certain lexical choices, e.g. Srctag system outputs always 1050 starts with the article the. Similarly, several phrase constructions appear to have been learned from out-of-domain data as these majorly appear in Srctag sentences having any tag from {<G> <WMT> <US> }. 18 For instance, "death registration" and "has been" in second and third examples.

  28] by generalized attentional modules, while retaining the source-target cross-attention component of early RNN-based architectures. This change makes all previous positions t -1, t -2 . . . 1 equally important in the selection of the current word (and likewise for source representations). It also enables to process all the target tokens in parallel during training, causing vast decrease in training time.

Figure 1 :

 1 Figure 1: A structured abstract with an overt organization, extracted from Scielo (fr-en) test set, a bio-medical corpus. In this example, the Introduction is missing, and anObjective part is present; the Result part is labeled 'FINDINGS'.

<M> <SCI> <H1> Figure 2 :

 <H1>2 Figure 2: Examples of fully tagged sentences from Scielo, a bio-medical corpus. Tags appear as prefixes in the source sentence.

Figure 1 are reproduced on Figure 2 .

 2 Figure 1 are reproduced on Figure 2. In this example, tags are prefixed to the source sentence: the first tag (<M>) specifies the bio-medical domain, the second tag identifies the corpus (<SCI>), and the third tag corresponds to the subsection of each sentence.

1 . 2 . 3 .

 123 Medline and Scielo: These included both abstracts with and without subheadings, sometimes also missing a title line. There were a total of 189 subheadings in these corpora, including variations in case, word order and morphology. Examples subheadings include: Presenting Concerns of the Patient, Sources of Information, Novel finding, Study Selection, PURPOSE OF REVIEW, Purpose of Review, Purpose of review etc. Edp: These documents included abstracts with and without subheadings, with a majority having titles. Total 45 subheadings with variations in case and word order were identified such as: Case report, Observation, Subjects and Methods, Commentary, Pedagogical objectives, etc. Cochrane: This corpus contains very structured summaries with 10 distinct subheadings according to the Cochrane nomenclature. These included: abs selection criteria, abs search strategy, abs data collection, summary title etc.14

Figure 3 :

 3 Figure 3: Confusion matrices for the Scielo testset (EN-FR): from left to right pre0, pre1 and pre2. First row baseline, Second row fine-tuned systems

  There are various causes for these length differences, which are sometimes as subtle as a change in a determiner, see example 3 where translation (a) contains a collective use for the singular determiner, which is very appropriate in scientific texts; and example 4 where (a) uses a noun group while (b) uses a verbal form. One syntactic pattern however emerges from this analysis: outputs (a) tend to favour adjectives over prepositional groups (see examples 5 and 6). More examples of variations obtained with different tags are in apprendix Appendix B.7. Related Work Document-level MT. (DLMT) is loosely defined by contrast to "sentence-level" MT (SLMT) and includes any technique aiming to handle translation phenomena requiring a context extending beyond isolated sentences. The targeted phenomena 1 -src Socioeconomic and geographic inequalities in infant mortality, 1990-2005 trg (a) Inégalités socioéconomiques et géographiques de mortalité infantile, 1990-2005. trg (b) Inégalités socioéconomiques et géographiques en matière de mortalité infantile, 1990-2005. 2-src The global burden of diarrhoeal disease, as estimated from studies published between 1992 and 2000. trg (a) La charge globale des maladies diarrhéiques, telle qu'estimée à partir d'études publiées entre 1992 et 2000. trg (b) La charge mondiale des maladies diarrhéiques, estimée d'après des études publiées entre 1992 et 2000. 3 -src Homicide in children and adolescents: a case-control study in Recife, Brazil. trg (a) Homicide chez l'enfant et l'adolescent: étude cas-témoins à Recife, Brésil. trg (b) Homicide chez les enfants et les adolescents: étude cas-témoins à Recife, Brésil. 4 -src Managing the effect of TRIPS on availability of priority vaccines. trg (a) Gestion de l'effet des ADPIC sur la disponibilité des vaccins prioritaires. trg (b) Gérer l'effet de l'ADPIC sur la disponibilité des vaccins prioritaires. 5 -src Poverty, child undernutrition and morbidity: new evidence from India. trg (a) Pauvreté, sous-nutrition infantile et morbidité: nouvelles données indiennes. trg (b) Pauvreté, sous-nutrition infantile et morbidité: nouveaux éléments concernant l'Inde. 6 -src Circulating vaccine-derived polioviruses: current state of knowledge. trg (a) Poliovirus circulants d'origine vaccinale: état actuel des connaissances. trg (b) Poliovirus circulants dérivés du vaccin: état actuel des connaissances.

Table 1 :

 1 . For instance, Subjects and Methods was mapped onto Standardized section heading and the corresponding tags

	155	Material and Methods (<MaM>), Pedagogical objectives may be replaced with
		Objectives (<OBJ>), Novel finding with Results (<RES>), and so on. We also
		found it appropriate to merge some sections, such as Discussion, which was
		merged with Conclusion.
		Document level corpora (see § 4.1.3) were first retrieved in XML format and
		structured into sub-sections based on subheading information. Each section was

Table 2 :

 2 Data sources used in our study. Test sets of both directions with only document information were combined.

  Models are trained with mixed precision and a batch size of 4096 tokens on 4 V100 GPUs for 300k updates until convergence, where convergence is measured on the development set BLEU score. Final parameters are chosen based on the best score on the development set (Medline18, 19) and the corresponding scores for that checkpoint are reported on Medline20 test set. Translations were generated using a beam size of 4. For Trgtag systems tags were removed before

	14 Recommandations for writing Cochrane's systematic reviews are in https://training.
	cochrane.org/handbook/.
	15 https://github.com/biomedical-translation-corpora/corpora
	16 https://fairseq.readthedocs.io/en/latest/models.html

Table 3 :

 3 BLEU, TER and chrF scores computed on Medline20 and Scielo test sets

with and without tags, where we contrast tags in source (Srctag), reference target tags (Trgtag with forced decoding) and predicted target tags (Trgtag). +ft identifies the corresponding fine-tuned systems. Note that for TER, a decrease in score signifies improvement, while for BLEU and chrF increase denotes improvement. The best result in each column is in bold and best score for predicted vs. reference tags are italicized .

Table 4 :

 4 Ablation

experiment: Output from model trained with only document structure tags (a) and only domain tags (b).

Table 5 :

 5 Evaluating the difference in translation quality of sentences with correctly predicted sections vs. wrongly predicted sections (Domain, Corpus, Section). 000 indicates that the three tags are wrongly predicted, 111 that they are all correct.

Table 6 :

 6 Using specific or generic section tags

	Having structure tags in	Test set	EN-FR	FR-EN
	source slightly improves BLEU	Srctag	Spec. Gen.	Spec. Gen.
	for the fine-tuned systems on	Scielo	37.6	37.5	37.6	37.5
	Medline (+0.6 and +0.7 respec-	+ft	37.6	37.6	39.7	39.8
	tively for EN-FR and FR-EN).	Medline20	41.8	41.4	41.6	41.6
	All other comparisons show lit-	+ft	42.6	42.0	45.4	44.7
	tle, insgnificant variations. The	Trgtag	Spec. Gen.	Spec. Gen.
	same small variations are ob-	Scielo	37.5	37.5	36.8	36.7
	served for Trgtag systems,	+ft	37.7	37.6	38.8	38.6
		Medline20	41.0	41.4	40.7	40.8
		+ft	41.9	42.3	44.3	44.0

Table 7 :

 7 Precision, Recall and F1 Score on Scielo baseline and fine-tuned systems for systems for section prediction (pre2).

	Word	Occurrence in Sections
	conclusion_NNS	CON
	controlled_JJ	MAM
	but_CC	RES
	difference_NN	RES
	however_RB	INT	CON
	may_MD	INT	CON
	trial_NNS	MAM	RES
	evidence_NN	RES	CON
	people_NNS	H1	OBJ	MAM
	outcomes_NNS	OBJ	MAM	CON
	studies_NNS	MAM	RES	CON
	quality_NN	MAM	RES	CON

Table 8 :

 8 Word occurrence in sections from 50

	When we predict all tags
	(pre0), systems have precision
	above 0.95 for Conclusion, Ti-
	tle, Material and Methods, Ob-
	jectives and 0.85 for Results. In
	terms of accuracy, we see an im-
	provement from 0.32 to 0.50 for
	EN-FR after fine-tuning and a
	slight improvement for FR-EN.
	most frequent words only. POS tags were gener-
	ated using nltk pos tagger.

Table 9 :

 9 Translation of Scielo test sets produced with fine-tuned systems from English

into French. For each example, we display the source sentence, then the translations obtained respectively with the tags <H1> (a) and <ABS> (b). Note that for all these examples, the two French outputs are equally fluent and adequate. They illustrate small variations in wordings reflecting the style differences in titles vs. regular texts.

Table A .

 A 11: Precision, Recall and F1 Score on Scielo baseline and fine-tuned systems for systems with and without abstract in reference comparison

					EN-FR					FR-EN
	Prefix	Section		Baseline			Fine-tuned			Baseline		Fine-tuned
			P	R	F1	P	R	F1	P	R	F1	P	R	F1
		<CON>	0.96	0.34	0.51	0.88	0.53	0.66	0.99	0.32	0.49	0.88	0.53	0.67
		<H1>	1.00	0.55	0.71	1.00	0.80	0.89	0.97	0.46	0.62	1.00	0.86	0.92
	Pre2	<INT>	0.33	0.12	0.18	1.00	0.12	0.22	0.20	0.20	0.20	0.50	0.20	0.29
		<MaM>	0.93	0.65	0.77	0.89	0.82	0.85	0.86	0.85	0.85	0.88	0.87	0.87
		<OBJ>	1.00	0.71	0.83	0.99	1.00	0.76	1.00	0.57	0.72	0.99	0.74	0.85
		<RES>	0.81	0.75	0.78	0.85	0.82	0.83	0.88	0.72	0.79	0.87	0.81	0.84
		accuracy	-	-	0.61	-	-	0.75	-	-	0.61	-	-	0.77
		macro avg	0.72	0.45	0.54	0.80	0.55	0.62	0.70	0.45	0.53	0.73	0.57	0.63
		weighted avg	0.92	0.61	0.72	0.91	0.75	0.82	0.93	0.61	0.71	0.91	0.77	0.83
		<ABS>	0.48	0.82	0.61	0.62	0.78	0.70	0.48	0.85	0.61	0.64	0.76	0.70
		<CON>	0.85	0.40	0.54	0.68	0.53	0.60	0.97	0.32	0.48	0.70	0.54	0.61
		<H1>	1.00	0.09	0.17	1.00	0.80	0.89	0.96	0.46	0.62	0.99	0.86	0.92
	Pre2 with	<INT>	0.01	0.12	0.02	0.25	0.12	0.17	0.07	0.20	0.11	0.50	0.20	0.29
	<ABS>	<MaM>	0.74	0.80	0.77	0.81	0.82	0.82	0.76	0.85	0.80	0.78	0.87	0.83
		<OBJ>	0.99	0.71	0.83	0.99	0.76	0.86	1.00	0.57	0.72	0.99	0.74	0.84
		<RES>	0.74	0.82	0.78	0.77	0.82	0.79	0.80	0.72	0.76	0.77	0.81	0.79
		accuracy	-	-	0.64	-	-	0.76	-	-	0.67	-	-	0.77
		macro avg	0.69	0.54	0.53	0.73	0.66	0.69	0.72	0.57	0.59	0.77	0.68	0.71
		weighted avg	0.75	0.64	0.61	0.78	0.76	0.76	0.77	0.67	0.67	0.78	0.77	0.77

https://github.com/fyvo/WMT-Biomed-Test

https://github.com/berkmancenter/mediacloud-sentence-splitter

https://ufal.mff.cuni.cz/ufal_medical_corpus

https://github.com/fyvo/CochraneTranslations/

https://md.taus.net/corona

http://eval.covid19-mlia.eu/task3/

These test sets were sentence-aligned with in-house tools and can be retrieved from https://github.com/fyvo/WMT-Biomed-Test.

https://www.lissa.fr/dc/#env=lissa

https://www.istex.fr/

https://crtt.univ-lyon2.fr/les-corpus-medicaux-du-crtt-613310.kjsp

http://statmt.org/2021

https://pypi.org/project/sentence-splitter/

Indeed the BLEU scores between these condition is mostly due to a lower brevity penalty when using <H1>.

This tag combination is used for all out-of-domain sentences in the training corpus.

PATR is a subpart of the Ufal corpus amounting to around 22% of bio-medical data without substructure information, was thus tagged with <M> <PATR> <US>.

Acknowledgements

This work was made possible thanks to the Saclay-IA computing platform. It was granted access to the HPC resources of IDRIS under the allocation 2021-[AD011011580R1, AD011011270R1, AD011011717] made by GENCI. The second author was partly funded by the French "Agence Nationale de la Recherche" (ANR) under grant ANR-22-CE23-0033 / MaTOS.

Appendix A. Precision and Recall scores for Confusion matrices

Table B.12: Example outputs with varying tag combinations. Lexical differences are marked in blue whereas grammatical changes are given in orange. Invalid tags in a particular combination are marked in red. BLEU, TER and chrF scores are computed for whole test-set.

An interesting example is the <G> <PATR> 19 <ABS> combination (last 1055 row in TableB.12), where two tags are wrong and the Srctag system used the word "formation", "nearby" and "encoders" instead respectively of "training", "outreach" and "coders" which are generated with the reference tags.

Appendix