Daniele Sanacore

Nabil Hathout

FRAME-LIKE STRUCTURES FOR MORPHOSEMANTIC DESCRIPTION

Cet article présente un nouveau formalisme pour la représentation des relations morphosémantiques dans le lexique construit du français, dans le cadre de la morphologie paradigmatique. Ce travail alimentera Démonette, une base de données morphologiques de grande taille. Nous proposons dans cette étude d'utiliser des structures inspirées des frames de la Sémantique des Frames pour décrire globalement l'ensemble des relations de sens qui s'établissent au sein des familles dérivationnelles et pour mettre en évidence la nature paradigmatique du lexique construit.

INTRODUCTION

In this article we propose a new formalism for the representation of semantic relations in derivational families (i.e. sets of derivationally related words) in the French constructed lexicon. We call this formalism "Morphosemantic Frames" (MFs) because it is inspired by the frames of Frame Semantics [START_REF] Fillmore | Frame semantics and the nature of language[END_REF][START_REF] Fillmore | Frame semantics for text understanding[END_REF].

In this work we focus on the morphosemantic analysis of derivational families and their description. To our knowledge, derivational morphology lacks a formalism able to describe the morphosemantic relations between the lexemes of derivational families and that account for their regularity in the lexicon. Morphosemantic Frames aim at filling this gap. Moreover, this formalism could be used for other languages in order to develop a common framework for the morphosemantic representation of derivational relations in their constructed lexicon.

MFs focus on the morphosemantic relations between lexemes in the derivational families. The framework is designed to represent the structural regularity of the lexicon, by showing that the same groups of semantic relations connect the members of several families. MFs are precise, economic and have a high predictive power. They account for all the semantic relations between the lexemes contained in derivational families; they are easy to read and at the same time are applicable to large datasets; MFs allow for the prediction of missing lexemes and for the reconstruction of the derivational families.

Following [START_REF] Hathout | Démonette, a French derivational morphosemantic network[END_REF], we consider that the meaning of a given lexeme is described by the contribution of each of the derivational relations where it is involved. For instance, the examples in (1), ( 2) and (3) show how several derivational relations between a verb and its derivatives determine its argument structure.

(1) ronfler.V 'snore'-ronfleur.N 'person who snores' chanter.V 'sing 'chanteur.N 'singer' In (1), we deduce from the relation between the verb and its derived noun in -eur that it has an agent argument and the derived noun denotes an instrument or a person that performs the action (chanteur and ronfleur).

(2) périr.V 'perish', périssable.A 'perishable'

nager.V 'swim', nageable.A 'swimmable' modifier.V 'modify', modifiable.A 'modifiable'
In (2), we can deduce from the relation between the verbs and their adjectives in -able that these verbs have a patient (in the case of perishable or modifiable) or a locative modifier (in the case of nageable). We therefore see how different derivational relations of a verb contribute to the identification of the content of its argument structure.

(3) pêcher.V 'fish', pêchable.A 'fishable'

In some cases, as with pêchable in (3) there is more than one possible interpretation: pêchable can be associated with the patient of the process, with the place where the process occurs, with the time period where the action is performed or with the instruments that are used [START_REF] Hathout | Enquête sur les dérivés enable[END_REF], which are all internal arguments. The configuration in (4) presents both the derived noun in -eur and the derived adjective in -able. The access to the entire family in (4) allows us to get more insight about the argument structure of the base verb than single word pairs as in (1), ( 2) and (3). We therefore need to provide descriptions of the entire derivational families that go beyond individual base-derivative couples.

Morphosemantic analysis also involves ontological properties. Let us consider two derivational families: the one of banane 'banana' in (5) and the one of balai 'broom' in (6). Even though these two families have two nouns as their roots, they are rather different in the concepts that they express. The difference stems from the ontological nature of their root noun: banane denotes a fruit and yields nouns like bananier and bananeraie which indicate respectively the plant and the plantation that produces that fruit. The noun balai denotes an artifact and its family includes the derived agent ineur, which stands for a person that uses it. The verb balayer and the noun balayage denote the action of using the instrument. These examples show that the ontological categories determine the architecture of the derivational families. This is confirmed by other families of words derived from fruits in (7) and ( 8) and from artifacts in ( 9) and ( 10 The families in ( 7) and ( 8) contain the same concepts and relations between concepts as the family of banane: they both include the noun for the tree (cerisier and amandier) and the noun of the plantation that produces the fruit (ceriseraie and amanderaie). Families ( 9) and ( 10), contain the same concepts as the family of balai in (6): the human agent (chronométreur and brosseur) and the action in verbal (chronométrer and brosser) and nominal forms (chronométrage and brossage).

These examples show that the ontological categories of the lexemes play an important role in the analysis and description of derivational families and that they must be represented in the Morphosemantic Frames.

Another difference between the families in ( 5), ( 7) and ( 8), and in ( 6), ( 9) and ( 10) is that the latter are event-related (their concepts denote the participants of some action) and that the former are not, as outlined in (Fradin, in press). The concepts of event-related families are therefore also relevant for the syntax.

The remaining of the paper is structured as follows: Section 2 outlines the theoretical background of MFs: paradigmatic morphology. Section 3 introduces Frame Semantics, which has inspired our formalism. Section 4 shows how the constitutive elements of Frame Semantics are adapted to MFs and used for morphosemantic descriptions. Section 5 describes the structure of MFs and shows how we represent the families and their structures. Section 6 discusses the implementation of Morphosemantic Frames in the derivational lexicon Démonette.

PARADIGMATIC MORPHOLOGY

As we just saw, derivational families are a fundamental object in our analysis and we need a theoretical framework that provides tools to describe and handle them. Paradigmatic derivational morphology responds precisely to this need. Paradigmatic derivational morphology accounts for the numerous regularities that are present in the lexicon [START_REF] Van Marle | On the paradigmatic dimension of morphological creativity[END_REF][START_REF] Stump | A paradigm-based theory of morphosemantic mismatches[END_REF][START_REF] Bauer | Derivational paradigms[END_REF]Booij, 2008 inter alia) and organizes families into paradigms.

A derivational family is a set of derivationally related lexemes (Hathout, 2011) as the ones in (11). In the following, the term 'family' is also used for subsets of derivational families (Bonami & Strnadova, 2019). (11) présenter.v 'present', présentation.N 'presentation', présentateur.N 'presentermasc', présentable.A 'presentable' In paradigmatic morphology, a paradigm is a set of derivational families structured around the same oppositions of content (Bonami & Strnadova, 2019). MFs being morphosemantic descriptions, we will only be concerned with morphosemantic paradigms, which means that we are interested only in the concepts expressed by the lexemes present in derivational families and the semantic relations between these concepts. For example, Table 1 presents three families that we can include in the same morphosemantic paradigm since the lexemes that they contain express the same concepts, regardless of the fact that these concepts are realized by different formal means (conversion for vol, suffixation in -ion for présentation and in -age for lavage). These three families are all made up of a verb expressing an event, an agent noun and a noun denoting the action. The families in ( 5), ( 6), ( 7), ( 8), ( 9) and ( 10) can be grouped in two paradigms, each one being defined by the sets of semantic relations shared by (5, 7, 8) and (6, 9, 10).

Figure 1 represents the paradigm containing the families of banane, cerise and amande. The three families contain the same concepts and the same relations between these concepts. They are thus aligned under the same paradigm. Moreover, other families rooted in fruit nouns can be aligned under the same morphosemantic structure. Alignment is a key feature for paradigms because it gives them their predictive power, one of the features that we need for our formalism. Two pairs of morphologically related words (w1, w2) and (w3, w4) are aligned when the two pairs are linked by the same content relation (Bonami & Strnadova, 2019). When another noun is analysed as a fruit name (e.g. abricot 'apricot'), the predictive power of the paradigm allows us to deduce that its family also contains a name for the plant (abricotier 'apricot tree') and a name for the plantation that produces this fruit (abricoteraie 'apricot plantation') . 6), ( 9) and ( 10) with an instrument noun, the agent that makes use of that instrument and the action in nominal and verbal forms. As outlined in the introduction, the concepts expressed in this paradigm are relevant for the syntax-semantics interface. This is not the case in the Fruit Paradigm.

In both examples we have an abstract structure (the morphosemantic paradigm) represented at the top of each figure, and its realizations (the families) represented by graphs formed by continuous lines. In the derivational families, the lines represent morphological relations between the lexemes. As it will be seen in section 4, Morphosemantic Frames describe morphosemantic paradigms like those in Figures 1 and2.

FRAME SEMANTICS

This section presents Frame Semantics and the concepts borrowed and adapted for morphosemantic description. The theoretical framework of Frame Semantics is based on the assumption that words represent categories of experience and evoke in the mind of the listener representations of realworld situations, called frames. The best-known implementation of Frame Semantics is FrameNet (Baker et al., 1998;[START_REF] Ruppenhofer | FrameNet II: Extended theory and practice[END_REF], a resource designed for the semantic annotation of English texts. FrameNet has inspired the creation of equivalent resources in several other languages [START_REF] Lenci | Building an Italian FrameNet through Semi-automatic Corpus Analysis[END_REF]Candito et al., 2014 inter alia).

In FrameNet a frame is characterized by a number of frame elements (FEs), which correspond to situational roles that are characteristic of the situation described in the frame. For example, the ARREST frame has Authorities, Charges and Suspect as frame elements. The frame is first described by a sentence (the frame description) that globally defines the represented cognitive situation and how the frame elements are involved in it (12). It also contains partial descriptions that present the cognitive situation from the point of view of each frame element (13). In addition, some FEs are associated with ontological categories (e.g. Authorities is given the Sentient label). (12) ARREST: Authorities charge a Suspect, who is under suspicion of having committed a crime (the Charges), and take him/her into custody (13) AUTHORITIES (Sentient): The Authorities charge the Suspect with committing a crime and take him/her into custody.

CHARGES: Charges identifies a category within the legal system; it is the crime with which the Suspect is charged SUSPECT: The Suspect is taken into custody, under suspicion of having committed a crime.

Frames are also characterized by lexical units (LUs), that is lemmas that evoke the cognitive situation described by the frame. The LUs of the ARREST frame are given in ( 14). ( 14) apprehend.V, apprehension.N, arrest.N, arrest.V, book.V, bust.N, bust.V, collar.V, cop.V, nab.V, summons.V

They also contain examples of sentences that realize them in texts. Several sentences like those in ( 15), ( 16) and ( 17) may evoke the same cognitive structure (the same frame) and illustrate the use of different LUs. Corpus sentences thus the concrete realization of the frame.

(15)

The police ARRESTED Harry on charges of manslaughter.

(16)

The seven were BOOKED on marijuana possession charges.

(17) Almost four million people in England and Wales were SUMMONSED in the first half of this year for failing to pay.

To summarize, in FrameNet, frames are composed of frame elements, frame descriptions, lexical units and are realized by corpus sentences. In the semantic frames, frame elements and frame descriptions represent abstract structures instanciated by the lexical units and the corpus sentences. This abstract/concrete distinction is also key for Morphosemantic Frames and derivation families, as showed in section 4.

MORPHOSEMANTICS DESCRIPTION WITH FRAMES

In FrameNet different sentences containing different lexical units may be structured in the same way when they evoke the same cognitive structure (the same frame). An analogous configuration exists in morphosemantics: different families are structured on the same meaning oppositions and instanciate (evoke) the same morphosemantic paradigm, as shown in Figures 1 and2. Morphosemantic Frames represent abstract paradigms instanciated by derivational families (aligned in concrete paradigms).

MFs include 'Morphosemantic Glosses' (MGs) which describe how the concepts of the morphosemantic paradigm are related to one another. MGs are sentences similar to the frame descriptions in FrameNet. For instance, the abstract paradigm in Figure 1 can be described with the Morphosemantic Glosses in (18). difference with the semantic frames ). Table 3 summarizes the correspondences between frames in FrameNet and MFs. The Morphosemantic glosses used for the description of the morphosemantic abstract paradigms correspond to the frame descriptions in semantic frames. In these glosses, the concepts expressed in the paradigm are related and defined simultaneously like frame elements in frame descriptions. Finally, the derivational families correspond to the lexical units and the concrete interdefining glosses that relate the members of a derivational family correspond to the corpus sentences. They instanciate the morphosemantic glosses just as corpus sentences realize the semantic descriptions.

Semantic Frames

Morphosemantic frames frame description morphosemantic glosses frame elements concepts forming of the paradigm lexical unit derivational families corpus sentences concrete glosses

THE STRUCTURE OF THE MORPHOSEMANTIC FRAMES

The elements introduced in section 4 make up the Morphosemantic Frames. What do they look like? Morphosemantic frames are structured in three layers: the first contains the Morphosemantic Glosses and a label identifying the corresponding semantic relation, as in Table 4. The objective of this label is defining the semantic relation between the elements in the semantic gloss. The definition of a label set for this purpose is in progress and the proposed labels are temporary. The second layer (Table 6) describes the association between the frame elements and their ontological labels. The third (Table 7) lists the families that realize the morphosemantic paradigm described in the MF.

Morphosemantic glosses+Semantic Relation label Une Plantation produit des Fruits Producer -Product

Une Plantation est plantée de Plantes de la même espèce

Group -Entity

Un Fruit pousse sur une Plante Entity -Plant Table 4. -Morphosemantic glosses and semantic labels for the Fruit Paradigm

The second layer associates each concept with an ontological category (Table 6). In this article, we use the categories of the FrSemCor project (Barque et al., 2020) for the nouns. A sample is presented in Table 5. These categories are a modified version of the Unique Beginners for Nouns used by Wordnet [START_REF] Miller | WordNet: An electronic lexical database[END_REF] and present different levels of granularity. They allow a high precision as in the cases of fruit and tree but also include more generic categories. However, this ontology has not been created for morphosemantics. This is why we intend to adapt it to the morphosemantics of French in the next phases of the research. For example, different categories of people can be expressed morphologically, like the specialists of a given domain (économiste 'economist', philologue 'philologist'), the people that adhere to a given doctrine (Marxiste 'Marxist', fasciste 'Fascist') or the shopkeepers (poissonier 'fishmonger', fleuriste 'florist'). These words are tagged as Person with the current classification, which cannot be useful to predict the remainder of their families, e.g. poissonnerie in the family of poissonnier. In the Fruit Paradigm, the label of Plantation is Plant preceded by the modifier Group. The same modifier can be applied for all collective nouns: people, (e.g. foule 'crowd'), animals (e.g. colonie 'colony'), artifacts (e.g. armement 'weapons'), etc. The verb and adjective ontology have not been worked out yet. For the moment, verbs may have two labels (stative situation or dynamic situation) and all adjectives are tagged as modifier. Table 7 presents the families that realize the Fruit Paradigm of Figure 1. We present in Tables 8, 9 and 10 the three layers for the morphosemantic frame that represents the Artifact Paradigm in Figure 2. In Table 8, the morphosemantic glosses correspond to the content of

THE DESCRIPTION OF MORPHOSEMANTIC FRAMES IN DEMONETTE

The Démonette database is a derivational resource that will be fed by this work. In this section, we outline its constitutive elements and show how Morphosemantic Frames will be implemented in Démonette. Démonette [START_REF] Hathout | Démonette, a French derivational morphosemantic network[END_REF], 2016;[START_REF] Namer | Demonette2 -Une base de données dérivationnelles du français à grande échelle: premiers résultats[END_REF]) is a large-scale derivational database for French. It implements the theoretical model ParaDis [START_REF] Namer | ParaDis and Démonette-From Theory to Resources for Derivational Paradigms[END_REF], where lexemes are grouped into families, which in turn are organized into paradigms. The design of Démonette is based on a cumulative conception of meaning, as presented in the introduction: the meaning of a derived word is the combination of the semantic properties of all the derivational relations where the word is involved. Démonette has four objectives [START_REF] Hathout | Démonette, a French derivational morphosemantic network[END_REF]: (i) connect the members of a derivational family by direct and indirect relations; (ii) label each relation semantically; (iii) provide the words in the database with phonological, morphological and semantic information; (iv) provide similar definitions to the words that are part of the same paradigms. Démonette is fed by several existing resources of different nature [START_REF] Namer | Demonette2 -Une base de données dérivationnelles du français à grande échelle: premiers résultats[END_REF]. In the Démonette database, an entry corresponds to a binary relation between two derivationally related lexemes lex1 and lex2 (e.g. laveur 'washer' and laver 'wash'). Contrarily to the relations in Morphosemantic Frames, in Démonette relations are oriented (e.g. laver → laveur and laveur →laver). The table that realizes the database includes the ontological types of lex1 and lex2 and the semantic relation between lex1 and lex2 as illustrated in On the other hand, the derivational database contains elements that can be used to reconstitute Morphosemantic Frames. Démonette provides a numerical identifier of the family of the lexemes, which means that the families can be recovered from the database. Démonette uses the same ontological labels as the MFs. Démonette also describes the semantic relations that hold between the pairs of lexemes, as shown in the fourth column in Table 11 (however, the label is single and not double) and gives an abstract definition in the last column. Finally, the semantic description is complemented by two glosses, a concrete and an abstract paraphrases which inter-define the two lexemes [START_REF] Namer | Demonette2 -Une base de données dérivationnelles du français à grande échelle: premiers résultats[END_REF] These sentences have the same structure as the Morphosemantic Glosses. Morphosemantic Glosses could thus be added to the database in order to fill in the semantic relations that are included in the same paradigm.

We can therefore fill in Démonette with the Morphosemantic Frames in a quite straightforward way. On the other hand, some information already in the database can be used to create the MFs: the binary relations between lexemes, the family of each binary relation in the database, the ontological labels and the concrete and abstract paraphrases.

CONCLUSION

In this article we presented our formalism for the representation of morphosemantic lexicon: the Morphosemantic Frames. MFs provide a semantic description of the derivational relations and an ontological categorization of the lexemes. We showed how derivational families instanciate the semantic paradigms described in the Morphosemantic Frames and highlighted the parallelism between the MFs and the semantic frames of FrameNet.

The adaptation of the constitutive elements of frame semantics to our task has been proven to be solid and capable of bringing out structural regularities in the lexicon. Moreover, we showed the main advantages of MFs with some examples: their readability, the fact that they can be applied on large-scale data and their predictive power. We described the type of morphosemantic information that we need to convey in our MFs and how we express it (abstract glosses, ontological labels and relation labels).

In the near future, our objective is to develop a large enough number of frames like those proposed in this article and describe on a large scale the data contained in Démonette. On the longer term, we aim at something like a FrameNet for morphosemantics (in a prototypical form) and its implementation in Démonette in a first moment. Fine-grained description would also help us deal with lexical gaps, a problem that we will need to be addressed in the frames.

Another important question we intend to address concerns the ontological categories needed for morphosemantics and the labelling of the semantic relations. The ontology must provide categories adapted for verbs and adjectives. Our approach will be bottom-up, starting from the direct observation of data.

  'banana', bananier.N 'banana tree', bananeraie.N 'banana plantation', (6) balai.N 'broom', balayeur.N 'male sweeper', balayer.V 'sweep', balayage.N 'sweeping'

Figure 1 .

 1 Figure 1. -Fruit paradigm

Une

  Plantation ne comporte que des Plantes de la même espèce 'A Plantation only contains Plants of the same species'

Families

RÉFÉRENCES

  BAKER, C. F., FILLMORE, C. J., & LOWE, J. B. (1998). The Berkeley Framenet project. In Proceedings of the 17th international conference on Computational linguistics-Volume 1 86-90. Association for Computational Linguistics. BARQUE L., HAAS P., HUYGHE, R., TRIBOUT, D., CANDITO, M., CRABBÉ, B., & SEGONNE, V. (2020). FrSemCor: Annotating a French corpus with supersenses. In LREC-2020.

Table 3

 3

. -Correspondences between frames and MFs

Table 5 .

 5 -Sample of the noun ontology in FrSemCor

	Entit
	y
	Living & animate_entity
	Animal (e.POS+Ontology
	Fruit.N	Fruit
	Plante.N	Plant
	Plantation.N	Group x Plant

g. fourmilier 'ant-bear') Person (e.g. défenseur 'defender') Non_Animate_entity Food (e.g. orangeade 'orangeade') Substance (e.g. chlorure 'chlorure') Artifact (e.g. aspirateur 'vacuum cleaner') Plant (e.g. bananier 'banana tree')

Table 6 .

 6 -Frame Elements and ontological labels for the Fruit paradigm

Table 7 .

 7 -Families that realize the MF of the Fruit paradigm

Table 8 .

 8 Table 2, and semantic relations in the paradigm are labelled with general categories like Agent -Event, Instrument -Event, Agent -Instrument and Synonymy. -Morphosemantic glosses and semantic labels for the Artifact ParadigmIn Table9, ontological classes assigned to the concepts of the Artifact paradigm correspond to the more specific values in the UB hierarchy: instruments are Artifacts; agents are human beings (labelled with the category Person) and events belong to the class of Dynamic situation.

	Morphosemantic glosses+Semantic Relation label
	Un Agent utilise un Instrument	Agent -Instrument
	Un Agent réalise l'acte de Événement	Agent -Event
	Un Agent realise l'Action	Agent -Event
	Un Instrument est utilisé pour réaliser une Action	Instrument -Event
	Un Instrument est utilisé pour Événement	Instrument -Event
	L'Action est l'acte de Événement	Synonymy
	POS+Ontology
	Instrument.N	Artifact
	Agent.N	Person
	Événement.V	Dynamic situation
	Action.N	Act

Table 9 .

 9 -Frame Elements and ontological labels for Artifact paradigm

Table 10

 10 contains the families of Figure2; they realize the Artifact Paradigm.

			Families
	Instrument.N	Evenément.V	Agent.N	Action.N
	balai	balayer	balayeur	balayage
	chronomètre	chronométrer	chronométreur	chronométrage
	brosse	brosser	brosseur	brossage
	Table 10. -Families that realize the MF for Artifact paradigm

Table 11 .

 11

	Lex1-Lex2	Onto1	Onto2	Sem. Rel.	Abstract
					Definition
	bananeraie-	Group x	Plant	collection	Un Lex2 est un
	bananier	Plant			ensemble de Lex1
					'A Lex2 is a
					collection of Lex1'
	balayer-balayage	dyn-situation act	synonymy	Lex1 c'est faire
					Lex2
					'Lex1 is doing
					Lex2'
		Table 11. -Semantic description in Démonette

  as shown in Table12. Table 12. -Concrete and abstract paraphrases in Démonette

	(Lex1-Lex2)	Concrete paraphrase	Abstract paraphrase
	bananeraie-	Une bananeraie ne comporte	Une Group x Plant.N ne
	bananier	que des bananiers	comporte que des Plant.N
	balayer-balayage	Le balayage est l'acte de	act.N est l'acte de
		balayer	sit_dyn.V

Un Fruit pousse sur une Plante 'A Fruit grows on a Plant'

The frame elements in the Morphosemantic Glosses can be instanciated with the lexemes of the families of the paradigm. ( 19) shows how these lexemes of the family of banane fit in the structure of the Morphosemantic Glosses. We can do the same for all the other families aligned under the morphosemantic paradigm in Figure 1 (e.g. cerise or amande) and to the artifact paradigm in Figure 2. Its morphosemantic glosses and concrete sentences are presented in Table 2. In order for these sentences to sound natural and easy to read, we limit the number of frame elements in the MGs to two or three (this is one


Jérôme Dockès

Gaël Varoquaux

Jean-Baptiste Poline

Preventing dataset shift from breaking machine-learning biomarkers

Machine learning brings the hope of finding new biomarkers extracted from cohorts with rich biomedical measurements. A good biomarker is one that gives reliable detection of the corresponding condition. However, biomarkers are often extracted from a cohort that differs from the target population. Such a mismatch, known as a dataset shift, can undermine the application of the biomarker to new individuals. Dataset shifts are frequent in biomedical research, e.g. because of recruitment biases. When a dataset shift occurs, standard machine-learning techniques do not suffice to extract and validate biomarkers. This article provides an overview of when and how dataset shifts breaks machine-learning extracted biomarkers, as well as detection and correction strategies.

Introduction: dataset shift breaks learned biomarkers

Biomarkers are measurements that provide information about a medical condition or physiological state [START_REF] Strimbu | What are biomarkers?[END_REF]. For example, the presence of an antibody may indicate an infection; a complex combination of features extracted from a medical image can help assess the evolution of a tumor. Biomarkers are important for diagnosis, prognosis, and treatment or risk assessments.

Complex biomedical measures may carry precious medical information, as with histopathological images or genome sequencing of biopsy samples in oncology. Identifying quantitative biomarkers from these requires sophisticated sta-tistical analysis. With large datasets becoming accessible, supervised machine learning provides new promises by optimizing the information extracted to relate to a specific output variable of interest, such as a cancer diagnosis [START_REF] Andreu-Perez | Big data for health[END_REF][START_REF] Faust | Deep learning for healthcare applications based on physiological signals: A review[END_REF][START_REF] Deo | Machine learning in medicine[END_REF]. These methods, cornerstones of artificial intelligence, are starting to appear in clinical practice: a machine-learning based radiological tool for breast-cancer diagnosis has recently been approved by the FDA1 .

Can such predictive biomarkers, extracted through complex data processing, be safely used in clinical practice, beyond the initial research settings? One risk is the potential mismatch, or dataset shift, between the distribution of the individuals used to estimate this statistical link and that of the target population that should benefit from the biomarker. In this case, the extracted associations may not apply to the target population [START_REF] Kakarmath | Best practices for authors of healthcare-related artificial intelligence manuscripts[END_REF]. Computer aided diagnostic of thoracic diseases from X-ray images has indeed been shown to be unreliable for individuals of a given sex if built from a cohort over-representing the other sex [START_REF] Larrazabal | Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis[END_REF]. More generally, machine-learning systems may fail on data from different imaging devices, hospitals, populations with a different age distribution, etc. . Dataset biases are in fact frequent in medicine. For instance selection biases -eg due to volunteering self-selection, non-response, dropout...- [START_REF] Rothman | Epidemiology: an introduction[END_REF][START_REF] Tripepi | Selection bias and information bias in clinical research[END_REF] may cause cohorts to capture only a small range of possible patients and disease manifestations in the presence of spectrum effects [START_REF] Ransohoff | Problems of spectrum and bias in evaluating the efficacy of diagnostic tests[END_REF]Feinstein, 1978, Mulherin and[START_REF] Mulherin | Spectrum bias or spectrum effect? subgroup variation in diagnostic test evaluation[END_REF]. Dataset shift or dataset bias can cause systematic errors that cannot be fixed by acquiring larger datasets and require specific methodological care.

In this article, we consider predictive biomarkers identified with supervised machine learning. We characterize the problem of dataset shift, show how it can hinder the use of machine learning for health applications [START_REF] Woo | Building better biomarkers: brain models in translational neuroimaging[END_REF][START_REF] Wynants | Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal[END_REF], and provide mitigation strategies.

A primer on machine learning for biomarkers 2.1 Empirical Risk Minimization

Let us first introduce the principles of machine learning used to identify biomarkers. Supervised learning captures, from observed data, the link between a set of input measures (features) X and an output (e.g. a condition) Y : for example the relation between the absorption spectrum of oral mucosa and blood glucose concentration [START_REF] Kasahara | Noninvasive glucose monitoring using mid-infrared absorption spectroscopy based on a few wavenumbers[END_REF]. A supervised learning algorithm finds a function f such that f (X) is as close as possible to the output Y . Following machine-learning terminology, we call the system's best guess f (X) for a value X a prediction, even when it does not concern a measurement in the future.

Empirical Risk Minimization, central to machine learning, uses a loss function L to measure how far a prediction f (X) is from the true value Y , for example the squared difference:

L(Y, f (X)) = (Y -f (X)) 2 .
(1)

The goal is to find a function f that has a small risk, which is the expected loss on the true distribution of X and Y , i.e. on unseen individuals. The true risk cannot be computed in practice: it would require having seen all possible patients, the true distribution of patients. The empirical risk is used instead: the average error over available examples,

R(f ) = 1 n n i=1 L(y i , f (x i )) , (2)
where {(x i , y i ) , i = 1, . . . , n} are available (X, Y ) data, called training examples. The statistical link of interest is then approximated by choosing f within a family of candidate functions as the one that minimizes the empirical risk R(f ).

The crucial assumption underlying this very popular approach is that the prediction function f will then be applied to individuals drawn from the same population as the training examples {x i , y i }. It can be important to distinguish the source data, used to fit and evaluate a machine-learning model (e.g. a dataset collected for research), from the target data, on which predictions are meant to be used for clinical applications (e.g. new visitors of a hospital). Indeed, if the training examples are not representative of the target population -if there is a dataset shift -the empirical risk is a poor estimate of the expected error, and f will not perform well on individuals from the target population.

Evaluation: Independent test set and cross-validation

Once a model has been estimated from training examples, measuring its error on these same individuals results in a (sometimes wildly) optimistic estimate of the expected error on unseen individuals (Friedman et al. [2001, Sec. 7.4], Poldrack et al. [2020, Sec. 1, "Association vs Prediction"]). Indeed, predictors chosen from a rich family of functions are very flexible and can learn rules that fit tightly the training examples but fail to generalize to new individuals. This is called overfitting.

To obtain valid estimates of the expected performance on new data, the error is measured on an independent sample held out during training, called the test set. The most common approach to obtain such a test set is to randomly split the available data. This process is usually repeated with several splits, a procedure called cross-validation [START_REF] Arlot | A survey of cross-validation procedures for model selection[END_REF], Friedman et al., 2001, Sec. 7].

When training and test examples are chosen uniformly from the same sample, they are drawn from the same distribution (i.e. the same population): there is no dataset shift. Some studies also measure the error on an independent dataset [e.g. [START_REF] Beck | Systematic analysis of breast cancer morphology uncovers stromal features associated with survival[END_REF][START_REF] Jin | reproducible, and neuroscientifically interpretable imaging biomarkers for alzheimer's disease[END_REF]. This helps establishing external validity, assessing whether the predictor will perform well outside of the dataset used to define it [START_REF] Bleeker | External validation is necessary in prediction research:: A clinical example[END_REF]. Unfortunately, the biases in participant recruitment may be similar in independently collected datasets. For example if patients with severe symptoms are difficult to recruit, this is likely to distort all datasets similarly. Testing on a dataset collected independently is therefore a useful check, but no silver bullet to rule out dataset shift issues.

False solutions to tackling dataset shift

We now discuss some misconceptions and confusions with problems not directly related to dataset shift.

"Deconfounding" does not correct dataset shift for predictive models Dataset shift is sometimes confused with the notion of confounding, as both settings arise from an undesired effect in the data. Confounding comes from causal analysis, estimating the effect of a treatment -an intervention, sometimes fictional-on an outcome.

A confounder is a third variable -for example age, or a comorbidity-that influences both the treatment and the outcome. It can produce a non-causal association between the two [See Hernán and Robins, 2020, Chap. 7, for a precise definition]. However, the machine-learning methods we consider here capture statistical associations, but do not target causal effects. Indeed, for biomarkers, the association itself is interesting, whether causal or not. Elevated body temperature may be the consequence of a condition, but also cause a disorder. It is a clinically useful measure in both settings. Tools for causal analysis are not all useful for prediction, as pointed out by seminal textbooks: "if the goal of the data analysis is purely predictive, no adjustment for confounding is necessary [...] the concept of confounding does not even apply." [Hernán and Robins, 2020, Sec. 18.1], or [START_REF] Pearl | The seven tools of causal inference, with reflections on machine learning[END_REF]. In prediction settings, applying procedures meant to adjust for confounding generally degrades prediction performance without solving the dataset shift issue. Figure 1 demonstrates the detrimental effect of "deconfounding" on simulated data: while the target population is shifted due to a different age distribution, removing the effect of age also removes the separation between the two outcomes of interest. The same behavior is visible on real epidemiologic data with age shifts, such as predicting the smoking status of participants in the UKBiobank study [START_REF] Sudlow | Uk biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age[END_REF], as shown in Figure 2. Drawing training and testing samples with different age distributions highlights the effect of these age shifts on prediction performance (see Appendix B for details on the procedure). For a given learner and test population, training on a different population degrades prediction. For example, predictions on the old population are degraded when the model is trained on the young population. A flexible model (Gradient Boost-ing) outperforms the linear model with or without dataset shift. "Regressing out" the age (as in the second column of Figure 1, "+ regress-out" strategy in Figure 2) degrades the predictions in all configurations.

For both illustrations on simulated and real data (Figure 1 and 2), we also demonstrate an approach suitable for predictive models: reweighting training examples giving more importance to those more likely in the test population. This approach improves the predictions of the overconstrained (misspecified) linear model in the presence of dataset shift, but degrades the predictions of the powerful learner. The non-linear model already captures the correct separation for both young and old individuals, thus reweighting examples does not bring any benefit but only increases the variance of the empirical risk. A more detailed discussion of this approach, called importance weighting, is provided in Section 5.

Training examples should not be selected to be homogeneous To obtain valid predictive models that perform well beyond the training sample, it is crucial to collect datasets that represent the whole population and reflect its diversity as much as possible [START_REF] Kakarmath | Best practices for authors of healthcare-related artificial intelligence manuscripts[END_REF][START_REF] England | Artificial intelligence for medical image analysis: a guide for authors and reviewers[END_REF][START_REF] O'neil | Weapons of math destruction: How big data increases inequality and threatens democracy[END_REF]. Yet clinical research often emphasizes the opposite: very homogeneous datasets and carefully selected participants. While this may help reduce variance and improve statistical testing, it degrades prediction performance and fairness. In other words, the machine-learning system may perform worse for segments of the population that are under-represented in the dataset, resulting in uneven quality of care if it is deployed in clinical settings. Therefore in predictive settings, where the goal is machine-learning models that generalize well, large and diverse datasets are desirable.

Simpler models are not less sensitive to dataset shift Often, flexible models can be more robust to dataset shifts, and thus generalize better, than linear models [START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF], as seen in Figures 1 and2. Indeed, an over-constrained (ill-specified) model may only fit well a restricted region of the feature space, and its performance can degrade if the distribution of inputs changes, even if the relation to the output stays the same (i.e. when covariate shift occurs, Section 7.1).

Dataset shift does not call for simpler models as it is not a small-sample issue. Collecting more data from the same The task is to classify patients (orange) from healthy controls (blue), using 2-dimensional features. Age, indicated by the shade of gray, influences both the features and the probability of disease. Left: generative process for the simulated data. Age influences both the target Y and the features X, and Y also has an effect on X. Between the source and target datasets, the distribution of age changes. The two arrows point towards increasing age and represent the Healthy and Diseased populations, corresponding to the orange and blue clouds of points in the right panel. The grayscale gradient in the arrows represents the increasing age of the individuals (older individuals correspond to a darker shade). Throughout their life, individuals can jump from the Healthy trajectory to the Diseased trajectory, which is slightly offset in this 2-dimensional feature space. As age increases, the prevalence of the disease increases, hence the Healthy trajectory contains more individuals of young ages (its wide end), and less at older ages (its narrow end) -and vice-versa for the Diseased trajectory. Right: predictive models In the target data (bottom row), the age distribution is shifted: individuals tend to be older. Elderly are indeed often less likely to participate in clinical studies [START_REF] Heiat | Representation of the elderly, women, and minorities in heart failure clinical trials[END_REF]. First column: no correction is applied. As the situation is close to a covariate shift (Section 7.1), a powerful learner (RBF-SVM) generalizes well to the target data. An over-constrained model -Linear-SVM -generalizes poorly.

Second column: wrong approach. To remove associations with age, features are replaced by the residuals after regressing them on age. This destroys the signal and results in poor performance for both models and datasets. Third column: Samples are weighted to give more importance to those more likely in the target distribution. Small circles indicate younger individuals, with less influence on the classifier estimation. This reweighting improves prediction for the linear model on the older population.

sources will not correct systematic dataset bias.

Preferential sample selection: a common source of shift

In 2017, competitors in the million-dollar-prize data science bowl used machine learning to predict if individuals would be diagnosed with lung cancer within one year, based on a CT scan. Assuming that the winning model achieves satisfying accuracy on left-out examples from this dataset, is it ready to be deployed in hospitals? Most likely not. Selection criteria may make this dataset not representative of the potential lung cancer patients general population.

Selected participants verified many criteria, including being a smoker and not having recent medical problems such as pneumonia. How would the winning predictor perform on a more diverse population? For example, another disease could present features that the classifier could mistakenly take for signs of lung cancer. Beyond explicit selection criteria, many factors such as age, ethnicity, or socioeconomic status influence participation in biomedical studies [START_REF] Henrich | Most people are not weird[END_REF][START_REF] Murthy | Participation in cancer clinical trials: race-, sex-, and age-based disparities[END_REF][START_REF] Heiat | Representation of the elderly, women, and minorities in heart failure clinical trials[END_REF][START_REF] Chastain | Racial disproportionality in covid clinical trials[END_REF] Predicting smoking status in the UKBiobank (10-fold CV, n train = 90K, n test = 9K) Figure 2. Predicting the smoking status of UKBiobank participants. Different predictive models are trained on 90K UKBiobank participants and tested on 9K participants with a possibly shifted age distribution. "young → old" means the training set was drawn from a younger sample than the testing set. Models perform better when trained on a sample drawn from the same population as the testing set. Reweighting examples that are more likely in the test distribution ("+ reweighting" strategy, known as Importance Weighting, Section 5) alleviates the issue for the simple linear model, but is detrimental for the Gradient Boosting. Regressing out the age ("+ regress-out" strategy) is a bad idea and degrades prediction performance in all configurations.

[Oakden- [START_REF] Oakden-Rayner | Hidden stratification causes clinically meaningful failures in machine learning for medical imaging[END_REF][START_REF] Gianfrancesco | Potential biases in machine learning algorithms using electronic health record data[END_REF][START_REF] Barocas | Fairness and Machine Learning[END_REF][START_REF] Abbasi-Sureshjani | Risk of training diagnostic algorithms on data with demographic bias[END_REF][START_REF] Cirillo | Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare[END_REF].

The examples above are instances of preferential selection, which happens when members of the population of interest do not have equal probabilities of being included in the source dataset: the selection S is not independent of (X, Y ). Preferential sample selection is ubiquitous and cannot always be prevented by careful study design [START_REF] Bareinboim | Controlling selection bias in causal inference[END_REF]. It is therefore a major challenge to the identification of reliable and fair biomarkers. Beyond preferential sample selection, there are many other sources of dataset shifts, e.g. population changes over time, interventions such as the introduction of new diagnostic codes in Electronic Health Records [START_REF] Sáez | Ehrtemporalvariability: delineating temporal dataset shifts in electronic health records[END_REF], and the use of different acquisition devices.

The selection mechanism influences the type of dataset shift

The correction for a dataset shift depends on the nature of this shift, characterized by which and how distributions are modified [START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF]. Knowledge of the mechanism producing the dataset shift helps formulate hypotheses about distributions that remain unchanged in the target data [START_REF] Schölkopf | On causal and anticausal learning[END_REF], Peters et al., 2017, Chap. 5].

Figure 3 illustrates this process with a simulated example of preferential sample selection. We consider the problem of predicting the volume Y of a tumor from features X extracted from contrast CT images. These features can be influenced not only by the tumor size, but also by the dosage of a contrast agent M . The first panel of Figure 3 shows a selection of data independent of the image and tumor volume: there is no dataset shift. In the second panel, selection depends on the CT image itself (for example images with a low signal-to-noise ratio are discarded).

As [START_REF] Pearl | Causal inference in statistics: A primer[END_REF]. Y is the lesion volume to be predicted (i.e. the output). M are the imaging parameters, e.g. contrast agent dosage. X is the image, and depends both on Y and M (in this toy example X is computed as X := Y +M + , where is additive noise. S indicates that data is selected to enter the source dataset (orange points) or not (blue points). The symbol ⊥ ⊥ means independence between variables. Preferentially selecting samples results in a dataset shift (middle and bottom row). Depending on whether Y ⊥ ⊥ S | X, the conditional distribution of Y | X -here lesion volume given the image -estimated on the selected data may be biased or not.

is available, it may be possible to leverage it in order to improve robustness to dataset shift [e.g. [START_REF] Subbaswamy | Preventing failures due to dataset shift: Learning predictive models that transport[END_REF].

5 Importance weighting: a generic tool against dataset shift

Importance weighting is a simple approach to dataset shift that applies to many situations and can be easy to implement.

Dataset shift occurs when the joint distribution of the features and outputs is different in the source (data used to fit the machine-learning model) and in the target data. Informally, importance weighting consists in reweighting the available data to create a pseudo-sample that follows the same distribution as the target population.

To do so, examples are reweighted by their importance weights -the ratio of their likelihood in target data over source data. Examples that are rare in the source data but are likely in the target data are more relevant and therefore receive higher weights. A related approach is importance sampling -resampling the training data according to the importance weights. Many statistical learning algorithmsincluding Support Vector Machines, decision trees, random forests, neural networks -naturally support weighting the training examples. Therefore, the challenge relies mostly in the estimation of the appropriate sample weights and the learning algorithm itself does not need to be modified.

To successfully use importance weighting, no part of the target distribution should be completely unseen. For example, if sex (among other features) is used to predict heart failure and the dataset only includes men, importance weighting cannot transform this dataset and make its sex distribution similar to that of the general population (Figure 4). Conversely, the source distribution may be broader than the target distribution (as seen for example in Figure 1).

Importance weights can also be applied to validation examples, which may produce a more accurate estimation of generalization error on target data.

Importance weighting is a well-known approach and an important body of literature focuses on its application and the estimation of importance weights. It is illustrated on small datasets for the prediction of breast cancer in [START_REF] Dudík | Correcting sample selection bias in maximum entropy density estimation[END_REF] and heart disease in [START_REF] Kouw | A review of domain adaptation without target labels[END_REF]. However, it cannot always be applied: some knowledge of the target distribution is required, and the source distribution must cover its support. Moreover, importance weighting can increase the variance of the empirical risk estimate, and thus sometimes degrades performance -as seen in Figure 2. It is therefore a straightforward and popular approach to consider, but not a complete solution. It is particularly beneficial when using a simple learning model which cannot capture the full complexity of the data, such as the linear models in Figure 1. Indeed, simple models are often prefered in biomedical applications because they are easy to interpret and audit.

Men

In Appendix A, we provide a more precise definition of the importance weights, as well as an overview of how they can be estimated and used.

Other approaches to dataset shift

Beyond importance weighting, many other solutions to dataset shift have been proposed. They are typically more difficult to implement, as they require adapting or desiging new learning algorithms. However, they may be more effective, or applicable when information about the target distribution is lacking. We summarize a few of these approaches here. A more systematic review can be found in [START_REF] Kouw | A review of domain adaptation without target labels[END_REF]. [START_REF] Weiss | A survey of transfer learning[END_REF] and [START_REF] Pan | A survey on transfer learning[END_REF] give systematic reviews of transfer learning (a wider family of learning problems which includes dataset shift).

The most obvious solution is to do nothing -ignoring the dataset shift. This approach should be included as a baseline when testing on a sample of target data -which is a prerequisite to clinical use of a biomarker [START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF][START_REF] Woo | Building better biomarkers: brain models in translational neuroimaging[END_REF]. With flexible models, this is a strong baseline that can outperform importance weighting, as in the right panel of Figure 2.

Another approach is to learn representationstransformations of the signal-that are invariant to the shift [START_REF] Achille | Emergence of invariance and disentanglement in deep representations[END_REF]. Some deep-learning methods strive to extract features that are predictive of the target while having similar distributions in the source and target domains [e.g. [START_REF] Long | Learning transferable features with deep adaptation networks[END_REF], or while preventing an adversary to distinguish source and target data ["domain-adversarial" learning, e.g. [START_REF] Tzeng | Adversarial discriminative domain adaptation[END_REF]. When considering such methods, one must be aware of the fallacy shown in Figure 1: making the features invariant to the effect driving the dataset shift can remove valuable signal if this effect is not independent of the outcome of interest.

It may also be possible to explicitly model the mapping from source to target domains, e.g. by training a neural network to translate images from one modality or imaging device to another, or by relying on optimal transport [START_REF] Courty | Optimal transport for domain adaptation[END_REF].

Finally, synthetic data augmentation sometimes helpsrelying on known invariances e.g. for images by applying affine transformations, resampling, etc. . or with learned generative models [e.g. [START_REF] Antoniou | Data augmentation generative adversarial networks[END_REF].

Performance heterogeneity and fairness

It can be useful not to target a specific population, but rather find a predictor robust to certain dataset shifts. Distributionally robust optimization tackles this goal by defining an ambiguity, or uncertainty set -a set of distributions to which the target distribution might belongthen minimizing the worse risk across all distributions in this set [see Rahimian and Mehrotra, 2019, for a review]. The uncertainty set is often chosen centered on the empirical (source) distribution for some divergence between distributions. Popular choices for this divergence are the Wasserstein distance, f -divergences (e.g. the KL divergence) [START_REF] Duchi | Learning models with uniform performance via distributionally robust optimization[END_REF], and the Maximum Mean Discrepancy [START_REF] Zhu | Kernel distributionally robust optimization[END_REF]. If information about the target distribution is available, it can be incorportated in the definition of the uncertainty set. An approach related to robust optimization is to strive not only to minimize the empirical loss L(Y, f (X)) but also its variance [START_REF] Maurer | Empirical bernstein bounds and sample variance penalization[END_REF], [START_REF] Namkoong | Variance-based regularization with convex objectives[END_REF].

It is also useful to assess model performance across values of demographic variables such as age, socioeconomic status or ethnicity. Indeed, a good overall prediction performance can be achieved despite a poor performance on a minority group. Ensuring that a predictor performs well for all subpopulations reduces sensitivity to potential shifts in demographics and is essential to ensure fairness [START_REF] Abbasi-Sureshjani | Risk of training diagnostic algorithms on data with demographic bias[END_REF]. For instance, there is a risk that machine-learning analysis of dermoscopic images underdiagnoses malignant moles on skin tones that are typically under-represented in the training set [START_REF] Adamson | Machine learning and health care disparities in dermatology[END_REF]. Fairness is especially relevant when the model output could be used to grant access to some treatment. As similar issues arise in many applications of machine learning, there is a growing literature on fairness [see e.g. Barocas et al., 2019, for an overview]. For instance, [START_REF] Duchi | Learning models with uniform performance via distributionally robust optimization[END_REF] show that distributionally robust optimization can help performance on under-represented subpopulations.

Multi-site datasets

Often datasets are collected across several sites or hospitals, or with different measurement devices. This heterogeneity provides an opportunity to train models that generalize to unseen sites or devices. Some studies attempt to remove site effects by regressing all features on the site indicator variable. For the same reasons that regressing out age is detrimental in Figure 1, this strategy often gives worse generalization across sites. Data harmonization, such as compensating differences across measurement devices, is crucial, but remains very difficult and cannot correct these differences perfectly [START_REF] Glocker | Machine learning with multi-site imaging data: An empirical study on the impact of scanner effects[END_REF]. Removing too much inter-site variance can lead to loss of informative signal. Rather, it is important to model it well, accounting for the two sources of variance, across participants and across sites. A good model strives to yield good results on all sites. One solution is to adapt ideas from robust optimization: on data drawn from different distributions (e.g. from several sites), [START_REF] Krueger | Out-of-distribution generalization via risk extrapolation (rex)[END_REF] show the benefits of minimizing the empirical risk on the worse site or adding penalties on the variance of the loss across sites.

Measures of prediction performance should aggregate scores at the site level (not pooling all individuals), and check the variance across sites and the performance on the worse site. Cross-validation schemes should hold out entire sites [START_REF] Woo | Building better biomarkers: brain models in translational neuroimaging[END_REF][START_REF] Little | Using and understanding cross-validation strategies. perspectives on saeb et al[END_REF].

Special cases of dataset shift

Categorizing dataset shift helps finding the best approach to tackle it [START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF], [START_REF] Moreno-Torres | A unifying view on dataset shift in classification[END_REF]. We summarize two frequently-met scenarios that are easier to handle than the general case and can call for different adjustments: covariate shift (Section 7.1) and prior probability shift (Section 7.2).

Covariate shift

Covariate shift occurs when the marginal distribution of X changes between the source and target datasets (i.e. p t (x) = p s (x)), but P (Y | X) stays the same. This happens for example in the second scenario in Figure 3, where sample selection based on X (but not Y ) changes the distribution of the inputs. If the model is correctly specified, an estimator trained with uniform weights will lead to optimal predictions given sufficient training data [prediction consistency Shimodaira, 2000, Lemma 4]. However the usual (unweighted) estimator is not consistent for an over-constrained (misspecified) model. Indeed, a over-constrained model may be able to fit the data well only in some regions of the input feature space (Figure 1). In this case reweighting training examples (Section 5) to give more importance to those that are more representative of the target data is beneficial [START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF][START_REF] Schölkopf | On causal and anticausal learning[END_REF]. Figure 5 illustrates covariate shift.

Prior probability shift

Another relatively simple case of dataset shift is prior probability shift. With prior probability shift (a.k.a. label shift or target shift), the distribution of Y changes but not P (X | Y ). This happens for example when disease prevalence changes in the target population but manifests itself in the same way. Even more frequently, prior probability shift arises when one rare class is over-represented in the training data so that the dataset is more balanced, as when extracting a biomarker from a case-control cohort, or when the dataset is resampled as a strategy to handle the class imbalance problem [START_REF] He | Learning from imbalanced data[END_REF]. Prior probability shift can be corrected without extracting a new biomarker, simply by adjusting a model's predicted probabilities using Bayes' rule [as noted for example in [START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF][START_REF] Schölkopf | On causal and anticausal learning[END_REF]. When the classes are well separated, the effect of this correction may be small, i. is correctly captured [START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF]. Thus the polynomial fit of degree 4 performs well on the new dataset. However, an overconstrained learner such as the linear fit can benefit from reweighting training examples to give more importance to the most relevant region of the feature space.

may generalize well without correction. Figure 6 illustrates prior probability shift.

Conclusion

Ideally, machine learning biomarkers would be designed and trained using datasets carefully collected to be representative of the targeted population -as in [START_REF] Liu | Sensitive and specific multi-cancer detection and localization using methylation signatures in cell-free dna[END_REF]. To be trusted, biomarkers ultimately need to be evaluated rigorously on one or several independent and representative samples. However, such data collection is expensive. It is therefore useful to exploit existing datasets in an opportunistic way as much as possible in the early stages of biomarker development. When doing so, correctly accounting for dataset shift can prevent wasting important resources on machine-learning predictors that have little chance of performing well outside of one particular dataset.

We gave an overview of importance weighting, a simple tool against dataset shift. Importance weighting needs a clear definition of the targeted population and access to a diverse training dataset. When this is not possible, distributionally robust optimization may be promising alternative, though it is a more recent approach and more difficult to implement. Despite much work and progress, dataset shift remains a difficult problem. Characterizing its impact

Balanced dataset

Target dataset

Decision boundary Original fit Corrected for label shift and the effectiveness of existing solutions for biomarker discovery will be important for machine learning models to become more reliable in healthcare applications.

We conclude with the following recommendations:

• be aware of the dataset shift problem and the difficulty of out-of-dataset generalization. Do not treat crossvalidation scores on one dataset as a guarantee that a model will perform well on clinical data.

• collect diverse, representative data.

• use powerful machine-learning models and large datasets.

• consider using importance weighting to correct biases in the data collection, especially if the learning model may be over-constrained (e.g. when using a linear model).

• look for associations between prediction performance and demographic variables in the validation set to detect potential generalization or fairness issues.

• do not remove confounding signal in a predictive setting.

These recommendations should help designing fair biomarkers and their efficient application on new cohorts.

Alternatively, a discriminative model can be trained to distinguish source and target examples. In the specific case of preferential sample selection, this means estimating directly the probability of selection P (S = 1) (cf Equation ( 6)). In general, the shift is not always due to selection: the source data is not necessarily obtained by subsampling the target population. In this case we denote T = 1 if an individual comes from the target data and T = 0 if it comes from the source data. Then, a classifier can be trained to predict from which dataset (source or target) a sample is drawn, and the importance weights obtained from the predicted probabilities [Sugiyama and Kawanabe, 2012, Sec. 4.3]:

w(x, y) = P (T = 1 | X = x, Y = y) P (T = 0) P (T = 0 | X = x, Y = y) P (T = 1) , (7)
The classifier must be calibrated (i.e. produce accurate probability estimates, not only a correct decision), see [START_REF] Niculescu-Mizil | Predicting good probabilities with supervised learning[END_REF]. Note that constant factors such as P (T = (T = 1) usually do not matter and are easy to estimate if needed. This discriminative approach is effective because the distribution of (T | X = x, Y = y) is much easier to estimate than the distribution of (X, Y | T = t) : T is a single binary variable whereas (X, Y ) is high-dimensional and often continuous.

The classifier does not need to distinguish source and target examples with high accuracy. In the ideal situation of no dataset shift, the classifier will perform at chance level. On the contrary, a high accuracy means that there is little overlap between the source and target distributions and the model will probably not generalize well.

A.2.2 What distributions differ in source and target data?

When computing importance weights, it is possible to exploit prior knowledge that some distributions are left unchanged in the target data. For example,

p t (x, y) p s (x, y) = p t (y | x) p t (x) p s (y | x) p s (x) . (8)
Imagine that the marginal distribution of input X differs in source and target data, but the conditional distribution of the output Y given the input stays the same: p t (x) = p s (x) but p t (y | x) = p s (y | x) (a setting known as covariate shift).

Then, the importance weights simplify to w(x, y) = p t (x) p s (x) .

In this case, importance weights can be estimated using only unlabelled examples (individuals for whom Y is unkown) from the target distribution.

Often, the variables that influence selection (e.g. demographic variables such as age) are lowerdimensional than the full features (e.g. high-dimensional images), and dataset shift can be corrected with limited information on the target distribution, with importance weights or otherwise.

Moreover, even if additional information Z that predicts selection but is independent of (X, Y ) is available, it should not be used to compute the importance weights. Indeed, this would only increase the weights' variance without reducing the bias due to the dataset shift [Hernán and Robins, 2020, Sec. 15.5].

B Tobacco smoking prediction in the UKBiobank

We consider predicting the smoking status of participants in the UKBiobank study to illustrate the effect of dataset shift on prediction performance. 6,000 participants are used in a preliminary step to identify the 29 most relevant predictive features (listed in appendix B.1), by cross-validating a gradient boosting model and computing permutation feature importances. We then draw two samples of 100K individuals from the rest of the dataset, that have different age distributions. In the young sample, 90% of individuals come from the youngest 20% of the dataset, and the remaining 10% are sampled from the oldest 20% of the dataset. In the old sample, these proportions are reversed. We then perform 10-fold cross validation. For each fold, both the training and testing set can be drawn from either the young or the old population, resulting in four tasks on which several machine-learning estimators are evaluated. We use this experiment to compare 2 machinelearning models: a simple one -regularized linear Support Vector Classifier, and a flexible one -Gradient Boosting. For each classifier, 3 strategies are considered to handle the dataset shift: (i) baseline -the generic algorithm without modifications, (ii) Importance Weighting (Section 5), and (iii) the (unfortunately popular) non-solution: "regressing out the confounder" -regressing the predictive features on the age and using the residuals as inputs to the classifier.

The results are similar to those seen with simulated data in Figure 1. For a given learner and test population, training on a different population degrades the prediction score. For example, if the learner is to be tested on the young population, it performs best when trained on the young population. Gradient Boosting vastly outperforms the linear model in all configurations. Regressing out the age always degrades the prediction; it is always worse than the unmodified baseline, whether a dataset shift is present or not. Finally, Importance Weighting (Section 5) improves the predictions of the over-constrained (misspecified) linear model in the presence of dataset shift, but degrades the prediction of the powerful learner used in this experiment. This is due to the fact that the Gradient Boosting already captures the correct separation for both young and old individuals, and therefore Importance Weighting does not bring any benefit but only reduces the effective training sample size by increasing the variance of the empirical risk.

B.1 Features used for tobacco smoking status prediction

The 30 most important features were identified in a preliminary experiment with 6,000 participants (that were not used in the subsequent analysis). One of these features, "Date F17 first reported (mental and behavioural disorders due to use of tobacco)", was deemed trivial -too informative, as it directly implies that the participant does smoke tobacco, and removed. The remaining 29 features were used for the experiment described in Section 3.

• Forced expiratory volume in 1-second (FEV1), predicted percentage

• Lifetime number of sexual partners

• Age first had sexual intercourse

• Age when last took cannabis

• Ever taken cannabis

• Forced expiratory volume in 1-second (FEV1), predicted

• Acceptability of each blow result

• Mouth/teeth dental problems

• Coffee intake

• FEV1/ FVC ratio Z-score

• Alcohol intake frequency.

• Date J44 first reported (other chronic obstructive pulmonary disease)

• Former alcohol drinker

• Average weekly spirits intake

• Year of birth

• Acceptability of each blow result

• Date of chronic obstructive pulmonary disease report

• Leisure/social activities

• Morning/evening person (chronotype)

• Mean sphered cell volume

• Lymphocyte count

• Townsend deprivation index at recruitment

• Age hay fever, rhinitis or eczema diagnosed

• Age started oral contraceptive pill

• White blood cell (leukocyte) count

• Age completed full time education

• Age at recruitment

• Workplace had a lot of cigarette smoke from other people smoking

• Wheeze or whistling in the chest in last year

Figure 1 .

 1 Figure 1. Classification with dataset shift -regressing out a correlate of the shift does not help generalization.The task is to classify patients (orange) from healthy controls (blue), using 2-dimensional features. Age, indicated by the shade of gray, influences both the features and the probability of disease. Left: generative process for the simulated data. Age influences both the target Y and the features X, and Y also has an effect on X. Between the source and target datasets, the distribution of age changes. The two arrows point towards increasing age and represent the Healthy and Diseased populations, corresponding to the orange and blue clouds of points in the right panel. The grayscale gradient in the arrows represents the increasing age of the individuals (older individuals correspond to a darker shade). Throughout their life, individuals can jump from the Healthy trajectory to the Diseased trajectory, which is slightly offset in this 2-dimensional feature space. As age increases, the prevalence of the disease increases, hence the Healthy trajectory contains more individuals of young ages (its wide end), and less at older ages (its narrow end) -and vice-versa for the Diseased trajectory. Right: predictive models In the target data (bottom row), the age distribution is shifted: individuals tend to be older. Elderly are indeed often less likely to participate in clinical studies[START_REF] Heiat | Representation of the elderly, women, and minorities in heart failure clinical trials[END_REF]. First column: no correction is applied. As the situation is close to a covariate shift (Section 7.1), a powerful learner (RBF-SVM) generalizes well to the target data. An over-constrained model -Linear-SVM -generalizes poorly. Second column: wrong approach. To remove associations with age, features are replaced by the residuals after regressing them on age. This destroys the signal and results in poor performance for both models and datasets. Third column: Samples are weighted to give more importance to those more likely in the target distribution. Small circles indicate younger individuals, with less influence on the classifier estimation. This reweighting improves prediction for the linear model on the older population.

Figure 3 .

 3 Figure 3. Sample selection bias: three examples. On the right are graphs giving conditional independence relations[START_REF] Pearl | Causal inference in statistics: A primer[END_REF]. Y is the lesion volume to be predicted (i.e. the output). M are the imaging parameters, e.g. contrast agent dosage. X is the image, and depends both on Y and M (in this toy example X is computed as X := Y +M + , where is additive noise. S indicates that data is selected to enter the source dataset (orange points) or not (blue points). The symbol ⊥ ⊥ means independence between variables. Preferentially selecting samples results in a dataset shift (middle and bottom row). Depending on whether Y ⊥ ⊥ S | X, the conditional distribution of Y | X -here lesion volume given the image -estimated on the selected data may be biased or not.

Figure 4 .

 4 Figure4. Dataset shifts that may or may not be compensated by reweighting -Left: distribution of sex can be balanced by downweighting men and upweighting women. Right: women are completely missing; the dataset shift cannot be fixed by importance weighting.

Figure 5 .

 5 Figure5. Covariate shift: P (Y | X) stays the same but the feature space is sampled differently in the source and target datasets. A powerful learner may generalize well as P (Y | X) is correctly captured[START_REF] Storkey | When training and test sets are different: characterizing learning transfer[END_REF]. Thus the polynomial fit of degree 4 performs well on the new dataset. However, an overconstrained learner such as the linear fit can benefit from reweighting training examples to give more importance to the most relevant region of the feature space.

Figure 6 .

 6 Figure6. Prior probability shift: when P (Y ) changes but P (X | Y ) stays the same. This can happen for example when participants are selected based on Y -possibly to have a dataset with a balanced number of patients and healthy participants: X ← Y → S . When the prior probability (marginal distribution of Y ) in the target population is known, this is easily corrected by applying Bayes' rule. The output Y is typically low-dimensional and discrete (often it is a single binary value), so P (Y ) can often be estimated precisely from few examples.

https://fda.report/PMN/K192854

Author contributions Jérôme Dockès, Gaël Varoquaux and Jean-Baptiste Poline participated in conception, literature search, data interpretation, and editing the manuscript. Jérôme Dockès wrote the software and drafted the manuscript. Both Gaël Varoquaux and Jean-Baptiste Poline contributed equally to this work (as last authors).

Competing interests statement The authors declare that there are no competing interests.

Software and data availability

The source files used to create this publication can be found in this repository:

https://github.com/neurodatascience/ dataset_shift_biomarkers. UKBiobank data can be obtained from https://www.ukbiobank.ac.uk.

A Definition and estimation of importance weights

We will implicitly assume that all the random variables we consider admit densities and denote p s and p t the density of the joint distribution of (X, Y ) applied to the source and target populations respectively. If the support of p t is included in that of p s (meaning that p s > 0 wherever p t > 0), we have:

(3) where L is the cost function and f is a prediction function, E source (resp. E target ) the expectation on the source (resp. target) data. The risk (on target data) can therefore be computed as an expectation on the source distribution where the loss function is reweighted by the importance weights:

If ŵ are empirical estimates of the importance weights w, it is possible to compute the reweighted empirical risk:

Rather than being weighted, examples can also be resampled with importance or rejection sampling [START_REF] Zadrozny | Cost-sensitive learning by cost-proportionate example weighting[END_REF][START_REF] Zadrozny | Learning and evaluating classifiers under sample selection bias[END_REF]. Importances can also be taken into account for model selection -for example in [START_REF] Sugiyama | Covariate shift adaptation by importance weighted cross validation[END_REF] examples of the test set are also reweighted when computing cross-validation scores. [START_REF] Cortes | Sample selection bias correction theory[END_REF] study how errors in the estimation of the weights affect the prediction performance.

A.1 Preferential Sample selection and Inverse Probability weighting

In the case of preferential sample selection (Section 4), the condition that requires for the support of p t to be included in the support of p s translates to a requirement that all individuals have a non-zero probability of being selected: P (S = 1 | x, y) > 0 for all (x, y) in the support of p t . When this is verified, by applying Bayes' rule the definition of importance weights in Equation ( 4) can be reformulated [see Cortes et al., 2008, Sec. 2.3]:

w(x, y) = P (S = 1)

These weights are sometimes called Inverse Probability weights [START_REF] Hernán | A structural approach to selection bias[END_REF] or Inverse Propensity scores [START_REF] Austin | An introduction to propensity score methods for reducing the effects of confounding in observational studies[END_REF]. Training examples that had a low probability of being selected receive higher weights, because they have to account for similar individuals who were not selected.

A.2 Computing importance weights

In practice p t (x, y), which is the joint density of (X, Y ) in the target data, is not known. However, it is not needed for the estimation of p t /p s . More efficient estimation hinges on two observations: estimation of both densities separately is not necessary to estimate their ratio, and variables that have the same distribution in source and target data can be factored out.

Here we describe methods that estimate the true importance weights p t /p s , but we point out that reweighting the training examples reduces the bias of the empirical risk but increases the variance of the estimated model parameters. Even when the importances are perfectly known, it can therefore be beneficial to regularize the weights [START_REF] Shimodaira | Improving predictive inference under covariate shift by weighting the log-likelihood function[END_REF].

A.2.1 Computing importance weights does not require distributions densities estimation

Importance weights can be computed by modelling separately p s and p t and then computing their ratio [Sugiyama and Kawanabe, 2012, Sec. 4.1]. However, distribution density estimation is notoriously difficult; non-parametric methods suffer from the curse of dimensionality and parametric methods depend heavily on the correct specification of a parametric form. But estimating both densities is more information than is needed to compute the sample weights. Instead, one can directly optimize importance weights in order to make the reweighted sample similar to the target distribution, by matching moments [START_REF] Sun | Return of frustratingly easy domain adaptation[END_REF] or mean embeddings [START_REF] Huang | Correcting sample selection bias by unlabeled data[END_REF][START_REF] Zhang | Domain adaptation under target and conditional shift[END_REF], minimizing the KLdivergence [START_REF] Sugiyama | Direct importance estimation with model selection and its application to covariate shift adaptation[END_REF], solving a least-squares estimation problem [START_REF] Kanamori | A least-squares approach to direct importance estimation[END_REF] or with optimal transport [START_REF] Courty | Optimal transport for domain adaptation[END_REF].

C Glossary

Here we provide a summary of some terms and notations used in the paper.

Target population the population on which the biomarker (machine-learning model) will be applied.

Source population the population from which the sample used to train the machine-learning model is drawn.

Selection in the case that source data are drawn (with non-uniform probabilities) from the target population, we denote by S = 1 the fact that an individual is selected to enter the source data (e.g. to participate in a medical study).

Provenance of an individual when samples from both the source and the target populations (e.g. Appendix A.2.1) are available, we also denote T = 1 if an individual comes from the target population and T = 0 if they come from the source population.

Confounding in causal inference, when estimating the effect of a treatment on an outcome, confounding occurs if a third variable (e.g. age, a comorbidity, the seriousness of a condition) influences both the treatment and the outcome, possibly producing a spurious statistical association between the two. This notion is not directly relevant to dataset shift, and we mention it only to insist that it is a different problem. See [START_REF] Hernán | Causal inference: What if[END_REF], Chap. 7, for a more precise definition.

Domain adaptation the task of designing machinelearning methods that are resilient to dataset shift -essentially a synonym for dataset shift, i.e. another useful search term for readers looking for further information on this problem.