Cyril Pommier

• Heterogenous data • Different spatial scales Six calibrated discs of lamina were cut from this leaf, dried at 50 °C during 48 °C and weighed, and specific leaf area (SLA, cm 2 g -1 ) was computed. Leaf discs were ground to fine powder for analysis of leaf carbon isotope composition (δ 13 C), carbon (C M ) and nitrogen (N M ) contents. One-milligram subsamples of ground material were used for measuring the CO 2 produced by combustion and its 13 CO 2 / 12 CO 2 ratio by a continuous flux isotope ratio mass spectrometer. The discrimination between atmospheric CO 2 and plant material was calculated. • Investigations are research programmes with defined aims. They can exist at various scales: e.g. grant-funded programme of work with various published components; a single experiment.

• One investigation holds one to many studies.

MIAPPE main sections -Study

• A "study" (or experiment)  One study = one location  comprises a series of assays (or measurements) undertaken to answer a particular biological question.

• Metadata

 experiment as a whole  timing, location, statistical design, cultural pratices (but not event lists), etc...

• Examples of Studies:

•

MIAPPE main sections -Biological material

• Biological material being studied • Plus its source (stock center, gen bank, etc…).

• Crucial for integrating phenotyping data with genomic or genetic data.

•• 3

 3 Minimal and sufficient list of metadata: • The objective of the experiment • Who contributed to the experiment • What were the experimental procedures • What was the biological material experimented • ... Papoutsoglou et al. (2020) Enabling reusability and interoperability of plant phenomic datasets with MIAPPE 1.1. New Phytol, 227:260-273; https://doi.org/10.1111/nph.16544 Computer scientist friendly : • Explicit Data Model: ISA-Tools and Breeding API (BrAPI) compatibility. • Validation framework and toolbox • Semantic representation in OWL: Phenotyping Experiment Ontology (PPEO) using OWL (http://agroportal.lirmm.fr/ontologies/PPEO )Biologist friendly• Clear definitions and examples • Excel templates • Trainings Specification with multiple implementations• File Archive: ISA Tab • Semantic: Plant Phenotyping Experiment Ontology • Web Service: Breeding API p. 24 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Phenotype Technical Standard •Breeding API http://brapi.org/ •International collaboration ◆ Standard Open Web Service API ◆ Information Exchange, Main target: Breeding ◆ Excellence in Breeding platform (CGIAR, Peter Selby) •Major Elixir, Emphasis Contribution◆ Phenotyping specification •Connect data repositories and tools: ◆ Genotype visualization (Flapjack) ◆ Studies graph preview and filtering ◆ BrAPPS : Tools integrable in any BrAPI compliant System ◆ https://www.brapi.org/brapps.php ◆ R analysis environment ◆ Field data capture ◆ FAIR Data discovery  Elixir FAIDARE Selby et al. Bioinformatics (2019), doi.org/10.1093/bioinformatics/btz190 for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Questions Plant Phenotyping standards : why and who MIAPPE overview through Crop & Forest use case Crop ontology MIAPPE tools & web services: BrAPI, PPEO Ontology p. 26 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier MINIMUM INFORMATION ABOUT A PLANT PHENOTYPING EXPERIMENT MIAPPE Overview and use cases p. 27 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Two examples to illustrate how to capture important information about the phenotyping experiments: • MAIZE: [1] Millet et al. 2019 (https://doi.org/10.15454/IASSTN): A multi-site experiment in a network of European fields for assessing the maize yield response to environmental scenarios. • POPLAR: [2] Monclus et al. 2012 (http://dx.doi.org/10.1186/1471-2229-12-173): Integrating genome annotation and QTL position to identify candidate genes for productivity, architecture and water-use efficiency in Populus spp Capturing important information about phenotyping experiments using the MIAPPE standard p. 28 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Millet et al 2019 [1] -Material&methods section on phenotyping experiments • A panel of 256 maize hybrids was grown with two water regimes (irrigated or rainfed). • Location: seven fields in 2012 and 2013, plus one site in Chile in 2013 • This resulted in 29 experiments defined as the combination of one year, one site and one water regime, with two and three repetitions for rainfed and irrigated treatments, respectively. • A detailed environmental characterisation was carried out, with hourly records of micrometeorological data and soil water status, and associated with precise measurement of phenology. • « grain.yield»: yiel adjusted at 15% grain moisture, in ton per hectare (t ha-1). «grain.number»: number of grain per square meter. «grain.weight»: individual grain weight (mg). «anthesis»: male flowering (pollen shed), in thermal time cumulated since emergence (d20°C). «silking»: female flowering (silking emergence), in thermal time cumulated since emergence (d20°C). «plant.height»: plant height, from ground level to the base of the flag leaf (highest) leaf (cm). «tassel.height»: plant height including tassel, from ground level to the highest point of the tassel (cm). «ear.height»: ear insertion height, from ground level to ligule of the highest ear leaf (cm). Field trial established in April 2003 located in France (Ardon, 47°49'41''N, 1°54'39''E, 110 m),Italy Cavallermaggiore ((44° 43' 0'' N) , (7° 41' 0'' E)), UK Headley ((51° 7' 0'' N) , (-1° 10' 0'' W) ) • The biological material consisted of a cloned 336 F1 progeny from an interspecific cross between the female Populus deltoides (Bartr. Ex Marsh.) '73028-62' from Illinois and the male P. trichocarpa (Torr. and Gray) '101-74' from Washington State. • The trial was established from 25 cm-homogenous hardwood cuttings planted at a plant density of 6670 trees per ha. The trial was and consisted in 6 randomized complete blocks where each F1 genotype and each parent was represented by one replicate. • Circumference and stem height were measured at the end of the first (winter 2003-2004) and second (winter 2004-2005) as described in Dillen et al. Forest Ecol Manag. 2007, 252 (1-3): 12-23). Growth increment in height and circumference during the second growing season were calculated. • Leaf traits where measured in 2003: one fully illuminated mature leaf was collected on each tree according to Monclus et al. http://doi.org/10.1111/j.1469-8137.2005.01407.x ).

  by their position in each block: row and column

•

  Metadata Similar to Archive deposition  DOI, title, description, associated publications/people, ... • Examples Investigation : • MAIZE [1]: the whole set of multilocal and pluriannual phenotyping experiments • POPLAR [2]: the whole set of multilocal measurements over three years p. 34 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier

•

  POPLAR [2] : 3 locations over one to several years (Ardon_2003) (Ardon_2003-2005) internal accession number (mandatory) • BioSample ID Plant Samples used in the study: detached leaves, ...

•

  Metadata Minimal fields from Multicrop Passport Descriptor (MCPD) standard  GPS location for forest tree / in situ material provenance p. 37 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier MIAPPE main sections -Biological material for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier MIAPPE main sections -Observed Variable (Assay) • An Observed Variable (assay)  specific measurement.  Targets a trait  with a method and a scale.  Trait: Phenotype or Environment • Metadata  Crop Ontology Trait Dictionary:  trait, method, scale • Example: • In each study of Millet et al. [1] :  Plant level: 20 variables  Genotype/ Study level: 19 variable both phenotype and environment  Plot level: 9 variables  E.g. Female flowering days to silking D20deg  E.g. Plant height (cm) for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier MIAPPE main sections -Observation Unit, Samples • Observation units  objects in the study  Observations/measures are made on observation units  Treatments values are made on observation units  Also used for environmental variables.• Metadata are specific to MIAPPE: identifiers, location, replication, treatments, …• A sample • portion of plant tissue extracted from an observation unit • sub-plant observations and/or molecular studies.• Metadata : identifiers, information about processing, … p. 41 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril PommierExample:Monclus et al., 2012, [2] 6 randomized blocks 1 observation unit = one tree No treatment 6 replicates defined by their position in each block: row and column processing of the leaf disks depending on the measurement: can be captured by different sets of samples (e.g. if subsampling generates repetitions) or in the method of the observed variable.

  model -Other Important sections p. 44 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier MIAPPE main sections -Experimental Factor • Biotic or abiotic experimental factor  Its effects are evaluated in the study.  Takes different values in the observation units • Metadata: name, description and value • Example : Maize dataset [2]  Two Experimental factors/treatments: Rainfed, Watered  Identified by dedicated metadata  Block organisation  Study the effect of Environment/Drought on the Biological material p. 45 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier

•

  REST is an architecture design for creating Web Services using the well known HTTP standard • Requests are made with URLs • Data is represented with JSON BrAPI 101 Slides, P. Selby

European Networks National Networks Global Networks Interoperability in International network 14 International data standards

	Web services	Minimal information
	Controlled vocabularies Trait dictionaries	MCPD
		p. 15

• Phenotype = Genotype x Environment x Cultural practice => different silos of metadata definition • Provenance is complex Data portals Data Archives MCPD Platform IS … p. 13 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier PLANT DATA STANDARDS : WHO p. 14 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier

Pommier Phenotype Semantic Standard: Ontologies

	Community driven recomendations Plant data standards contributors • WheatIS: http://wheatis.org/DataStandards.php Semantic  Description of the data  Controlled vocabularies: term name and definitions • Describing traits/features in specific plant species • Formatting and Organizing the data Structure • Community story • Data Models • Standards : CSV, VCF, GFF, MIAPPE • Crop Ontology Trait + Method + Scale Semantic model
	 • Dedicated presentation Ontologies: semantic links between terms	(www.miappe.org) , etc…
		Biologist driven	• Biologist & Computer scientist driven
		Persistent Unique
		Identifiers
		URI, gene ID, accessions ID, Trait • Data integration and sharing • NAPPN • North American Plant Phenotyping • https://nappn.plant-phenotyping.org/ Technical Network ID, DOI,…
		• Interoperability : tools and systems
		• GA4GH
			p. 20 p. 21	17

Gathered to solve interoperability standard problem p. 16 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Sharing standards: standards registries p. 17

Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier p. 18 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier PLANT DATA STANDARDS p. 19 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Data standards for FAIR • Breeding API www.brapi.org • Computer scientist driven Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril

Pommier Phenotype Structure Standard

	Minimal Information About Plant Phenotyping Experiment : version 1.1 (Jan 2019)
	www.miappe.org
	• Many stakeholders



Elixir

, Emphasis, Bioversity, North American PPN • Open Community:  Request for comments  Github Feature requests  Mailing lists  Meetings & Workgroups • Crops and woody plants

p. 22 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier

occurrence at a particular time  Natural, e.g. rain, unwanted pathogen attack  Cultural practice, e.g. planting, watering, etc.  Whole study level or at the observation unit level  It is not the studied Factor but an additional information

	Environment parameter Study Name Sowing density	Environment parameter Value Event Description 20 seeds/m2	Date

Monclus et al., 2012

Rain 15mm of rain on the orchard 2012-03-15 MIAPPE main sections -Event • Event  Discrete • Metadata: name, description, time/date • Examples: In Poplar, [2] • the field establishment date, 2003. • the orchard was subjected to 15mm of rain on March 15, 2012 (fiction). p. 46 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Rooting medium composition Ca (XEO:00058): 5 mg/L; ...

MIAPPE main sections -Environment

• Environment

parameters  Constant throughout the study  Did not change between observation units or assays.  Environment characteristics that vary over time, i.e. environmental variables, should be recorded as Observed Variables

  Crop ontology MIAPPE tools & web services: BrAPI, PPEO Ontology

	Study	Genotype CIR HT SUR	Study	Genotype H CIR
	Domaine de Valcros 6579	600 17 1	La Nerthe 6579	23 650
	Domaine de Valcros 6580	482 14 1	La Nerthe 6580

• Metadata :name, value • Example: p. 47 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Perspectives • Open community  Github  Mailing list • Outreach • Training • Elixir Service bundles p. 48 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Questions Plant Phenotyping standards : why and who MIAPPE overview through Crop & Forest use case p. 51 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier Standard model p. 67 Plant Data Managment for Phenotyping Experiments 17 Dec 2020 / webinar / Cyril Pommier


Thomas Ferté
email: thomas.ferte@u-bordeaux.fr

Sébastien Cossin

Thierry Schaeverbeke

Thomas Barnetche

Vianney Jouhet

Boris P Hejblum

Keywords: Methodology, Formal analysis, writing - Methodology, Resources, Writing -review, Thierry Schaeverbeke: Conceptualisation, writing -review, Conceptualisation, writing -review, Vianney Jouhet Conceptualization, Methodology, Supervision, writing -review (5): electronic health records, high-throughput phenotyping, phenotypic big data, precision medicine

Electronic Health Records (EHRs) often lack reliable annotation of patient medical conditions. Phenorm, an automated unsupervised algorithm to identify patient medical conditions from EHR data, has been developed.

PheVis extends PheNorm at the visit resolution. PheVis combines diagnosis codes together with medical concepts extracted from medical notes, incorporating past history in a machine learning approach to provide an interpretable "white box" predictor of the occurrence probability for a given medical condition at each visit.

PheVis is applied to two real-world use-cases using the datawarehouse of the University Hospital of Bordeaux: i) rheumatoid arthritis, a chronic condition; ii) tuberculosis, an acute condition (cross-validated AUROC were respectively 0.943 [0.940 ; 0.945] and 0.987 [0.983 ; 0.990]). PheVis performs well for chronic conditions, though absence of exclusion of past medical history by natural language processing tools limits its performance in French for acute conditions. It achieves significantly better performance than state-of-the-art methods especially for chronic diseases.

Introduction

As the amount of data collected on a daily basis from hospital health care system keeps increasing, [START_REF] Kim | The Evolving Use of Electronic Health Records (EHR) for Research[END_REF] the appeal for leveraging the full potential of these data for research purposes and to investigate clinical questions is also becoming stronger than ever. [START_REF] Beesley | The emerging landscape of health research based on biobanks linked to electronic health records: Existing resources, statistical challenges, and potential opportunities[END_REF][START_REF] Coorevits | Electronic health records: new opportunities for clinical research[END_REF][START_REF] Danciu | Secondary use of clinical data: The Vanderbilt approach[END_REF][START_REF] Sandhu | Secondary Uses of Electronic Health Record Data: Benefits and Barriers[END_REF] Yet, EHR data are quite different from research oriented data (e.g. cohort or trial data): i) they are less structured, more heterogeneous, ii) they present finer granularity, iii) data collection is done for health care purpose. [START_REF] Kim | The Evolving Use of Electronic Health Records (EHR) for Research[END_REF][START_REF] Wilcox | Leveraging Electronic Health Records for Phenotyping[END_REF][START_REF] Weiskopf | Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research[END_REF][START_REF] Hripcsak | Next-generation phenotyping of electronic health records[END_REF] Currently, one of the main barriers to use such data for studying disease risk factors is the necessity to first identify patients having diseases of interest, a task that we will denote as phenotyping.

Several approaches have been recently proposed to phenotype patients. [START_REF] Banda | Advances in Electronic Phenotyping: From Rule-Based Definitions to Machine Learning Models[END_REF][START_REF] Agarwal | Learning statistical models of phenotypes using noisy labeled training data[END_REF][START_REF] Halpern | Electronic medical record phenotyping using the anchor and learn framework[END_REF][START_REF] Wagholikar | Polar labeling: silver standard algorithm for training disease classifiers[END_REF][START_REF] Yu | Enabling phenotypic big data with PheNorm[END_REF] They often rely on either rule-based algorithms specifically designed with clinicians, or on supervised models trained on annotated patient datasets. Such algorithms are limited because their development is disease specific, must be (re-)started from scratch for every new disease and demand a lot of clinician expertise time. In addition, portability and generalization to new databases (e.g. different hospitals) can often fail, requiring once again the process to be reiterated in the new institution. Hripcsak and Albers defined high-throughput phenotyping as an approach that "should generate thousands of phenotypes with minimal human intervention". [START_REF] Hripcsak | Next-generation phenotyping of electronic health records[END_REF] In this perspective, multiple methods have been developed for automatic phenotyping. Agarwal et al. proposed XPRESS which learns on noisy labels. [START_REF] Agarwal | Learning statistical models of phenotypes using noisy labeled training data[END_REF] Halpern et al proposed Anchor which learns on so-called "anchor patients", i.e. patients with highly disease-specific features. [START_REF] Halpern | Electronic medical record phenotyping using the anchor and learn framework[END_REF] Wagholikar et al developed Polar, which learns on so called "polar patients", i.e extreme patients which are almost certain to either have or not have the disease. [START_REF] Wagholikar | Polar labeling: silver standard algorithm for training disease classifiers[END_REF] Finally Yu et al. developed PheNorm which learns the phenotype as a continuous score. [START_REF] Yu | Enabling phenotypic big data with PheNorm[END_REF] And they also developed SAFE which selects relevant features for phenotyping in an automated manner. [START_REF] Yu | Surrogate-assisted feature extraction for high-throughput phenotyping[END_REF] All these frameworks are unsupervised, in the sense that they require neither manual chart review nor complex rule definitions to classify phenotypes, and thus allow automated high-throughput phenotyping.

While those frameworks are appealing, they only consider phenotyping at the patient level and neglect the timing of illness onset and cure. Yet, we need increased resolution for phenotyping, especially for studying acute diseases (that can occur repeatedly) or for answering epidemiological questions (where temporal sequence is important): phenotyping at the visit level would allow to precisely take into account the dynamic evolution of patient's conditions. Besides, those frameworks were developed using English databases, leveraging advanced NLP tools and relying on rich terminologies not necessarily available in other languages. [START_REF] Yu | NILE: Fast Natural Language Processing for Electronic Health Records[END_REF][START_REF] Bodenreider | From French vocabulary to the Unified Medical Language System: A preliminary study[END_REF] Portability to other languages is not straightforward, as they still often lack resources of matching quality.

We propose a new, portable, approach for unsupervised algorithm extending PheNorm at the visit level:

PheVis. This new PheVis method accumulates past information to provide an up-to-date estimation of a phenotype probability at any given visit. This accumulation of previous information from EHR can be tuned to match the disease length, making PheVis a versatile tool suitable for both chronic and acute conditions. Section 2 presents the PheVis analysis and modeling strategy. Section 3 demonstrates PheVis performance for rheumatoid arthritis (RA) and tuberculosis (TB), a chronic and an acute condition respectively, using French EHRs from Bordeaux University Hospital. The method is compared to other state-of-the-art methods. Finally Section 4 discusses these findings, the limits of the approach, and offers a conclusion.

Materials and Methods

PheVis combines ICD10 (international classification of diseases 10 th revision) billing codes together with medical concepts extracted from clinical notes, incorporating past information through a user-tunable exponential decay. This creates a silver-standard surrogate of the medical condition of interest. Then variable selection (through elastic-net logistic regression) and pseudo-labelling (using random-forest) are performed, leveraging extreme values of this silver-standard. Finally, a logistic regression model is estimated on those noisy labels to provide an interpretable "white box" predictor of the occurrence probability for a given medical condition at each visit.

The different steps of PheVis are outlined in Figure 1 and are described below.

Input data

The input data of the PheVis approach are the clinical notes and the ICD10 codes from an EHR datawarehouse.

All the notes and ICD10 codes are collapsed by visit, and IAMsystem, a dictionary-based named entity recognition tool, is used to extract relevant Unified Medical Language System (UMLS) concept unique identifiers (CUIs, i.e. CUIs associated with disease to be phenotyped -see Section 2.3 for details). [START_REF] Cossin | IAM at CLEF eHealth 2018 : Concept Annotation and Coding in French Death Certificates[END_REF][START_REF] Bodenreider | The Unified Medical Language System (UMLS): integrating biomedical terminology[END_REF] CUIs features are the number of occurrence of the terms in the UMLS dictionary related to the CUIs. No semantic analysis is performed avoiding the distinction between current disease and past disease or negation. ICD10 codes are aggregated at the category level (i.e. the first three characters, M05.1 and M05.2 codes are both counted under the same category code M05). This results in a matrix 𝑋 ! !" " of dimension 𝜑 × 𝑃 , where 𝜑 is the total number of visits and 𝑃 the total number of ICD and CUI concepts. Rows of 𝑋 ! !" " are indexed by the vector 𝑘 #$ ∈ {1, … , 𝜑} with 𝑖 ∈ {1, … , 𝑛} the patient index and 𝑗 ∈ {1, … , 𝑣 # } the visit index. The 𝑣 # notation takes into account the variation of number of visits between patients so that 𝜑 = ∑ 𝑣 # % #&' . To facilitate reading, 𝑘 #$ will be denoted as 𝑘 when the patient information is not needed. Columns are indexed by 𝑝 ∈ {1, … , 𝑃} the covariate index.

Build a surrogate of the disease status

As there are no disease labels for the visits (hence requiring a phenotyping algorithm), a supervised model cannot be trained right away. To be able to train our phenotyping algorithm, we first build a surrogate variable expected to be close to the true disease status. This surrogate is based on the main ICD and UMLS codes that represent a disease.

We define 𝑚𝐶 ! the standardized sum of main disease concepts as:

𝑚𝐶 ! = 𝑍(𝑚𝑎𝑖𝑛𝐼𝐶𝐷 ! ) + 𝑍(𝑚𝑎𝑖𝑛𝐶𝑈𝐼 ! ) + 𝑚𝑖𝑛>𝑍(𝑚𝑎𝑖𝑛𝐼𝐶𝐷 ! ) + 𝑍(𝑚𝑎𝑖𝑛𝐶𝑈𝐼 ! )? and 𝑍(𝑥) = 𝑥 -𝜇 𝜎 (1)
𝑚𝑎𝑖𝑛𝐼𝐶𝐷 and 𝑚𝑎𝑖𝑛𝐶𝑈𝐼 are main concepts related to the disease. For example, for RA we used: To phenotype a given visit, it is necessary to take into account information available from previous visits as well.

-𝑚𝑎𝑖𝑛𝐼𝐶𝐷 ! = 𝑀05 ! + 𝑀06 ! with
For example, a patient can be diagnosed with RA at the age of 50 and have a visit at 52 for an infectious event containing no information about RA. RA being a chronic disease, we want to be able to predict RA in both visits.

To do so we propose to accumulate past history information with an exponential decay as follow:

𝑚𝐶𝑢𝑚𝑢𝑙 ! !" = 𝑚𝐶 ! !" + 𝑚𝐶𝑢𝑚𝑢𝑙 ! !"#$ 𝑒𝑥𝑝 (-𝜆𝐷 ! !" ) with 𝑚𝐶𝑢𝑚𝑢𝑙 ! !$ = 𝑚𝐶 ! !$ and 𝐷 ! !" = 𝑡 ! !" -𝑡 ! !"#$ (2)
𝜆 is a constant parameter tuned by the user that controls the "memory loss" of the algorithm. For easier interpretation one can prefer to set the value of the half-life equal to 𝑙𝑛(2) 𝜆 ⁄ . The natural half-life chosen is the usual duration of the disease (e.g. 180 days for TB and +∞ for RA -being a chronic disease currently without a cure). Setting the half-life to +∞ for RA is equivalent to simply accumulating the information of all previous visits.

The same exponential decay accumulation is applied to each ICD and UMLS codes. We also define five other features accumulating the information on the last day, last 5 days, last month and last year:

-𝑙𝑎𝑠𝑡𝑣𝑖𝑠 ! !" = 𝑚𝐶 ! !"#$ -𝑙𝑎𝑠𝑡5𝑣𝑖𝑠 ! !" = ∑ 𝑚𝐶 ! !% $(' )&$(* -𝑙𝑎𝑠𝑡𝑚𝑜𝑛𝑡ℎ ! !" = ∑ 𝑚𝐶 ! !% × 𝟏 + $(' )&' with 𝟏 + = 1 if 𝐷 ! !" -𝐷 ! !% ≤ 30𝑑𝑎𝑦𝑠, 0 otherwise -𝑙𝑎𝑠𝑡𝑦𝑒𝑎𝑟 ! !" = ∑ 𝑚𝐶 ! !% × 𝟏 , $(' )&' with 𝟏 , = 1 if 𝐷 ! !" -𝐷 ! !% ≤ 365𝑑𝑎𝑦𝑠, 0 otherwise -𝐶𝑢𝑚 ! !" = ∑ 𝑚𝐶 ! !" $ '
This yields an augmented matrix 𝑋 -of 𝜑 × (2𝑃 + 5) dimensions: CUIs and ICDs (P), their accumulate counts (P), and 5 new variables.

Variable selection and pseudo-labelling

We use the SAFE algorithm to select predictive variables of interest and reduce the dimensionality of the optimization problem. First we use IAM system to extract ICD10 and UMLS concepts in external resources: medical text books and Wikipedia disease specific chapter or page. [START_REF]Polyarthrite rhumatoïde[END_REF][START_REF]Polyarthrite rhumatoïde[END_REF][START_REF] Tuberculose | Wikipédia[END_REF][START_REF]Référentiel National de Pneumologie -CEP[END_REF] A concept and its accumulate count are kept in the model only if it is found in the two resources. As the true phenotype is not available, we categorize 𝑚𝐶𝑢𝑚𝑢𝑙 ! into 𝑆 ! = {0, 0.5, 1} to provide a surrogate to train models. Those three categories distinguish visits for which phenotype is really unlikely (i.e 𝑆 ! = 0), really likely (i.e 𝑆 ! = 1) or uncertain (i.e 𝑆 ! = 0.5) based on 𝑚𝐶𝑢𝑚𝑢𝑙 ! . To define the two thresholds separating the three categories of 𝑆 ! , we used 𝑚𝑎𝑖𝑛𝐼𝐶𝐷 ! which takes into account prevalence variability depending on the disease and the cohort. We first count the proportion of visits with at least one occurrence of mainICD code (i.e the mainICD ≥ 1 prevalence) that we denote 𝑞𝑢𝑎𝑛𝑡 +-#%./0 as :

𝑞𝑢𝑎𝑛𝑡 +-#%./0 = 1 𝜑 k 𝑚𝑎𝑖𝑛𝐼𝐶𝐷 ! ≥ 1 ! with 𝑞𝑢𝑎𝑛𝑡 +-#%./0 ∈ [0; 1] (3)
We divide this quantity by a constant 𝜔 that we set to 5 to define 𝑞𝑢𝑎𝑛𝑡 12341+1 as :

𝑞𝑢𝑎𝑛𝑡 12341+1 = 𝑞𝑢𝑎𝑛𝑡 +-#%./0 𝜔 with 𝜔 a constant (4)
This 𝑞𝑢𝑎𝑛𝑡 12341+1 proportion allows to define three categories: [0; 𝑞𝑢𝑎𝑛𝑡 12341+1 ], (𝑞𝑢𝑎𝑛𝑡 12341+1 ; 1 -𝑞𝑢𝑎𝑛𝑡 12341+1 ), [1 -𝑞𝑢𝑎𝑛𝑡 12341+1 ; 1] and we define the surrogate 𝑆 ! as:

𝑆 ! = r
0, if visit belongs to the 𝑞𝑢𝑎𝑛𝑡 12341+1 percentile of visits with the lowest 𝑚𝐶𝑢𝑚𝑢𝑙 ! 1, if visit belongs to the 𝑞𝑢𝑎𝑛𝑡 12341+1 percentile of visits with the highest 𝑚𝐶𝑢𝑚𝑢𝑙 ! 0.5 otherwise

For instance, if 20% of visits have at least one occurrence of main ICD codes, then, given 𝜔 = 5, 𝑆 ! is set to 1 for visits belonging to the 4% percentile with the highest 𝑚𝐶𝑢𝑚𝑢𝑙 ! , 𝑆 ! is set to 0 for visits belonging to the 4% percentile with the lowest 𝑚𝐶𝑢𝑚𝑢𝑙 ! and set to 0.5 otherwise. One can note that the higher the 𝜔 constant (i.e the extreme patients are more extreme), the more confident we are in the specificity of 𝑆 ! in {0,1} toward the true phenotype but the smaller training size is for next steps. We found 𝜔 = 5 to work well in our setting.

Then we train a logistic regression with elastic-net penalization to select a subset 𝑋 5 of relevant predictors from the 𝑋 -matrix using only the visits for which 𝑆 ! is either 0 or 1. 𝑋 5 is a 𝜑 × 𝑃′ matrix with the subset of predictors with non-zero estimated coefficients. Of note, 𝑚𝑎𝑖𝑛𝐼𝐶𝐷 and 𝑚𝑎𝑖𝑛𝐶𝑈𝐼 are always forced into the set of selected variables in 𝑋 5 , while 𝐶𝑢𝑚 ! is systematically removed for acute conditions.

We then assign a pseudo-label {0,1} to all visits. This increases the number of visits available to train the final logistic regression, and also adds visits with more uncertain phenotype status which overall results in smoother predicted probabilities and better performance. To perform this pseudo-labelling, we train a random-forest with majority vote for trees aggregation for which 𝑆 ! is either 0 or 1. The trained model is then used to predict the pseudo-label 𝑃𝐿 ! ∈ {0,1} status for each visit. Disease probability can be estimated at this step. However, random forest variable importance is hard to interpret in a clinical perspective which is particularly annoying for unsupervised learning without gold standard label to evaluate the accuracy of the model. To ease the understanding of the disease representation, probabilities are estimated through a logistic regression.

Probability estimation

To estimate the disease occurrence probability, we used a noising-denoising logistic regression with random intercept similarly to PheNorm. First, 𝑚𝑎𝑥 (10 * , 𝜑) visits are randomly sampled with replacement with inverse probability weighting defined as . Then we perform a noising-denoising step to force the algorithm to use other variables than the main ICD and UMLS concepts (and thus avoid overfitting with respect to the surrogate construction). Every value of explanatory variables has a probability of 𝑝 ;14% = 0.3 to be replaced by the mean of the explanatory variable as in PheNorm. [START_REF] Yu | Enabling phenotypic big data with PheNorm[END_REF] For instance if M05 ICD10 code mean occurrence is 0.2 then each visit has a probability of 0.3 to have its true M05 value replaced by 0.2. This noisy matrix of dimension

𝑚𝑎𝑥(10 * , 𝜑) × 𝑃′ is denoted X !" < : X !" < = r 𝑚𝑒𝑎𝑛>𝑋 ; " ?, if 𝑟 ;14% &' = 1 X !" = , if 𝑟 ;14% &' = 0 with 𝑟 ;14% ∼ 𝐵𝑒𝑟𝑛(𝑝 ;14% ) (6)
For the denoising step a logistic regression with random intercept is used:

𝑙𝑜𝑔𝑖𝑡 ~𝑃 •𝑃𝐿 ! !" = 1€• = 𝑋 ! !" " % > 𝛽 " + 𝑏 ?# with 𝑏 ?# ~ 𝑁(0, 𝜎 ? @ ) (7)
And finally the probability of having the disease is estimated on the noise free matrix as:

𝑃(𝐷𝑖𝑠𝑒𝑎𝑠𝑒 ! = 1) = 𝑒 AB &' ( C ) D ' 1 + 𝑒 EB &' ( F ) D ' (8)
This final probability illustrates the level of confidence of the estimated phenotype based on the used variables.

Results

Application design

We illustrate the PheVis method on RA, a chronic disease which cannot be cured, and active TB, an acute disease which usually last between 6 to 12 months. [START_REF]Polyarthrite rhumatoïde[END_REF][START_REF]Polyarthrite rhumatoïde[END_REF][START_REF] Tuberculose | Wikipédia[END_REF][START_REF]Référentiel National de Pneumologie -CEP[END_REF] The model performance was evaluated on an imperfect gold standard for both diseases: for RA we used the presence of at least one rheumatoid arthritis form, a form Ten different prediction models were evaluated for each disease: (i) our proposed PheVis approach, for which we set 𝜆 IJ = K% (@)

specifically
.%M = 0 and 𝜆 >N = K% (@) 'O? (tuberculosis typically lasting around 6 months), (ii) XPRESS for which we defined the silver standard as the main ICD code presence, (iii) ANCHOR for which we defined the anchor visits as the one with at least one main ICD and main CUI code, (iv) the elastic-net and (v) the random forest POLAR algorithms for which we adapted the polar patient definition to our setting (negative polar visits neither main ICD codes and neither main CUI codes, whereas the positive polar visits had both), (vi) PheNorm, (vii) a supervised elastic-net with accumulate variables (denoted as ENET_CUM) and also viii) without accumulate variables (ENET_NOCUM), (ix) a supervised random forest with accumulate variables (RF_CUM) and (x) without cumulated variables (RF_NOCUM). Of note, supervised models (ENET_NOCUM, RF_NOCUM, ENET_CUM and RF_CUM) used the gold-standard labels for training. We used SAFE to select variables used for prediction by each algorithm in order to ease their comparison.

Application results

Figure 2 shows individual PheVis and other state-of-the-art methods predictions for four patients. For RA, as the information is accumulated over time in PheVis, the model is able to maintain relatively stable predictions over time even if there is not much information about RA in a visit. Without this feature, other approaches display highly variables predictions oscillating over time. For TB, the advantage of accumulating information confronts the problem of accumulating too much. This problem is increased in French because the majority of natural language processing tools and terminological systems were developed for the English language. Tools to detect negation and past history are not yet implemented in Bordeaux University Hospital datawarehouse. [START_REF] Garcelon | Improving a full-text search engine: the importance of negation detection and family history context to identify cases in a biomedical data warehouse[END_REF] Prediction of PheNorm is really close to 0.5 for all visits. This can be explained by the final step of the PheNorm algorithm which is a mixture model on the predicted sum of main ICD and main CUI. As our dataset is largely imbalanced, with many more negative patients than positive ones, both normal distributions of the mixture model are concentrated on the negative class. Because they are close to each other, the probability of belonging to each class (positive or negative phenotype) is really close. However, as shown on figure 3, those probabilities are not constant and still give good prediction performance according to AUROC and AUPRC, in spite of being poorly interpretable. Other methods do not have this problem, as they mainly learn on binary silver standard. Table 3 details the performances of the algorithm in the train and test sets compared to other methods and with different hyperparameters (half-life, 𝜔 and pseudo-labels). More elements justifying pseudo-labels are available in the supplementary material. As expected, for TB the advantage of PheVis is less important because the information is accumulated over a shorter time period, still it performs well especially in term of AUROC compared to other unsupervised methods. Supervised methods worked significantly better with accumulated variables supporting the importance of taking into account past history to predict actual phenotype. Among the other unsupervised methods, we can denote that PheNorm and the random forest POLAR seem to be the best methods, however, as shown on figure 2 PheNorm has serious calibration problem on unbalanced datasets.

XPRESS failed to reach convergence in our setting possibly because it uses lasso penalization instead of elasticnet as in SAFE or PheVis, or ridge as in ANCHOR or POLAR. Both for POLAR and the supervised methods, random forest was able to significantly improve the performance of the model, probably because it is able to learn more complex structure than penalized linear regression.

Table 3 shows point performance for two arbitrary phenotyping decision rules: i) a predicted probability above 0.5 ii) a probability above the threshold maximizing the sum of the precision and the recall. Specificity and negative predictive values are good partly because the diseases are rare at the visit level. Matching the results from figure 3, the sensitivity/positive predictive values trade-off is better for RA than for TB.

We investigated the lack of performance for TB phenotyping by comparing the number of visits with at least one occurrence of main TB CUI among patients with at least one visit labelled TB positive by chart-review. Visits labelled TB negative by chart-review with TB positive past history were significantly more likely to have at least one occurrence of main TB CUI compared to visits labelled TB negative without TB past history (31.0% vs 4.4%, chi-2 p-value = 2.13 x 10 -62 ). Also there was no signifant difference compared to visits with current TB (31.0% vs 29.1%, chi-2 p-value = 0.43).

Figure 2 : Individual prediction of rheumatoid arthritis (RA) and tuberculosis (TB). Each column corresponds to a disease, each row to a patient. Yellow areas corresponds to visits having the disease, white areas corresponds to visits without disease. Each vertical bar corresponds to a visit. Patient 1 has no disease. Patient 2 has RA which is well estimated by PheVis, other algorithms have high variability in their prediction. Patient 3 and 4 have tuberculosis. For patient 3, that information is still trailing behind after the patient is cured for PheVis, partly because of the lack of advanced natural language processing tool hindering the distinction between past and actual disease history. PheNorm predictions are close to 0.5 because the mixture model fails to learn meaningful probabilities in this extremely imbalanced setting. XPRESS fails to estimate any probability higher than 0 for TB because of convergence failure and RA probabilities are either almost 0 or 1 providing a binary interpretation of the disease status rather than a continuous one. Both PL_RF and Anchor are too volatile to provide a trusted interpretation of their output probability.

Conclusions

We developed PheVis as an unsupervised automatic phenotyping algorithm at the visit level. Our innovative approach resembles the human medical probabilistic approach of diagnosis as the output is a probability taking into account the uncertainty of the information inside EHR. [START_REF] Owens | Medical Decision-Making: Probabilistic Medical Reasoning[END_REF] It is able to achieve interesting performances for RA, which is promising for other chronic conditions. While PheVis represent a significant improvement over the current state-of-the-art thanks to its versatile and tunable information accumulating feature, it also suffers from limitations when it comes to acute conditions such as TB. The algorithm is fully not requiring any (time-consuming and expensive) manual chart review, and can in theory be used for different kinds of medical conditions (either acute or chronic). However, the optimal values of hyperparameters might vary depending on the disease of interest and the EHR. In our setting, even if 𝜔 = 5 worked well for both diseases, setting it to 10 or 20 would have increase RA phenotyping performances.

PheVis adds many innovations to the previous PheNorm algorithm it builds upon: the needs for standardizing the information from medical notes and ICD codes, the accumulation of past history with exponential decay, the definition of silver standard using ICD codes to take into account prevalence of the disease, and pseudo-labelling to improve performance and increase stability of predicted probabilities. Also we demonstrated the portability (and limitations) of those methods in French and in a different datawarehouse than the one used to develop PheNorm, with consistent performances for phenotyping RA compared to Yu et al. [START_REF] Yu | Enabling phenotypic big data with PheNorm[END_REF] Our application setting is different from the other methods original paper, mainly because there is intrapatient correlation between visits phenotype. This is not accounted for in Phenorm, XPRESS or POLAR where the learning is at patient level, nor in ANCHOR because it learns on acute diseases. [START_REF] Agarwal | Learning statistical models of phenotypes using noisy labeled training data[END_REF][START_REF] Halpern | Electronic medical record phenotyping using the anchor and learn framework[END_REF][START_REF] Wagholikar | Polar labeling: silver standard algorithm for training disease classifiers[END_REF][START_REF] Yu | Enabling phenotypic big data with PheNorm[END_REF] As in POLAR or ANCHOR, our dataset is largely unbalanced towards the negative class (e.g 1.1% epilepsy prevalence in POLAR, 2.0% septic shock prevalence in ANCHOR) contrary to PheNorm (lowest disease prevalence was RA with 22.5%). This unbalanced setting seems to favor unsupervised learning with silver standard (PheVis, POLAR, ANCHOR, XPRESS) in terms of calibration. In terms of performance, PheVis performed better for both diseases on both AUROC and AUPRC. Random forest POLAR and PheNorm have close performances. As those three methods rely on different approaches, future developments might be able to leverage and combine each of their strengths.

These phenotyping algorithms are highly sensitive to the input features, which emphasizes the need for finer natural language processing tools able to perform semantic analysis. The use of other features such as biological test results or treatment should also be considered, as they should be highly predictive of the phenotype, but further works is needed to define how they could be integrated into the silver-standard surrogate strategy used in PheVis. Also, instead of providing raw variables as input of the algorithm, PheVis could benefit from embedding or tensor factorization approaches that might provide more informative variables but would also increase the complexity for non-expert users. [START_REF] Henderson | Phenotyping through Semi-Supervised Tensor Factorization (PSST)[END_REF][START_REF] Glicksberg | Automated disease cohort selection using word embeddings from Electronic Health Records[END_REF][START_REF] Ho | Limestone: High-throughput candidate phenotype generation via tensor factorization[END_REF] Our performance evaluation is made against an imperfect gold standard, mainly due to the lack of large annotated patient reference sets. For TB, the gold standard was manually curated, while for RA, we used a highly specific form but which might lack sensitivity: interestingly, upon manual inspection it appeared that PheVis was able to accurately recover RA patients visits of 5 patients who were not treated in the Rheumatology department of the University Hospital of Bordeaux and thus had no record of this specific form, resulting in a failure of the gold standard. Such phenomena might underestimate the algorithm performance.

PheVis can provide a probability for a large set of diseases and medical conditions with little effort. The performances might vary depending on the disease of interest, the database quality and the EHR language but were better than state-of-the-art method in our study. The use of those estimated probabilities opens new horizon for the use of EHR for medical and epidemiological research purposes.

' 6 (

 6 68 & &')68 & :('(6(68 & &'))('(68 & ) in order to balance the training set. This new 𝑚𝑎𝑥(10 * , 𝜑) × 𝑃′ matrix is denoted 𝑋 ;

INPUTmatICD:ENDFigure 1 :

 1 Figure 1: Pseudo-code of PheVis.

Figure 3

 3 Figure 3 shows the performance of PheVis and the other methods on the test set for both TB and RA. For RA, PheVis significantly outperforms any other unsupervised method, both in term of AUROC (RA: 0.943 [0.940 ; 0.945], TB: 0.987 [0.983 ; 0.990]) and AUPRC (RA: 0.754 [0.744 ; 0.763], TB: 0.299 [0.198 ; 0.403]).

Figure 3 :

 3 Figure 3: Phenotyping unsupervised and supervised methods performances comparison on the test set. University Hospital of Bordeaux. Confidence Intervals for AUROC and AUPRC are represented by horizontal and vertical segments respectively. PL_RIDGE: POLAR method with ridge logistic regression. PL_RF: POLAR method with random forest. ENET: Supervised elastic-net logistic regression with (CUM) or without (NOCUM) accumulate variables. RF: Supervised random forest with or without accumulate variables.

Table 1

 1 used by rheumatologists at the University Hospital of Bordeaux in usual RA care, for tuberculosis we manually reviewed patients with at least one mention of tuberculosis treatment while other patients were considered not having the disease. Latent tuberculosis was labelled as tuberculosis negative because, even if the bacterium is the same, symptoms, diagnosis and treatment are different. Patients were included in the study cohort if i) they had been hospitalized at the University Hospital of Bordeaux at least once since 2010 and ii) if they had either one primary or secondary ICD code of RA (M05 or M06), or one biological measurement of Anti-Citrullinated Peptide Antibody. The cohort was split into training and test datasets at the patient level with a 70% to 30% ratio. The cohort is described in Table1, highlighting the discrepancy between ICD, CUI and goldstandard justifying the need for automated phenotyping algorithms. Description of phenotyping cohort. University Hospital of Bordeaux.

		Train set	Test set
		Patients	Visits	Patients	Visits
	n	9,102	237,875	2,359	62,004
	Gold standard RA (%)	953 (10.5) 27,077 (11.4)	274 (11.6)	7,883 (12.7)
	ICD RA 1 ≥ 1 (%)	3,682 (40.5) 21,448 (9.0)	901 (38.2)	5,823 (9.4)
	CUI RA ≥ 1 (%)	3,703 (40.7) 32,775 (13.8)	952 (40.4)	8,632 (13.9)
	Gold standard TB (%)	49 (0.5)	618 (0.3)	5 (0.2)	90 (0.1)
	ICD TB 2 ≥ 1 (%)	88 (1.0)	277 (0.1)	15 (0.6)	50 (0.1)
	CUI TB ≥ 1 (%)	647 (7.1)	2,393 (1.0)	147 (6.2)	439 (0.7)
	1: ICD RA: M05, M06
	2: ICD TB: A15, A16, A17, A18, A19

Table 2 :

 2 Phenotyping unsupervised and supervised methods comparison. University Hospital of Bordeaux.

		Test set		Train set
	Algorithm	AUROC	AUPRC	AUROC	AUPRC
	Rheumatoid Arthritis
	ANCHOR	0.845 [0.839 ; 0.850] 0.578 [0.565 ; 0.590] 0.827 [0.824 ; 0.830] 0.481 [0.476 ; 0.487]
	ENET_CUM	0.957 [0.954 ; 0.961] 0.913 [0.908 ; 0.918] 0.986 [0.985 ; 0.987] 0.953 [0.951 ; 0.955]
	ENET_NOCUM	0.875 [0.871 ; 0.880] 0.717 [0.706 ; 0.728] 0.865 [0.862 ; 0.867] 0.687 [0.682 ; 0.692]
	PHENORM	0.870 [0.865 ; 0.875] 0.625 [0.612 ; 0.637] 0.855 [0.852 ; 0.857] 0.577 [0.571 ; 0.584]
	PHEVIS F/5/D	0.942 [0.939 ; 0.944] 0.759 [0.750 ; 0.768] 0.659 [0.656 ; 0.662] 0.424 [0.418 ; 0.429]
	PHEVIS T/1/D	0.929 [0.926 ; 0.932] 0.708 [0.698 ; 0.718] 0.933 [0.932 ; 0.934] 0.676 [0.671 ; 0.681]
	PHEVIS T/10/D	0.951 [0.948 ; 0.953] 0.772 [0.762 ; 0.782] 0.951 [0.950 ; 0.952] 0.745 [0.739 ; 0.750]
	PHEVIS T/2/D	0.946 [0.944 ; 0.948] 0.736 [0.727 ; 0.746] 0.944 [0.943 ; 0.945] 0.689 [0.684 ; 0.694]
	PHEVIS T/20/D	0.957 [0.955 ; 0.959] 0.798 [0.789 ; 0.806] 0.951 [0.950 ; 0.952] 0.765 [0.760 ; 0.770]
	PHEVIS T/5/0	0.900 [0.896 ; 0.904] 0.571 [0.559 ; 0.582] 0.898 [0.896 ; 0.900] 0.524 [0.518 ; 0.529]
	PHEVIS T/5/D	0.943 [0.940 ; 0.945] 0.754 [0.744 ; 0.763] 0.943 [0.942 ; 0.944] 0.717 [0.712 ; 0.722]
	PL_RF	0.825 [0.819 ; 0.831] 0.616 [0.605 ; 0.626] 0.813 [0.810 ; 0.816] 0.546 [0.540 ; 0.552]
	PL_RIDGE	0.832 [0.826 ; 0.837] 0.533 [0.521 ; 0.544] 0.821 [0.818 ; 0.824] 0.496 [0.490 ; 0.502]
	RF_CUM	0.994 [0.993 ; 0.995] 0.975 [0.973 ; 0.977] 0.999 [0.999 ; 1.000] 0.999 [0.998 ; 0.999]
	RF_NOCUM	0.849 [0.844 ; 0.855] 0.742 [0.733 ; 0.750] 0.845 [0.843 ; 0.848] 0.729 [0.724 ; 0.734]
	XPRESS	0.700 [0.695 ; 0.705] 0.387 [0.376 ; 0.397] 0.693 [0.690 ; 0.695] 0.345 [0.339 ; 0.350]
	Tuberculosis
	ANCHOR	0.745 [0.687 ; 0.798] 0.075 [0.037 ; 0.134] 0.635 [0.608 ; 0.660] 0.128 [0.100 ; 0.155]
	ENET_CUM	0.910 [0.873 ; 0.944] 0.262 [0.163 ; 0.356] 0.928 [0.914 ; 0.941] 0.451 [0.405 ; 0.488]
	ENET_NOCUM	0.677 [0.631 ; 0.725] 0.225 [0.136 ; 0.319] 0.667 [0.650 ; 0.685] 0.188 [0.152 ; 0.222]
	PHENORM	0.722 [0.663 ; 0.789] 0.209 [0.124 ; 0.304] 0.682 [0.658 ; 0.707] 0.097 [0.075 ; 0.122]
	PHEVIS F/5/D	0.987 [0.983 ; 0.991] 0.309 [0.204 ; 0.411] 0.729 [0.711 ; 0.747] 0.157 [0.131 ; 0.185]
	PHEVIS T/1/D	0.986 [0.982 ; 0.990] 0.248 [0.167 ; 0.334] 0.951 [0.940 ; 0.960] 0.191 [0.164 ; 0.220]
	PHEVIS T/10/D	0.987 [0.984 ; 0.991] 0.282 [0.185 ; 0.382] 0.844 [0.823 ; 0.862] 0.153 [0.128 ; 0.176]
	PHEVIS T/2/D	0.987 [0.983 ; 0.991] 0.249 [0.168 ; 0.335] 0.847 [0.827 ; 0.865] 0.177 [0.149 ; 0.205]
	PHEVIS T/20/D	0.697 [0.651 ; 0.751] 0.158 [0.084 ; 0.254] 0.646 [0.628 ; 0.662] 0.069 [0.052 ; 0.089]
	PHEVIS T/5/0	0.757 [0.704 ; 0.806] 0.139 [0.080 ; 0.218] 0.506 [0.485 ; 0.526] 0.003 [0.002 ; 0.003]
	PHEVIS T/5/D	0.987 [0.983 ; 0.990] 0.299 [0.198 ; 0.403] 0.853 [0.834 ; 0.870] 0.191 [0.164 ; 0.216]
	PL_RF	0.710 [0.646 ; 0.768] 0.249 [0.160 ; 0.338] 0.634 [0.609 ; 0.661] 0.149 [0.120 ; 0.179]
	PL_RIDGE	0.681 [0.622 ; 0.737] 0.011 [0.006 ; 0.021] 0.598 [0.573 ; 0.627] 0.013 [0.010 ; 0.018]
	RF_CUM	0.856 [0.807 ; 0.898] 0.475 [0.366 ; 0.576] 0.985 [0.977 ; 0.991] 0.942 [0.923 ; 0.958]
	RF_NOCUM	0.690 [0.640 ; 0.744] 0.278 [0.183 ; 0.374] 0.672 [0.656 ; 0.690] 0.232 [0.198 ; 0.269]
	XPRESS	0.500 [0.500 ; 0.500] 0.001 [0.001 ; 0.002] 0.500 [0.500 ; 0.500] 0.003 [0.002 ; 0.003]
	Bold: median AUROC or AUPRC superior to PHEVIS T/5/D
	PHEVIS Pseudo-labels/ ω /Half-life. Pseudo-labels is True (T) or False (F). ω is the constant defining 𝑞𝑢𝑎𝑛𝑡 !"#$!%! . Half-life is 0
	or the disease duration (D), 180 for TB and +∞ for RA

PL_RIDGE: POLAR method with ridge logistic regression. PL_RF: POLAR method with random forest. ENET: Supervised elastic-net logistic regression with (CUM) or without (NOCUM) cumulated accumulate variables. RF: Supervised random forest with or without cumulated accumulate variables.

Table 3

 3 PheVis performance on the test set for different thresholds of the output probabilities.

	Disease	Threshold	SE	SP	PPV	NPV
	Rheumatoid arthritis 0.5	0.740	0.942	0.651	0.961
		Optimal P-R* (0.322)	0.761	0.936	0.633	0.964
	Tuberculosis	0.5	0.300	1.000	0.519	0.999
		Optimal P-R* (1.30 10 -9 )	0.989	0.945	0.025	1.000
	* Threshold maximizing the precision recall sum.
	Se: sensitivity -Sp: specificity -PPV: positive predictive value -NPV: negative predictive value

Supplementary material

ICD10 codes

Algorithm performance

Competing Interests

None.

Highlights

• Electronic Health Record phenotyping is challenging especially at the visit level.

• PheVis is a new unsupervised approach extending PheNorm to visit level.

• Incorporating accumulated features to take into account disease dynamic increase model performances.

• PheVis outperforms other phenotyping algorithms at the visit level.

Graphical abstract Bibliography