Merci À Alain

Ibrahim Anastasiia

Maryan, Peng, Phong, Qing, Ruihua Marcello Martin

Angel

Dian Daniel

Firas

Youcef Philip

Côme

Houzhi, Hoang Guillaume S Enzo

Nissrine William ; Merci À Assaf

Guillaume Cyril

Sothea ; À Aaraona

Clément Bogdan Fabio

Ottavio, Sylvain Nathan Mohan

Azar, Benjamin Abdel

Barbara

Ziad Marc

Je Remercie Nathalie

Valérie

T Brown

B Mann

N Ryder

M Subbiah

J D Kaplan

P Dhariwal

A Neelakan- Tan

P Shyam

G Sastry

A Askell

S Agarwal

A Herbert-Voss

G Krueger

T Henighan

R Child

A Ramesh

D Ziegler

J Wu

C Winter

C Hesse

M Chen

E Sigler

M Litwin

S Gray

B Chess

J Clark

C Berner

S Mccan- Dlish

A Radford

I Sutskever

D Amodei

Keywords: Machine learning, Deep learning, Medical claims data, Contrastive learning, Random forests, Online logistic regression vi

Developing efficient algorithms to learn appropriate representations of structured data, including sequences of timestamped events, is a major and central challenge in machine learning. To this end, deep learning has become popular in modeling structured data, sometimes combined with some pre-training techniques. At the same time, classical machine learning methods, such as random forest and regression, always have an important place because of their efficiency.

In this thesis, we make some contributions to the theoretical and numerical studies of some machine learning problems, as well as the application of deep learning on the public health data.

The first contribution introduces a new model ZiMM (Zero-inflated Mixture of Multinomial distributions), and an Encoder-Decoder (ED) end-to-end deep neural networks architecture, modelling the healthcare pathways for post-surgical complications prediction. ZiMM-ED is applied on the healthcare claims data coming from the French national system of health data (SNDS), which is a non-clinical database, only containing timestamped reimbursement codes for drugs purchases, medical procedures and hospital diagnoses. In particular, we consider the complication until the 18-th month after the surgery, which is also blurry since we only observe this from the purchase of drugs from a specific family. Our experiments show that ZiMM-ED improves several baselines in terms of prediction performance. At the same time, ZiMM-ED paves the way for leveraging such a dataset with little pre-processing, using deep neural networks. Hence, we take advantage of a database which is until now only used for administrative ends (healthcare expense reimbursement), and we show the predictive power of the deep neural networks on it with a precise application.

The second contribution focuses on the theoretical study of contrastive representation learning, a technique recently becoming popular and experimentally shown to be efficient for self-supervised learning. Relying on some previous works introducing the theoretical framework, we extend the guarantees for the quality of the representations learned during the pre-training phase, with the contrastive loss and multiple negative samples, the quality being measured by the predictive performance of some down-stream supervised tasks. Furthermore, we provide a convergence guarantee for the minimization of the contrastive training error with gradient descent of an over-parametrized neural network encoder. These theoretical results, combined with some illustrative experiments, open doors for a better understanding of the typical pre-training -fine-tuning practices.

Next, the third contribution consists of introducing a new algorithm of type random forest, that we name WildWood. While the standard random forest algorithm uses bootstrap out-of-bag samples to compute out-of-bag score, WildWood uses these samples to produce improved predictions given by an aggregation of all the possible sub-trees for each fully-grown tree in the forest: this computation is precise and efficient thanks to the context tree weighting algorithm. We show that theoretically, the loss of such an aggregation is comparable to the loss of the best possible subtree. We propose an open-source Python implementation of WildWood with the histogram strategy which accelerates split finding. Our implementation is fast and competitive compared to other well-established ensemble methods, such as standard random forest and extreme gradient boosting algorithms.

Finally, the last chapter of this thesis is devoted to the problem of online logistic regression and considers the regret with respect to the 2 -ball of radius B. While it is known that proper algorithms with a regret logarithmic in the number of rounds n necessarily suffer from an exponential factor in B, some improper algorithms, Bayesian and non-Bayesian, were recently introduced with better regret guarantees. For the purpose of obtaining an optimal regret guarantee, we introduce two improper non-Bayesian algorithms, OSMP and AOSMP, based on the one-step minmax strategy, with the exact loss function for OSMP, and an approximated loss function for AOSMP. Our regret analysis relies, among others, on the generalized self-concordance property of the logistic function. For OSMP, although we find an upper-bound for the instant regrets, we explain why obtaining improved regret upper-bounds is a difficult question, to which AOSMP brings an answer similar to the state-of-the-art one.

Résumé

Le développement d'algorithmes efficaces pour apprendre des représentations appropriées de données structurées, telles que des séquences d'événements datés provenant de la vie réelle, est un défi majeur et central de l'apprentissage automatique. Dans cette optique, l'apprentissage profond est devenu populaire pour modéliser des données structurées, parfois combiné avec des techniques de pré-entraînement. En même temps, d'autres méthodes d'apprentissage statistique plus "classiques", comme les forêts aléatoires ou la régression, occupent toujours une place importante dans la pratique à cause de leur efficacité.

Dans cette thèse, nous apportons quelques contributions à l'étude théorique et numérique de certains problèmes de l'apprentissage statistique, ainsi que l'application de l'apprentissage profond aux données de la santé publique.

La première contribution consiste à introduire un nouveau modèle appelé ZiMM (Zero-inflated Mixture of Multinomial distributions), et une architecture Encodeur-Décodeur (ED) de réseaux de neurones profonds entraînés de-bout-en-bout, modélisant les parcours de soins pour la prédiction des complications post-chirurgicales. ZiMM-ED est appliqué aux données de santé de remboursement de soins provenant du Système National des Données de Santé (SNDS) en France, qui est une base de données non-clinique, contenant seulement les codes de remboursement datés d'achats de médicaments et des diagnostics hospitaliers. En particulier, nous considérons les complications jusqu'au 18e mois après la chirurgie, ce qui correspond à des observations "floues" car seulement observées à partir des achats de médicaments d'une famille spécifique. Nos expériences montrent les améliorations en termes de performance prédictive de ZiMM-ED par rapport à plusieurs modèles de référence. ZiMM-ED ouvre la voie de l'exploitation d'un tel jeu de données avec peu de prétraitement à grâce aux réseaux de neurones profonds. Cette base de données est jusque-là utilisée principalement pour des raisons administratives (remboursement des soins de santé), et nous montrons le pouvoir prédictif des réseaux de neurones profonds dessus sur une telle base de données avec un cas précis.

La deuxième contribution porte sur l'étude théorique de l'apprentissage contrastif de représentation, une technique récemment devenue populaire et expérimentalement efficace pour l'entraînement auto-supervisé. En se basant sur quelques résultats proposant des cadres d'étude théoriques, nous étendons la garantie pour la qualité des représentations apprises dans la phase pré-entrainement non-supervisé avec une perte contrastive et de multiples échantillons négatifs, la qualité étant mesurée en termes de performance prédictive pour les tâches supervisées en aval. En outre, nous fournissons une garantie de convergence quant à la minimisation de la perte contrastive avec la descente de gradient pour un encodeur de réseaux de neurones sur-paramétré. Ces résultats théoriques, combinant des expériences numériques, ouvrent des portes pour une meilleure compréhension des pratiques de pré-entrainement -affinement très utilisées aujourd'hui en apprentissage profond.

La troisième contribution consiste à introduire un nouvel algorithme de type forêt aléatoire, que nous nommons WildWood. Alors que l'algorithme standard de forêt aléatoire utilise des échantillons bootstrap out-of-bag seulement pour calculer des scores, WildWood utilise ces échantillons pour améliorer les prédictions en calculant l'agrégation de tous les sous-arbres possibles de chaque arbre dans la forêt : ce calcul est exact et efficace grâce à l'algorithme de context tree weighting. Nous montrons que théoriquement, la perte induite par une telle agrégation est comparable à celle du meilleur sous-arbre possible. Nous proposons une implémentation Python open-source de WildWood avec une stratégie d'histogramme qui permet d'accélérer la recherche des coupures impliquées dans la construction des arbres. Notre implémentation est rapide et compétitive en comparaison avec d'autres algorithmes ensemblistes bien connus, par exemple la forêt aléatoire standard et les algorithmes d'extrême gradient boosting.

Enfin, le dernier chapitre de cette thèse est consacré à la régression logistique en ligne et considère le regret par rapport à la boule 2 de rayon B. Alors qu'il est connu que les algorithmes propres avec regret logarithmique en le nombre d'itérations n subissent nécessairement un facteur exponentiel en B dans leur borne de regret, quelques algorithmes impropres, bayésiens et non-bayésiens, ont été introduits récemment avec des meilleures garanties. Dans le but d'obtenir une garantie de regret optimale, nous proposons deux algorithmes impropres et non-bayésiens, OSMP et AOSMP, reposant sur une stratégie "minmax à une étape", avec la fonction de perte exacte pour OSMP, et une fonction de perte approchée pour AOSMP. Nos analyses de regret s'appuient entre autres sur la propriété de self-concordance généralisée de la fonction logistique. Pour OSMP, malgré une borne supérieure obtenue pour les regrets instantanés, nous expliquons en quoi l'amélioration des bornes de regret est une question difficile, à laquelle AOSMP apporte une réponse comparable à l'état de l'art de la garantie de regret.

Mots clefs :

Apprentissage statistique, Apprentissage profond, Données de santé, Apprentissage contrastif, Forêts aléatoires, Régression logistique en ligne

Résumé détaillé en français

Cette thèse traite du sujet général de l'apprentissage automatique (Machine learning), un cadre général pour le problème de l'extraction d'informations et de la prise de décisions à partir de données. Alors que la dernière décennie a vu le renouveau des réseaux de neurones, rebaptisés apprentissage profond (Deep learning), et ses nombreuses applications réussies dans le monde réel grâce à des jeux de données plus importants et à des capacités de calcul plus fortes, la recherche méthodologique en apprentissage automatique est plus que jamais cruciale pour le progrès d'un tel domaine, permettant la conception de nouveaux algorithmes avec des propriétés souhaitables ainsi que la compréhension plus profonde de certains succès pratiques.

Dans cette thèse, j'ai travaillé sur les applications de l'apprentissage profond sur des données de santé publique, ainsi que sur des études théoriques et numériques de certains problèmes d'apprentissage automatique. La première contribution consiste à exploiter les données de santé de remboursement de soins, disponibles dans le Système National des Données de Santé (SNDS), en utilisant l'apprentissage profond (Chapitre 2). Ensuite, les contributions se concentrent sur les thèmes de l'apprentissage contrastif (Chapitre 3), les méthodes basées sur les arbres d'ensemble (Chapitre 4), et la régression logistique en ligne (Chapitre 5).

Ce résumé en français presénte dans l'ordre chaque contribution de façon succincte.

Chapitre 2 : ZiMM, un modèle d'apprentissage profond pour les complications à long terme et floues avec des données de santé de remboursement de soins non-cliniques

Le travail présenté dans le Chapitre 2 est une tentative de relever certains défis dans l'application de l'apprentissage profond aux données de santé (EHRs ou celles de remboursement), sous forme d'une étude d'un cas médical concret.

Nous considérons les problèmes de modélisation et de prédiction d'une complication à long terme et "floue" qui se produit après une procédure médicale telle qu'une chirurgie. Il ne s'agit pas ici d'une complication à court terme liée à la procédure elle-même, mais d'une complication à long terme que les cliniciens ne peuvent pas expliquer facilement, car elle dépend de séquences inconnues d'événements passés qui se sont produits avant la procédure. La complication n'est observée qu'indirectement, de manière "floue", par le biais de prescriptions longitudinales de médicaments sur une longue période de temps après la procédure médicale. Notre motivation pour utiliser le SNDS, une base de données de réclamations médicales non cliniques, est son exhaustivité au niveau de la population nationale. En effet, nous considérons un jeu de données extraites contenant les remboursements de soins médicaux de presque tous les résidents français ayant subi une intervention chirurgicale pour des problèmes prostatiques, avec un historique entre 1,5 et 5 ans.

Pour mieux saisir les complications à long terme et floues, nous introduisons le modèle ZiMM (Zero-inflated Mixture of Multinomial distributions model). Désignons par n i = B b=1 y i,b le nombre total de complications floues du patient i, avec B le nombre de laps de temps post-chirurgical que nous considérons (18 mois en occurrence). Nous supposons que n i ∈ {0, 1, . . . , B} est distribué (conditionnellement à x i ) comme n i ∼ Categorical(π 0 (x i ), π 1 (x i ), . . . , π B (x i )), où π b (x i ) sont tels que B b=0 π b (x i ) = 1 et π b (x i ) 0 (sortant d'une activation softmax par exemple). Ces paramètres correspondent à la distribution catégorielle propre au patient i. Ensuite, on suppose que la distribution de y i conditionnelle à n i = b et x i suit soit une distribution de Dirac sur le vecteur (de taille B) [0, . . . , 0] lorsque b = 0, soit une distribution multinomiale de paramètres b et p b,1 (x i ), . . . , p b,B (x i ), à savoir

y i |(x i , n i = b) ∼ δ [0,...,0]
si b = 0, Multinomial(b, p b,1 (x i ), . . . , p b,B (x i )) sinon, où p b,1 (x i ), . . . , p b,B (x i ) sont les paramètres d'une distribution multinomiale lorsque n i = b, pour chaque b = 1, . . . , B. Une fois encore, ces paramètres sont spécifiques au patient i. En plus de cette modélisation probabiliste des complications post-chirurgicales, nous construisons une architecture d'apprentissage profond de-bout-en-bout, appelée ZiMM Encodeur-Decodeur (ZiMM ED), capable d'apprendre à partir des modèles complexes, irréguliers, hétérogènes et épars d'événements de santé observés dans une base de données de remboursements de soins uniquement. Les composantes de ZiMM ED incluent embedding des codes médicaux, mécanisme d'attention pour agréger au sein d'une même fenêtre de temps, Long Short Term Memory, concaténation avec les embeddings issues des features statics, comme l'illustre la Figure 1.

Nos expériences montrent que ZiMM ED améliore plusieurs modèles prédictifs de comparaison, y compris les approches d'apprentissage classiques et d'apprentissage profond, et qu'il permet de travailler sur un tel ensemble de données avec un minimum de travail de prétraitement.

Chapitre 3 : Garanties théoriques sur l'apprentissage contrastif non supervisé de représentation pour la classification et sur sa convergence La motivation du travail présenté dans le Chapitre 3 est de mieux comprendre les techniques d'entraînement non-supervisées et auto-supervisées récemment populaires, du point de vue de la modélisation.

En suivant le cadre proposé pour la première fois dans Saunshi et al. [2019], supposons que nous sommes capables d'échantillonner des paires positives (x, x + ) à partir de la distribution

D sim (x, x + ) = c∈C ρ(c)D c (x)D c (x + ), (1)
à savoir, (x, x + ) est échantillonné comme un mélange de paires indépendantes conditionnellement à une classe latente partagée, échantillonnée selon ρ. D'autre part, nous supposons que nous pouvons échantillonner négatif des échantillons x -à partir de la distribution

D neg (x -) = c∈C ρ(c)D c (x -). (2)
Sous les hypothèses données dans les équations (1.1) et (1.2), nous considérons la perte contrastive non supervisée suivante avec N échantillons négatifs,

L N un (f ) = E (x,x + )∼D sim X -∼D ⊗N neg   -log   exp f (x) f (x + ) exp (f (x) f (x + )) + x -∈X -exp (f (x) f (x -))     ,
(3) où D ⊗N neg représente le produit tensoriel N de la distribution D neg donnée par l'équation (2).

Proposition (Proposition 3.3). Considérons la perte non supervisée L N un (f ) de l'équation (1.3) avec N échantillons négatifs. Supposons que ρ est uniforme sur C et que 2 k + 1 N C . Alors, pour toute fonction encodeur f : X → R d , nous avons

L sup,k (f ) L µ sup,k (f ) k 1 -τ + N L N un (f ) -τ + N log(N + 1) avec τ + N = P c i = c, ∀i | (c, c 1 , • • • , c N ) ∼ ρ ⊗N +1 .
Ces résultats sont valables pour un grand nombre arbitraire de négatifs et sont découplés du nombre de tâches de classification. Ainsi, nous fournissons de nouvelles garanties théoriques pour la performance de classification des modèles entraînés de manière contrastive dans le cas de tâches de classification multivoie, en utilisant multiple échantillons négatifs. Nous étendons les résultats de Saunshi et al. [2019] pour montrer que la performance de l'entraînement non supervisé se reflète sur une tâche de classification ultérieure dans le cas de tâches multiples et lorsqu'un nombre élevé d'échantillons négatifs est utilisé.

Nous fournissons également un résultat de convergence (Théorème 3.5) pour un algorithme explicite (descente de gradient), lors de l'entraînement d'un réseau de neurones sur-paramétré pour l'apprentissage non supervisé de représentations contrastives.

Chapitre 4 : WildWood, un nouvel algorithme de forêt aléatoire Dans le travail présenté dans le Chapitre 4, nous proposons WildWood, qui conserve tous les ingrédients de la forêt aléatoire (Random Forest, RF), y compris le bootstrap, le sous-échantillonnage des features, une procédure similaire de croissance des arbres, avec un prédicteur d'arbres amélioré et des calculs efficaces implémentés en Python.

Considérons un arbre T entièrement développé. Rappelons qu'en RF standard, le prédicteur d'arbre correspondant à T s'écrit f (x) = y Cv(x) pour tout x ∈ X , où C(x) est la cellule contenant x associée à la feuille v : il s'agit de trouver la plus petite cellule contenant x, et de produire la prédiction de cette cellule. Dans WildWood, nous améliorons ce prédicteur d'arbre par le mécanisme suivant : comme prédicteur pour T , WildWood utilise

f (•) = T ⊂T π(T )e -ηL T y T (•) T ⊂T π(T )e -ηL T avec π(T ) = 2 -T , ( 4
)
où la somme est calculée sur tous les sous-arbres T de T enracinés à root, η > 0 est le paramètre de température, T désigne le nombre de noeuds dans T moins son nombre de feuilles qui sont également des feuilles de T , L T est la perte cumulative de la prédiction du sous-arbre T sur oob échantillons, i.e. L T := i∈Ioob ( y T (x i ), y i ), et y T est la prédiction du sous-arbre T de manière classique. La fonction de prédiction (4) peut être vue comme une agrégation des prédictions y T (•) de tous les sous-arbres T , pondérées par leurs performances sur oob échantillons avec un prior π(T ) = 2 -T . Il s'agit en effet d'une manière non-gloutonne d'élaguer les arbres : les poids ne dépendent pas seulement de la qualité d'une seule coupure, mais aussi de la performance de chacune des coupures successives. En plus, le théorème suivant donne une garantie théorique sur les échantillons oob, pour l'agrégation de sous-arbres, sous une hypothèse sur l'exp-concavité de la fonction de perte. Ce théorème stipule que le prédicteur donné par (4) est capable de performer presque aussi bien que le meilleur sous-arbre oracle T ⊆ T sur les échantillons oob, avec un taux de O( T /n oob ) qui est optimal pour les inégalités oracle de sélection de modèle [Tsybakov, 2003] ; en comparaison, trouver un oracle argmin T ∈T i∈Ioob ( f (x i ), y i ) est infaisable sur le plan computationnel, ce qui nécessite d'essayer tous les sous-arbres.

Theorem (Inégalité Oracle, Théorème 4.2 reformulé). Supposons que la fonction de perte soit η-exp-concave. Alors, la fonction de prédiction f donnée par (4) satisfait l'inégalité oracle

1 n oob i∈Ioob ( f (x i ), y i ) inf T ⊂T 1 n oob i∈Ioob ( y T (x i ), y i ) + C T η(n oob + 1)
, où l'infimum est pris sur tous les sous-arbres T ⊂ T enracinés à root, et C = log 2.

Une conséquence de ce théorème est un calcul efficace de f (x), en exploitant le fait que les arbres dans WildWood sont développés en profondeur. Le calcul de f (x) n'augmente en effet que d'un facteur 2 la complexité de calcul d'un RF standard.

En plus de ce calcul efficace de l'agrégation de sous-arbres, WildWood supporte nativement les caractéristiques catégorielles et implémente une stratégie d'histogramme, pour accélérer la recherche de fractionnement. Notre implémentation de WildWood est disponible sur le répertoire GitHub https://github.com/pyensemble/ wildwood.

Nous avons fait des expériences extensives pour évaluer les performances prédictives et les temps d'exécution de notre implémentation de WildWood, voir les tableaux 4.1 et 4.2 du Chapitre 4 de ce manuscrit pour les paramètres et les résultats détaillés des expériences. Toutes les expériences peuvent être reproduites à l'aide de scripts Python sur le même répertoire. Dans l'ensemble, en tant qu'un algorithme Random Forest amélioré, WildWood a montré des performances compétitives en termes de pouvoir prédictif et de temps d'exécution, par rapport aux algorithmes Random Forest standard et Extreme Gradient Boosting.

Chapitre 5 : Régression logistique en ligne, vers un algorithme efficace avec une meilleure garantie de regret

Le point de départ du travail présenté dans le Chapitre 5 est la question suivante : avec éventuellement quelques adaptations, une version en ligne du SMP [Mourtada and Gaïffas, 2019] pourrait-elle atteindre une garantie de regret comparable à l'excès de risque du SMP ? Nous présentons deux algorithmes candidats, OSMP et AOSMP, ainsi que leur analyse du regret.

Commençons par considérer le regret λ-ridge pénalisé jusqu'au temps t pour la régression logistique en ligne, qui s'écrit Nous introduisons le One-Step Minmax Predictor (OSMP), qui prend le "meilleur choix possible" de prédiction y t contre "le pire choix possible" de la vraie valeur y t ∈ {-1, +1} et du paramètre de l'adversaire θ, mesuré par le regret λ-ridge pénalisé ci-dessus, à savoir

y t = argmin ŷ∈R sup yt∈{-1,1} sup θ∈R d
( y, y t ) + L t-1 -(θ x t , y t ) + L λ,t-1 (θ) , avec L t-1 := t-1 s=1 ( y s , y s ), et L λ,t-1 (θ) := t-1 s=1 (θ x s , y s ) + λ θ 2 . Nous montrons que de façon équivalente (Lemme 5.4), l'OSMP prédit

y t = -L +1 λ,t + L -1 λ,t
avec L y λ,t := inf θ∈R d (θ x t , y) + L λ,t-1 (θ) pour y ∈ {-1, +1}. Cela donne un algorithme explicite (l'Algorithme 1) pour le calcul de l'OSMP. Concernant l'analyse du regret de l'OSMP, nous écrivons le regret de la façon suivante Regret n (θ) = n t=1 ( y t , y t ) -n t=1 (θ x t , y t ) λ θ 2 + n t=1 r t , où nous désignons r t := ( y t , y t ) -L λ,t + L λ,t-1 le regret instantané du temps t. Nous déduisons la borne supérieure suivante

r t e • σ ( θ t , x t ) • x t 2 (∇ 2 L λ,t (θt)) -1
dont la preuve repose sur la self-concordance généralisée [START_REF] Bach | Self-concordant analysis for logistic regression[END_REF] de la fonction L λ,t . Cependant, notre analyse de regret pour l'OSMP s'arrête là. En particulier, la valeur de la matrice hessienne ∇2 L λ,t (θ t ) dépend du point θ t où elle est évaluée, et nous ne connaissons pas de moyen de contrôler une telle matrice hessienne conduisant à une borne supérieure de regret qui est logarithmique en n et sans facteur exponentiel en BR, voir la Section 5.5.3 pour plus de détails.

Pour contourner cette difficulté, nous introduisons le Approximated One-Step Minmax Predictor (AOSMP) suivant, défini par

y t = argmin y∈R sup yt∈{-1,+1}
sup θ∈R d L t-1 + ( y, y t ) -(θ x t , y t ) + Lt-1 (θ) + λ θ 2 avec Lλ,t-1 (θ) := t-1 s=1 ˜ (θ x s , y s ) + λ θ 2 . Cette formulation étant dans le même esprit minmax que l'OSMP, la différence est que dans l'AOSMP, nous remplaçons la fonction L t-1 par son approximateur quadratique Lt-1 . Nous nous appuyons sur l'approximation quadratique de la fonction logistique introduite pour la première fois dans Jézéquel et Pour les mêmes raisons que pour l'OSMP, l'AOSMP prédit

y t = -L+1 λ,t + L-1 λ,t
avec Ly λ,t := inf θ∈R d (θ x t , y) + Lλ,t-1 (θ) pour y ∈ {-1, +1}, et cela donne un algorithme explicite (Algorithme 2). Ensuite, nous sommes en mesure d'appliquer les techniques classiques avec les formes quadratiques pour borner la somme des pseudo regrets instantanés, pour obtenir la borne suivante (Theorème 5. 19) sur le regret de l'AOSMP Regret n (θ) e • (1 + BR)d log 1 + nR 2 8d(1 + BR)λ + λ θ 2 .

En particulier, le choix λ = R 2 donne

Regret n e • (1 + BR)d log 1 + n 8d(1 + BR)

+ B 2 R 2 .
Ainsi nous obtenons pour l'AOSMP une borne supérieure de regret similaire à celle d'AIOLI [Jézéquel et al., 2020], en partie parce que nous utilisons la même approximation quadratique. Globalement, dans ce travail, nous proposons et analysons deux algorithmes inspirés du SMP [Mourtada and Gaïffas, 2019] This thesis deals with the general topic of machine learning, a general framework for the problem of retrieving information and making decisions from data. While the recent decade has seen the revival of neural networks re-branded as deep learning [START_REF] Lecun | Deep learning[END_REF] and its many successful real-world applications with the help of larger datasets and stronger computational capacities, methodological research in machine learning is more than ever crucial to the progress of such a field, enabling the design of new algorithms with desirable properties as well as the deeper understanding of some practical success.

In this thesis, I worked on the applications of deep learning on public health data, and also on theoretical and numerical studies of some machine learning problems. The first contribution consists of leveraging the medical claims data available in French national system of health data using deep learning (Chapter 2). Then, the contributions are focused on the topics of contrastive learning (Chapter 3), ensemble tree-based methods (Chapter 4), and online logistic regression (Chapter 5).

This introduction chapter gives a quick overview for each of these works. In the rest of this opening chapter, we give a presentation of each of the problems that appears in this thesis, as well as their background and some necessary tools, followed by a brief summary of the contributions of the present thesis.

Deep Learning and applications in healthcare

Healthcare is an important application area of artificial intelligence and machine learning, with profound consequence and significant potential benefice for social good [Bommasani et al., 2021;[START_REF] Villani | Donner un sens à l'intelligence artificielle : pour une stratégie nationale et européenne[END_REF]. At the same time, developing efficient algorithms to learn appropriate representations of structured data, including sequences of timestamped events, is a major yet challenging problem in machine learning. As deep learning has shown impressive performance in many application domains (Computer Vision [Chen et

Large observational databases in healthcare: EHRs, medical claims data

In the domain of healthcare, with a fast adaptation towards information systems in the past decades, it became possible to collect and store large amounts of healthcare data, which are rich in information and sensitive by their nature. These large amounts of patient-centered data were made available through some different stakeholders in the domain of healthcare providers: clinics and hospitals, insurance companies, public institutions. This broad kind of large observational databases are the type of data that we are interested in this work. We can roughly distinguish between two categories of healthcare data that are acquired by re-purposing administrative ones: Electronic Health Records (EHRs) and medical claims data. These two types of data share lots of similarities with some specificities [FDA, 2021], while sometime confusions are made in using the two terms.

Under the name of EHRs are generally designed data directly produced by healthcare providers, such as hospitals and clinics, for clinical care support and billing purpose. EHRs are often fine-grained, including demographic information such as gender, age, location, sometimes also patients' living habits such as smoking status, alcohol consumption and medical history. Most importantly, EHRs often come with exact information on provided cares, such as drug prescriptions, medical procedures, diagnoses, bed-side vital signs in case of Intensive Care Units (ICU), sometimes including also free-text medical reports, laboratory analysis including for example imaging and imaging reports, and other clinical observations and reports. A freely accessible database of ICU data and hospital information MIMIC-III [START_REF] Johnson | MIMIC-III, a freely accessible critical care database[END_REF] is often seen as a reference for EHRs, in particular for benchmarking works [Bellamy et al., 2020;[START_REF] Harutyunyan | Multitask learning and benchmarking with clinical time series data[END_REF]. The recently released successor MIMIC-IV [START_REF] Johnson | MIMIC-IV" (version 0.4)[END_REF] includes more complex and more complete data such as emergency department data, chest X-Ray images and free-text clinical notes.

Medical claims data are collected by insurance companies or agencies (thus indirectly from healthcare providers), with primarily administrative purpose of reimbursement. Medical claims data are often in form of timestamped events, i.e. drug prescriptions, medical procedures, hospital diagnoses. While medical claims data share many similar characteristics with EHRs, and often have the advantage of covering a larger number of population across numerous hospitals or healthcare providers, medical claims data often contain fewer details and may be less accurate than EHRs. For example, bed-side vital signs could rarely be included in medical claims data; the exam results of medical imaging is not contained, but the code for this imaging with indications on the body part is included in claims data (since this is required for reimbursement).

In France, the SNDS (National Health Data System -Système National des Données de Santé, formerly known as SNIIR-AM -National Health Insurance Information System -Système National d'Information Inter-Régimes), is an example of medical claims data [Tuppin et al., 2017]. It includes healthcare reimbursement records for nearly all French residents. Through the partnership between Caisse Nationale de l'Assurance Maladie (the French agency managing the national health insurance system -CNAM), and Data Science Initiative from École polytechnique, we had the opportunity to have access to an extract of the SNDS data with the help of SCALPEL3 [Bacry et al., 2020]. This database is at the core of our work presented in Chapter 2.

Challenges and opportunities with EHRs and medical claims data

Before going into details with SNDS, let us first describe in a non-exhaustive way some practical challenges relating to the healthcare data.

• Practitioners should pay particular attention on the data quality. As the primary goal for both EHRs and medical claims data is to assist the delivery of care and the billing, there could be mistakes in data entry, and biases from data collection. The data extraction step might also introduce some mistakes or biases.

• Benchmarking and reproducibility are important for the scientific community [Bellamy et al., 2020] to be able to compare across different datasets or different models, despite the sensitive nature of the healthcare data and possibly some compliance rules for in-house datasets.

• Although the predictive performance could be convincing for its own in some cases, models with interpretability are often preferred [Luo et al., 2019;[START_REF] Shickel | Deep EHR: a survey of recent advances in deep learning techniques for electronic health record (EHR) analysis[END_REF]Xiao et al., 2018] in the domain of healthcare.

• Data privacy is particularly important when it comes to healthcare data. Effective de-identification of patient data is a crucial step before any research: this is indeed the case for the SNDS [Tuppin et al., 2017].

Furthermore, some specific attention should be paid in works involving EHRs and medical claims data, in particular in the modelling of the healthcare pathways from these data.

• As a particular form of structured data with sequences of timestamped events, such as drugs prescriptions, medical procedures, hospital diagnoses, EHRs and claims data can also arise patient demographic information, medication dosage, etc. It is vital that the designed model is adapted to these specificities. Some practical examples includes, how to take into account medication dosage information alongside the drug in form of its CIP-13 codes? How to take into account the patient age information, as we know the age itself can tell a lot on the health status?

• Modelling the sequences of timestamped events is a crucial problem. Designing models that are adapted to data structure is important, in particular when in comes to deep neural networks models. Some recent research [START_REF] Li | BEHRT: transformer for electronic health records[END_REF] draw the parallel to NLP, in which case the medical codes are viewed as "words" such that the encounters or patients are viewed as "sentences" or "documents". Besides, the irregularity of the timestamps for the medical events that occur on the patient timeline, and the aggregation at different levels of the patient timeline, are some additional difficulties to this modelling problem.

• Learning appropriate and useful representations for medical codes is an important step, and can be a goal itself. Since there is often no explicit label in EHRs and medical claims data, data preprocessor may need to look at long term drug prescription history to be able to deduce some long-term diseases such as diabetes [Morel et al., 2019]; another direction is to design surrogate tasks such as masked prediction [START_REF] Li | BEHRT: transformer for electronic health records[END_REF]. Next, efficiently building representations for healthcare status from learned medical codes representations is also challenging.

• Effective evaluation is critical when it comes to assess models and pipelines.

In terms of prediction targets, while predicting the mortality and the lengthof-stay are some classical targets in EHRs ICU data [Rajkomar et al., 2018], disease detection or classification [START_REF] Li | BEHRT: transformer for electronic health records[END_REF], sequential prediction of medical events are also popular tasks on both EHRs and medical claims data. Most of these tasks, if not all, require careful and proper data processing to avoid that the features indirectly contain the labels. The lower-dimensional projection and visualization of learned representations for the medical codes might be helpful for qualitative assessment.

However, the research interest on EHRs and medical claims data is growing, especially using deep learning models recently. The reason is first and foremost for the applications opportunities in such a field, as well as enormous social impact it involves [Bommasani et al., 2021;[START_REF] Villani | Donner un sens à l'intelligence artificielle : pour une stratégie nationale et européenne[END_REF]. Also, efficiently and successfully modelling the sequences of timestamped events can be of interest for other situation other than the healthcare, for example user behavior modelling.

The French national system of health data (SNDS)

SNDS gathers reimbursement data from most state health insurance schemes, and aggregates data from multiple authorities such as hospitals and local agencies. At its creation in 2016, SNDS contained health reimbursements of 66 million French residents, representing 98.8% of the country's population [Tuppin et al., 2017]. Although SNIIR-AM was initially used to monitor health expenditures and to evaluate health care utilization across the country, from 2016, some drug safety studies [START_REF] Bezin | The national healthcare system claims databases in France, SNIIRAM and EGB: Powerful tools for pharmacoepidemiology[END_REF] The quality of SNDS data results from mandatory logging of reimbursed care in France, three data validation stages, and pseudonymization routines. Thanks to its history length, high population coverage, and quality, SNDS can be used to conduct epidemiological studies with a high statistical power and is almost exempt of representativity biases [START_REF] Neumann | Pioglitazone and risk of bladder cancer among diabetic patients in france: a population-based cohort study[END_REF]Tuppin et al., 2017].

There are two data sources for SNDS: DCIR (Inter-scheme consumption data -Données de Consommation Inter-Régimes) contains outpatients billing and reimbursement information, PMSI (medical information system program -Programme de Médicalisation des Systèmes d'Information) gathers hospital stay data.

DCIR contains demographic information of the beneficiaries (date of birth, gender, town of residence, and variables indicating whether patients are beneficiaries of specific social subsidies), and information on their possible disabilities or long-term diseases. More importantly, DCIR provides timestamped reimbursement information concerning drug purchases (drugs are coded with the ATC (Anatomical Therapeutic Chemical) classification system), medical procedures (coded with CCAM -the French medical procedures classification -Classification Commune des Actes médicaux), laboratory analyses (coded with NABM -the classification of clinical pathology proceduresnomenclature des actes de biologie médicale), and medical products (LPP -the list of product and services -Liste des produits et prestations).

PMSI is made up of four databases: MCO (acute care ward -Médecine, Chirurgie, Obstétrique et Odontologie), SSR (rehabilitation care -Soins de Suite et Réadaptation), HAD (home care -Hospitalisation À Domicile), and PSY (psychiatric care). These databases contain pseudonymized hospital stays summaries, i.e. starting and ending dates of hospital stays, diagnoses (coded with ICD-10the International Classification of Diseases, 10th revision), and medical procedures (coded with CCAM). In contrast to some well-know EHRs (for example MIMIC-III [START_REF] Johnson | MIMIC-III, a freely accessible critical care database[END_REF]), a difficulty with PMSI is that it comes from hospital stay summaries hence there is no information regarding the order or the temporality of the events, but only start-end dates with all the events happening throughout one hospital stay. PMSI also contains similar timestamped information for outpatient consultations.

Accessing to SNDS data requires authorization from the CNIL (Commission Nationale de l'Informatique et des Libertés), the French data protection authority, according to legal compliance and public interest. Recently, the Health Data Hub [Cuggia et al., 2018], a new governmental agency, was created. The aim is to make methodological research easier on SNDS and similar data, by centralizing health data, and by providing accesses with hardware and software infrastructure, at the same time under CNIL's compliance. Our extracted dataset is built upon the join and extraction work by the SCALPEL3 framework [Bacry et al., 2020].

Contribution: ZiMM, a deep learning model for long term and blurry relapses with non-clinical claims data

The work presented in Chapter 2 is an attempt to address some challenges described in the previous Section 1.1.1, in form of a study on a concrete medical example using recent tools from the deep learning.

In this work, we consider the problems of modeling and predicting a long-term and "blurry" relapse that occurs after a medical procedure, such as a surgery. We do not consider a short-term complication related to the procedure itself, but a longterm relapse that clinicians cannot explain easily, since it depends on unknown sets or sequences of past events that occurred before the procedure. The relapse is observed only indirectly in a "blurry" fashion, through longitudinal prescriptions of drugs over a long period of time after the medical procedure.

As stated previously, our motivation for using SNDS, a non-clinical medical claims database, is its exhaustivity population-wise, compared to EHRs coming from a single or a small set of hospitals. Indeed, we consider an extracted dataset containing the medical claims of almost all French residents who had surgery for prostatic problems, with a history between 1.5 and 5 years.

Patient timeline, study cohort, labels

A patient with several types of events is illustrated in Figure 1.1 using a timeline representation. This can be considered as a sequence of timestamped events z i k = (d i k , t i k ) with d i k medical code that happened as a medical event on the patient i at time t i k . we observe three types of events: drug purchases (blue), medical procedures (red) and diagnosis (yellow) before the medical act (prostate surgery in the example) that happens at time T i for patient i. All events are timestamped, and the time delta is a day long. Several events can occur the same day and some days have no event: events are typically very irregularly sampled. All these events, observed before T i , are used to learn the embedding vector x i ∈ R d of patient i. After T i , we only keep the events corresponding to the blurry relapses considered (drug purchases among a set of drugs for urinary problems). These blurry relapses are used to build the label vector y i ∈ N B of patient i.

S t a r

We extract patients with transurethral resection of the prostate (TURP) surgery in SNDS, through some specific CCAM codes provided by clinicians 1 . Some ad-ditional inclusion and exclusion criteria are applied to ensure data quality; in the main study, the final cohort includes 138 976 patients.

As far as the labels (post-surgical urinary problems) are concerned, a simple but efficient way to identify whether the urinary problems have not ceased or reappeared after a while is to see if the patient, at some point after the surgery, needs to take medications for these urination problems again. For that purpose, we use a list (provided by clinicians) of 136 CIP-13 drugs that are mainly related to urination problems. We choose to drive the prediction on a 18-months period after T i . In other words, we chose the number of buckets B = 18 and bucket size of 30 days (540 days total).

Zero-inflated Multinomial Mixture model

Here we consider a long-term (18 months) relapse (urination problems still occur despite surgery), which is blurry since it is observed only through the reimbursement of a specific set of drugs for urination problems. To best capture long-term and blurry relapses, we introduce Zero-inflated Mixture of Multinomial distributions model (ZiMM).

We denote by n i = B b=1 y i,b the overall number of blurry relapses of patient i. The following model is proposed to model the whole vector y i , so that n i is not fixed and includes zero-inflation, namely a parametrized likelihood for n i = 0. We suppose that n i ∈ {0, 1, . . . , B} is distributed (conditionally to x i ) as

n i ∼ Categorical(π 0 (x i ), π 1 (x i ), . . . , π B (x i )),
which means that

P(n i = k|x i ) = π k (x i ) for k ∈ {0, . . . , B},
where π b (x i ) are such that B b=0 π b (x i ) = 1 and π b (x i ) 0 (coming out of a softmax activation for instance). These parameters correspond to the categorical distribution specific to patient i. Then, we assume that the distribution of y i conditional to n i = b and x i follows either a Dirac distribution on vector (of size B) [0, . . . , 0] whenever b = 0, or a multinomial distribution of parameters b and p b,1 (x i ), . . . , p b,B (x i ), namely

y i |(x i , n i = b) ∼ δ [0,...,0] if b = 0, Multinomial(b, p b,1 (x i ), . . . , p b,B (x i )) otherwise,
where p b,1 (x i ), . . . , p b,B (x i ) are the parameters of a multinomial distribution whenever n i = b, for each b = 1, . . . , B. Once again, these parameters are specific to patient i.

On top of this probabilistic modelling of post-surgical complication, we build an end-to-end deep-learning architecture called ZiMM Encoder-Decoder (ZiMM ED) that can learn from the complex, irregular, highly heterogeneous and sparse patterns of health events that are observed through a claims-only database.

ZiMM Encoder

Codes are first tokenized, and each unique token is individually mapped to an embedding vector in R d E which is learned during training. We consider only tokens that occur at least 50 times in the training dataset. For any event occurring at time t T i in the observation period of patient i, we compute the "time horizon" as T i -t, namely the distance (in days) between the event and medical act. Moreover, whenever it makes sense, we compute endstart, the duration of the event, which corresponds to the duration of an hospital stay defined as the time between hospital admission and discharge. These two integers (in number of days) are also tokenized and replaced by a learned embedding vector. This allows the encoder to learn to put more or less emphasis on events that are close or far from T i , and to exploit the duration of events as a proxy for severity of medical procedures and diagnoses. The patient age in years at T i is also similarly bucketized and embedded.

Then, several events can occur at the same time (within the same day), so that the number of codes observed within a day is highly heterogeneous. Moreover, such codes are not likely to contribute equally to the vector representation of the day, and their order is not informative.

Hence, we use a self-attention mechanism [START_REF] Lin | A structured self-attentive sentence embedding[END_REF]Vaswani et al., 2017] using a bag-of-features approach to learn how to combine embedding vectors within the same day, following previous successful applications of self-attention for fusing disease embeddings [Luo et al., 2019].

In the following, we obtain sequence of fixed-sized embedding vectors that encode both medical and time information at each (non-empty) timestamp t T i , as displayed in Step (4) of Figures 1.2. Now, this sequence of vectors is used as the input of a stack of layers, including recurrent layers (LSTM [Hochreiter and Schmidhuber, 1997], bi-directional LSTM [Schuster and Paliwal, 1997], GRU [START_REF] Cho | Learning phrase representations using rnn encoder-decoder for statistical machine translation[END_REF]) or convolutional layers, in a sequence-to-one network architecture, since we want to output a single vector x i ∈ R d that encodes the full pathway of a patient before T i .

Finally, the output vector of the encoder is concatenated with an embedding vector of the age of the patient, leading to the final vector x i ∈ R d that encodes the full pathway of patient i, representing her health status in a whole, serving as input to the following decoder.

ZiMM Decoder

The decoder uses vector x i ∈ R d representation of health status of patient i to construct the parameters of the ZiMM model, and the whole architecture is trained against the negative log-likelihood of the ZiMM model computed at the label vector y i = [y i,1 , . . . , y i,B ] containing the blurry relapses. Since parameters {π b (x i )} b=1...,B and {p b,b (x i )} b,b ∈{1,...,B} 2 are highly dependent and are time-ordered, a specific architecture is used to model these dependencies: fully connected feed-forward layer (FFN) followed by a recurrent layer (RNN). Finally, a softmax activation is applied on h b t along t = 1, . . . , B to produce the parameters p b,1 (x i ), . . . , p b,B (x i ).

Experiments

Our experiments show that ZiMM ED improves several baselines, including nondeep learning and deep-learning approaches, and that it allows working on such a dataset with minimal preprocessing work. We implemented ZiMM under Tensorflow2. All our experiments use the same random data splitting into 70% of patients for training, 15% of patients for validation and 15% for testing. As for evaluation metrics, we use mean-AP, defined as the average of the area under the precision-recall curve (AUC-PR), namely we compute the average over the buckets b = 1, . . . , B of the AUC-PR for each bucket b, i.e., the AUC-PR of the prediction of y i,b . Moreover, we report also the AUC-PR and the AUC-ROC (area under the ROC curve) for the binary classification problem n i > 0 against n i = 0.

The predictive performance of all benchmarks and of ZiMM ED are presented in Table 1.1, where we report mean-AP scores on the test set, as well as AUC-PR and AUC-ROC scores for the binary classification problem. GBDT-based models on dynamic features (GBDT-DF) performs the best among all benchmark models, however the ZiMM ED architecture outperforms all the benchmark models both for the multi-output y i prediction and for the binary prediction.

In the ablation study, we modify some hyperparameters or change some components of ZiMM ED default, and report the impacts on performances in details in Table 2.4, where the first line reports the performances of ZiMM ED default. And we made careful discussion around each component of the model.

To wrap up, the work presented in Chapter 2 addresses the problem of predicting the blurry relapses of the TURP surgery, which is the first step towards an evidencebased approach using machine-learning to help the clinical decision.

Contrastive learning, unsupervised learning and supervised learning

The motivation of the work presented in Chapter 3 is to better understand the recently popular unsupervised and self-supervised training techniques, from a modelling point of view. The problem of learning appropriate and useful representations for words and sentences naturally arises in languages modelling, in the unsupervised setting. Let us trace the line of research of the recent successes of contrastive learning.

The skip-gram [Mikolov et al., 2013a] model considers the conditional probabilities p(c | w) where w stands for a word and c stands for a context. Denote θ the parameter for p(c | w; θ), then the goal is to find θ that maximizes the corpus probability

w c∈C(w) p(c | w; θ) = (w,c)∈D p(c | w; θ)
where C(w) denotes the set of all contexts related to word w, D denotes the set of all context and word pairs. Extracting such pairs (w, c) from a text is natural and easy. Denote v c and v w the vectorized representations (or embeddings) for a context c and a word w respectively. The neural networks language modelling literature usually models the conditional probability using the softmax function, namely

p(c | w; θ) = exp(v c v w ) c ∈C exp(v c v w )
where C denotes the set of all available context. Taking the log and combining both expressions leads to

(w,c)∈D log p(c | w; θ) = (w,c)∈D   v c v w -log c ∈C exp(v c v w )   .
Practitioners believe [START_REF] Goldberg | word2vec explained: deriving Mikolov et al.'s negativesampling word-embedding method[END_REF]Mikolov et al., 2013b] that training neural networks with the objective above will result in good words embeddings, in the sense that similar words that occur in similar contexts, will have similar embeddings. A downside of skip-gram is that it is expensive to compute the denominator term in the softmax, as well as the term log c ∈C exp(v c v w ), as it involves to sum over all possible contexts c ∈ C.

Based on this observation, the negative sampling [Mikolov et al., 2013b] aims to derive meaningful embeddings while being computationally efficient. Consider p(P = 1 | w, c; θ) the probability that a pair (w, c) actually comes from the corpus text, for any word w and any context c; and p(P = 0 | w, c; θ) the probability that a pair (w, c) actually does not come from the corpus text. These probabilities are modeled with the sigmoid function σ(u) = 1/(1 + exp(-u)), with

p(P = 1 | w, c; θ) = σ(v c v w ) = 1 -p(P = 0 | w, c; θ).
Then, the approach aims to maximize the following quantity, for any pair (w, c) in corpus,

p(P = 1 | w, c; θ) (w,c i ) / ∈D p(P = 0 | w, c i ; θ).
Taking the log, it yields

log σ(v c v w ) + (w,c i ) / ∈D log σ(-v c v w ).
With some practical tricks such as dynamic window size, down-sampling of frequent words, rare words pruning, this approach is experimentally shown [Mikolov et al., 2013b] to be efficient to learn words embedding from text. Several recent works further extend the idea of using naturally pair occurrence to unsupervisedly learn representations. Contrastive Predictive Coding (CPC) [START_REF] Oord | Representation learning with contrastive predictive coding[END_REF] learns representations of objects that natural lie on a sequence, by predicting future observations using a contrastive loss. The idea is to maximize the mutual information

I(x, c) = x,c p(x, c) log p(x | c) p(x)
between the encoded representations of (future) target x and (present) context c. The objective of CPC is the following contrastive loss, also called InfoNCE,

L = E X -log f k (x i,t+k , c i,t ) x j ∈X f k (x j , c i,t )
where X = {x 1 , . . . , x N } is a set of N random samples containing one positive sample from p(x t+k | c t ) and N -1 negative samples from the distribution p(x t+1 ); and f k (x t+k , c t ) stands for the density ratio between x t+k and c k ,

f k (x t+k , c t ) ∝ p(x t+k | c k )/p(x t+k ), modeled by f k (x t+k , c t ) = exp(g encoder (x t+k ) W k c k )
where W k c k denote a linear transformation of c k . CPC has practical success in audio, vision and natural languages [START_REF] Oord | Representation learning with contrastive predictive coding[END_REF].

In particular, combined with a pre-training strategy [START_REF] Hénaff | Data-efficient image recognition with contrastive predictive coding[END_REF], representations learned with the contrastive loss in an unsupervised fashion are shown to be useful for some down-stream classification tasks. [START_REF] Hénaff | Data-efficient image recognition with contrastive predictive coding[END_REF] uses a pretrained image encoder followed by simple classification layers, that are trained on a fraction of the labels available, allowing to achieve an accuracy comparable to that of a fully supervised end-to-end training. In the same pre-training -fine-tuning fashion with contrastive loss during the pre-training phase, MoCo [He et al., 2020],

SimCLR [Chen et al., 2020], SwAV [START_REF] Caron | Unsupervised learning of visual features by contrasting cluster assignments[END_REF], BYOL [START_REF] Grill | Bootstrap your own latent: A new approach to self-supervised learning[END_REF] are some improvements to successfully learn visual representations.

Let us summarize the common pre-training approach: provided a dataset, an encoder is trained using a contrastive loss whose minimization allows to learn encoders giving embeddings that are similar for pairs of samples (called the positives) that are close to each other (such as pairs of random data augmentations of the same image, see Chen et al. [2020]; He et al. [2020]), while such embeddings are contrasted for dissimilar pairs (called the negatives).

Although Saunshi et al. [2019] proposed a formalism together with results on classification performance based on unsupervisedly learned representation, these results do not explain the performance gain that is observed empirically [Chen et al., 2020;He et al., 2020] when a high number of negative samples are used. On the other hand, Allen-Zhu et al. [2019] proved that, over sufficient over-parametrization of deep neural networks, the optimization with GD and SDG of the 2 -loss converges, as well as for RNN.

However, despite growing efforts [Saunshi et al., 2019; Wang and Isola, 2020], few theoretical results have been obtained as of the time of our work. For instance, there is no clear theoretical explanation of how a supervised task could benefit from an upstream unsupervised pre-training phase, or what could be the theoretical guarantees for the convergence of the minimization procedure of the contrastive loss during this pretraining phase.

Contribution: theoretical guarantees on the contrastive unsupervised representation learning for classification and on its convergence

Following the framework first proposed in Saunshi et al. [2019], assume that we are able to sample positive pairs (x, x + ) from the distribution

D sim (x, x + ) = c∈C ρ(c)D c (x)D c (x + ), (1.1)
namely, (x, x + ) is sampled as a mixture of independent pairs conditionally to a shared latent class, sampled according to ρ. On the other hand, we assume that we can sample negative samples x -from the distribution

D neg (x -) = c∈C ρ(c)D c (x -). (1.2)
Under the assumptions given in Equations (1.1) and (1.2), we consider the following unsupervised contrastive loss with N negative samples,

L N un (f ) = E (x,x + )∼D sim X -∼D ⊗N neg   -log   exp f (x) f (x + ) exp (f (x) f (x + )) + x -∈X -exp (f (x) f (x -))     ,
(1.3) where D ⊗N neg stands for the N tensor product of the D neg distribution given by Equation (1.2).

Proposition (Proposition 3.3). Consider the unsupervised loss L N un (f ) from Equation (1.3) with N negative samples. Assume that ρ is uniform over C and that

2 k + 1 N C . Then, any encoder function f : X → R d satisfies L sup,k (f ) L µ sup,k (f ) k 1 -τ + N L N un (f ) -τ + N log(N + 1) with τ + N = P c i = c, ∀i | (c, c 1 , • • • , c N ) ∼ ρ ⊗N +1 .
These results hold for an arbitrary large number of negatives and decoupled from the number of classification tasks. We provide new theoretical guarantees for the classification performance of contrastively trained models in the case of multiway classification tasks, using multiple negative samples. We extend results from Saunshi et al. [2019] to show that unsupervised training performance reflects on a subsequent classification task in the case of multiple tasks and when a high number of negative samples is used.

Also, we provide a convergence result (Theorem 3.5) for an explicit algorithm (gradient descent), when training overparametrized deep neural network for unsupervised contrastive representation learning. We explain how results from Allen-

From Random Forest to WildWood

In the work presented in Chapter 4, we focus on ensemble tree-based algorithms. Under the batch supervised learning setting, data comes as a set of i.i.d. samples (x i , y i ) for i = 1, . . . , n, with features x i ∈ X and labels y i ∈ Y. We consider quantitative and categorical features.

Random Forest: general principles

Let us first provide some background on Random Forest [START_REF] Breiman | Random forests[END_REF], an important building block of WildWood.

Ensemble methods

In machine learning, under the name of ensemble methods, we refer to the general procedures that generate multiple predictors then combine them into one of better quality. The two principal families in the ensemble methods are boosting and bagging.

• Boosting [START_REF] Freund | A decision-theoretic generalization of on-line learning and an application to boosting[END_REF][START_REF] Friedman | Greedy function approximation: a gradient boosting machine[END_REF] consists in successively combining base classifiers returned by a weak learner, to create a more accurate learner. In other words, boosting algorithms combine "simple" predictors (i.e. predictors with big approximation error), for example decision trees with a small depth.

• Bagging (or bootstrap aggregation) [START_REF] Breiman | Bagging predictors[END_REF] consists in combining multiple realizations over bootstrap samples of a "complex" predictor (i.e.

noisy but approximately unbiased predictor), for example fully grown decision trees.

Random Forests (RF), proposed in [START_REF] Breiman | Random forests[END_REF], and inspired by previous works of Amit and Geman [1997]; [START_REF] Ho | The random subspace method for constructing decision forests[END_REF], was in the spirit of bagging, based on tree models [START_REF] Breiman | Classification and regression trees[END_REF]. Tree models are able to capture complex patterns, but fully grown trees overfit very easily on the training data: such property makes such tree-based models an ideal candidate for bagging. Denote M the number of individual trees. RF builds each tree on bootstrap samples: each time, n elements of index of training samples I = {1, . . . , n}, corresponding to in-the-bag (itb) samples, are uniformly selected at random with replacement. In classification tasks, RF takes the majority vote over predictions of all decision trees, or returns the average of probabilities of each class predicted by each tree; in regression tasks, RF takes the average predictions of all decision trees, i.e. [START_REF] Breiman | Random forests[END_REF], "each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest". We note that each tree is built independently of each others, so that tree training can be done in parallel.

Tree and tree predictor

In the following, we describe, in RF, the construction of each tree [START_REF] Breiman | Classification and regression trees[END_REF], and the predictor that corresponds to the tree, i.e. how f (•; Π m ) is constructed. For simplicity, we omit the index m = 1, . . . , M , and use I itb to denote indexes of in-the-bag samples for the tree we consider. We focus on binary trees here.

A tree-based predictor partitions the features space X into rectangles (for continuous features) or discrete subsets (for categorical features) called cells, and provides simple (often constant) predictions on each cell. Since the partition can be complex, even with simple cell predictions, tree-based methods are powerful and able to capture complex and non-linear patterns. For tree growing, we recursively partition the features space, by successively applying splits into two cells. A decision tree predictor f : X → Y has the following properties.

• A finite partition C of X is said to be a tree partition if it is associated to a finite ordered binary tree T : each node v that is different to root has a unique parent, and either has two child nodes -with left child node denoted v0 and right one denoted v1, in which case v is an interior node, or no child node -in which case v is a leaf ; -a family of splits Σ = (σ v ) v indexed by the interior nodes of T : a split is denoted by σ v = (j v , t v ), where j v ∈ {1, . . . , d} indicates the index of the feature, and t v indicates a range of the feature's values less than a threshold (split on a numerical feature) or a subset of the feature's modalities (split on a categorical feature).

We associate to each node v of T a cell C v ⊆ X , recursively in the following way the cell associated to root is C root = X ;

for each interior node v, given its cell C v and split σ v = (j v , t v ), we define

C v0 = {x ∈ C v : x jv ∈ t v } and C v1 = C v \ C v0 .
From this, the cells C v of all leaves v of T form a partition C of X , associated to (T , Σ).

• Then, we define the predictor of the tree (T , Σ) by

f (T ,Σ) : X → Y, x → y C(x) ,
where C(x) is the unique leaf containing x. A prediction y Cv is associated to each leaf v of T , i.e. to each cell C v of C . In regression tasks, the predictor of a leaf cell C v is usually the empirical mean of all itb training samples in

C v y Cv = 1 card{i ∈ I itb , x i ∈ C v } i∈Iitb,x i ∈Cv y i
In classification tasks, the predictor of C v can be the majority vote of all itb training samples in C v y Cv = majority vote{y i } i∈Iitb,x i ∈Cv or the density estimation based on all training samples {(x i , y i )} i∈Iitb,x i ∈Cv in the cell, e.g. empirical frequencies for each class, or a Bayes predictive posterior.

A tree is grown successively by finding a split at each node v, until meeting the stopping criterion. To find such a split σ v , we proceed greedily: for each feature indexed by j, we determine the optimal split σ v,j along the feature j, in the sense of the criterion; we do so and loop over all the currently available features j ∈ sub-sampling{1, . . . , d} (as for features sub-sampling); then we take the best of such splits σ v,j in the sense of the criterion, as the actual split σ v . To be able to do so, we need a criterion P(X ) → R to measure the qualities of possible splits: the smaller the criterion function value is, the more suitable the split is on the data contained in the current cell C v , the better the split is. In classification, the Gini index and the cross-entropy are popular choices for the criterion. In regression, the sum of squares is usually used as the criterion.

In RF [START_REF] Breiman | Random forests[END_REF], an additional source of randomness is the feature subsampling: for each split, m d features are randomly and uniformly selected (without replacement); only these m features can be candidates for the current split. Typical values are m = √ d or m = log d or m = d/3 (for regression). Feature subsampling allows to introducing more variability and to further reducing correlation between the trees -a particularly nice property with bagging. In RF, trees often stop growing only very lately, a typical stopping criterion checks that only a few number of training samples remains on a leaf or on a split. Random Forest. We recap the main ingredients in Random Forest as presented in [START_REF] Breiman | Random forests[END_REF] in the following.

• A bootstrapped sample of the training dataset is generated for each tree: each time, n samples are uniformly chosen with replacement from the training dataset {(x 1 , y 1 ), . . . , (x n , y n )}.

• Trees are constructed in parallel, and are fully grown until they meet a stopping criterion. With no tree pruning, each cell usually contains few data points.

• The randomness of the partition choice comes from the bootstrap and also from the features sub-sampling.

Since its introduction, Random Forest is one of the most widely used supervised learning algorithms, in both classification and regression tasks. It combines good predictive performances, weak training and inference cost while being parallel computation friendly, very few number of hyperparameters to tune, and at the same time enjoys some interpretability. Some development about Random Forest methodology include extremely randomized trees [Geurts et al., 2006], variable importance [Louppe et al., 2013], pruning method for ensemble [START_REF] Ren | Global refinement of random forest[END_REF], regularization mechanism with depth limitation in Random Forest [Zhou and Mentch, 2021], to cite few of them.

Aggregation algorithms

The exponential weight aggregation algorithm and the histogram strategy are the two principal novelties that WildWood introduce comparing to Random Forest.

Exponential weights algorithms

In the classical online learning literature such as Cesa-Bianchi and Lugosi [2006], the exponential weights aggregation is introduced as a strategy for prediction of deterministic individual sequences with expert advices. Formally, let Θ be the measurable space of the parameters of estimators, y θ,t the predictor related to the parameter θ at time t, and π a probability measure on Θ named prior. In the online learning setting, the exponential weight aggregation algorithm proposes to use the aggregator, at time t,

y t := Θ y θ,t v t (dθ) where dv t dπ (θ) := e -ηL θ,t-1
Θ e -ηL θ ,t-1 π(dθ ) with L θ,t denoting the loss of the estimator y θ,t related to θ at time t, evaluated on all data until time t in hindsight. This predictor is largely studied in Cesa-Bianchi and Lugosi [2006]. For example, with exp-concave loss function, uniform prior π, and discrete and finite-numbered parameters set Θ, running exponential weights algorithm guarantees a regret that is logarithmic in the size of Θ.

The same spirit can be translated into the general supervised learning setting. Denote f θ the predictor related to the parameter θ, the exponential weight aggregation algorithm uses the aggregator

f (•) := Θ f θ (•)v(dθ) where dv dπ (θ) := e -ηL θ
with L θ the loss of the estimator f θ , evaluated on all the training data. In regression and squared loss setting, a study [Dalalyan and Tsybakov, 2007] of exponential weights aggregation obtained sharp PAC-Bayesian risk bounds. When it comes to the aggregation of subtrees of a tree, we need to consider particular forms of prior π over the tree structure. Context tree weighting [START_REF] Catoni | Statistical Learning Theory and Stochastic Optimization: Ecole d'Eté[END_REF] is indeed useful in designing such aggregations efficiently.

Popular gradient boosting algorithms

In the family of boosting algorithms, extreme gradient boosting algorithms are powerful and often popular in data science challenges. Some successful examples are XGBoost [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF]] which provides a popular scalable tree boosting system widely adopted in industry, LightGBM [START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF] which introduces the "histogram strategy" for faster split finding, together with clever down-sampling and features grouping algorithms in order to achieve high performance in reduced computation times, and CatBoost [Prokhorenkova et al., 2017] which pays particular attention to categorical features using target encoding, while addressing the potential bias issues associated to such an encoding.

Contribution: WildWood, a new random forest algorithm

On the one hand, as a bagging algorithm, RF builds each tree based on bootstrapped training samples, using out-of-bag (oob) samples only for computing scores. On the other hand, as far as we know, there is no efficient implementation of Random Forest algorithm that implements histogram strategy and has native support for categorical features. In the work presented in Chapter 4, we propose WildWood (WW), which keeps all ingredients of RF including bootstrap, fully grown trees, features sub-sampling, similar tree growing procedure, with an improved tree predictor and efficient computations implemented in Python.

Improving the tree predictor by the subtrees aggregation with exponential weights

Consider a fully grown tree T . Recall that in standard RF, the tree predictor corresponding to T writes f (x) = y Cv(x) for any x ∈ X , where C(x) is the cell containing x associated to leaf v: this is to find the smallest cell containing x, and outputs the prediction of this cell. In WW, we improve that tree predictor by the following mechanism: as the predictor for T , WW uses

f (•) = T ⊂T π(T )e -ηL T y T (•) T ⊂T π(T )e -ηL T with π(T ) = 2 -T , (1.4)
where the sum is over all subtrees T of T rooted at root, η > 0 is temperature parameter, T denotes the number of nodes in T minus its number of leaves that are also leaves of T , L T is the cumulative loss of the prediction of subtree T over oob samples, i.e. L T := i∈Ioob ( y T (x i ), y i ), and y T is prediction of subtree T in its classical way. The prediction function (1.4) can be understood as an aggregation of the predictions y T (•) of all subtrees T , weighted by their performances on oob samples together with prior π(T ) = 2 -T ; this is indeed a non-greedy way to prune trees: the weights depend not only on the quality of one single split but also on the performance of each subsequent splits. Furthermore, the following Theorem gives a theoretical guarantee over oob samples, for subtree aggregation, under an assumption on exp-concavity of the loss function. This Theorem states that the predictor given by (1.4) is able to perform almost as well as the best oracle subtree T ⊆ T on oob samples, with an O( T /n oob ) rate which is optimal for model-selection oracle inequalities [Tsybakov, 2003]; in contrast, finding an oracle argmin T ∈T i∈Ioob ( f (x i ), y i ) is computationally infeasible, which requires trying out all subtrees.

Theorem (Oracle inequality, Theorem 4.2 restated). Assume that the loss function is η-exp-concave. Then, the prediction function f given by (1.4) satisfies the oracle inequality

1 n oob i∈Ioob ( f (x i ), y i ) inf T ⊂T 1 n oob i∈Ioob ( y T (x i ), y i ) + C T η(n oob + 1)
,

where the infimum is taken over all subtrees T ⊂ T rooted at root, and C = log 2.

As a consequence, we proved that the predictor (1.4) given by subtree aggregation, is almost as good as the best of any pruning of the same tree, i.e. any predictor that is constant on its leaves, see Corollary 4.3 for classification tasks with the log-loss, and Corollary 4.4 for regression tasks with the least-square loss.

Efficient implementation of WildWood

A naive computation of prediction function (1.4) would involve summing over all subtrees, the computation cost would be exponential in the number of nodes, which is infeasible, WW computes (1.4) exactly and efficiently thanks to context tree weighting, as stated in the following Theorem.

Theorem (Theorem 4.1 simplified). The prediction function (1.4) can be computed recursively along the path of x in T , with a computational cost in the same order with that of a standard RF.

A consequence of this Theorem is an efficient computation of f (x), exploiting the fact that trees in WW are grown in a depth-first fashion. Computing f (x) indeed only increases by a factor 2 the computational complexity of a standard RF.

Besides this efficient computation of subtree aggregation, WW natively supports categorical features and implements a histogram strategy, to accelerate split finding. Our implementation of WildWood is available at the GitHub repository https: //github.com/pyensemble/wildwood.

We conducted extensive experiments assessing the predictive performances and running times of our implementation of WildWood, see Tables 4.1 and 4.2 in Chapter 4 of this manuscript for detailed experiments settings and results. All experiments can be reproduced using Python scripts on the same repository. Overall, as an improved Random Forest algorithm, WildWood showed competitive performance in terms of predictive power and running time, compared to standard Random Forest and extreme gradient boosting algorithms.

Online logistic regression: towards an efficient algorithm with better regret guarantee

In the previous section, we consider the batch supervised learning, where data are a set of observations {(x 1 , y 1 ), . . . , (x n , y n )} obtained at once. Now in the work presented in Chapter 5, we move to the online learning setting, where data are revealed sequentially. In such a setting, in general, we do not make any assumption on the data distribution, and we look for some algorithms with a performance guarantee that is valid for any individual sequence (x t , y t ) t . Some classical references include Cesa-Bianchi and Lugosi [2006] as well as Hazan [2019]; Orabona [2019].

Online learning: general principles

Let us start by providing some general background on online learning. Denote Y the space of predictions and Y the space of labels. Given a loss function : Y × Y → R, the objective of online learning is to produce some step-by-step predictions for observations y 1 , y 2 , . . . ∈ Y that are accurate in the sense of the loss function.

More precisely, we formulate the setting of online learning as a game between an agent (who produces predictions) and an environment (which generates groundtruths). At each time step t = 1, 2, . . .,

• the agent chooses a prediction y t , based on all previous observations S t-1 = (y 1 , . . . , y t-1 );

• the environment reveals the actual label y t ;

• the agent suffers the loss ( y t , y t ) at time t.

A learning algorithm A, or a strategy, is a function of all previous observations that returns the prediction at time t, i.e. following A, the agent predicts y t = A(S t-1 ) at each time t.

The objective of the agent is to produce predictions ( y t ) t that minimize the cumulative loss n t=1 ( y t , y t ), where the number of total rounds n might or might not be known in advance. However, since we do not make any assumption on the distribution of (y t ) t , the cumulative loss can be big or small according to the "difficulty" of the observations (e.g. the environment may choose adversarially y t with the knowledge of the agent's prediction y t ). To take this fact into account, we introduce the notion of regret to effectively evaluate the quality of the successive predictions ( y t ) t , or the quality of an algorithm A. Let f ∈ Y be a predictor, the regret of successively playing ( y t ) t compared to f , is the excess cumulative loss versus using f as the single fixed predictor in hindsight,

Regret n (f ) := n t=1 ( y t , y t ) - n t=1 (f, y t )
where n stands for the number of total rounds, or the time-horizon. Consider a subclass F ⊆ Y of predictors, the regret of predictions ( y t ) t relative to F is defined by the excess cumulative loss compared to the best predictor f ∈ F as the single fixed predictor in hindsight, namely

Regret n := n t=1 ( y t , y t ) -inf f ∈F n t=1 (f, y t ).
(1.5)

In
Regret n = n t=1 (A(S t-1 ), y t ) -inf f ∈F n t=1 (f, y t ).
The exponential weight aggregation, as mentioned in Section 1.3.2, is a fundamental strategy of online learning.

Supervised online learning

The setting of supervised online learning is slightly different to the general setting of online learning: at time t, features x t ∈ X are revealed, then the agent predicts y t with the knowledge of x t . Formally, at each time step t = 1, 2, . . .,

• the environment reveals the features x t ∈ X ;

• the agent chooses a prediction y t , based on all previous observations S t-1 = ((x 1 , y 1 ), . . . , (x t-1 , y t-1 )) and x t ;

• the environment reveals the actual label y t ;

• the agent suffers the loss ( y t , y t ).

Under the setting of supervised online learning, predictors are functions of the features f : X → Y, x t → f (x t ) = y t . The inputs to a learning algorithm A include all the previous observations as well as the features of time t, i.e. the agent predicts y t = A(S t-1 , x t ). If the predictor returned by the learning algorithm x t → A(S t , x t ) belongs to the comparison class F, we say that the algorithm is proper; otherwise, the algorithm is improper. Classical online learning methods often focus on predictors of the form x t → A(S t-1 )(x t ), e.g. in the OCO setting described in the next paragraph: they are usually proper algorithms. There are similar notions in statistical learning theory [Bousquet et al., 2003; Shalev-Shwartz and Ben-David, 2014] for proper (or plug-in estimator, when the estimator takes value inside the comparison class) and improper (when the estimator is not restricted to the comparison class) algorithm. Let us conclude this remark on the proper and improper algorithms by pointing out that, in the case where an algorithm A effectively uses the features x t to build the predictor f at time t, A is necessarily improper.

Consider the parametrized comparison class F = {f θ : X → Y, θ ∈ Θ} where Θ is the space of the comparison parameter θ. Then, the regret of an algorithm A relative to F writes

Regret n := n t=1 (A(S t-1 , x t ), y t ) -inf θ∈Θ n t=1 (f θ (x t ), y t ).

Online convex optimization (OCO)

Assume that loss function : Y × Y → R is convex and differentiable, i.e. for any y ∈ Y, the function y → ( y, y) is convex and differentiable. We follow the general setting of online learning. It is practical to introduce the notation t to denote the loss function induced by the observation y t at time t, i.e. t : y → ( y, y t ). Then, for any y ∈ Y,

t ( y t ) -t ( y)
∇ t ( y t ), y t -y because of its convexity. Hence, denoting h t := ∇ t ( y t ), we can write an upperbound of the regret of the predictions ( y t ) as the regret produced by the linear functions h t , • . Namely,

Regret n = n t=1 ( y t , y t ) -inf ŷ∈ Y n t=1 ( y, y t ) n t=1 h t , y t -inf ŷ∈ Y n t=1 h t , y .
Therefore, we use the convexity of the loss function to reduce the problem into that of minimizing the linear regret. Although this reduction might not always be optimal, the use of convexity offers a practical starting point to design OCO [Hazan, 2019;Orabona, 2019] algorithms, usually efficient and proper.

Online-to-batch conversions

We present some notions in the batch learning, in particular their similarities and relationship with the online setting. Let π be a probability measure on Y, the quality of a predictor y ∈ Y is assessed by the expected loss, or its risk R( y) :=

E Y ∼π [ ( y, Y )].
The excess risk of a predictor y ∈ Y related to a comparison class F ⊆ Y is the difference between the risk induced by y and that of the best predictor

f ∈ F, namely E( y) := R( y) -inf f ∈F R(f ).
We can see that the notion of excess risk in batch learning is analogous to the notion of regret (Equation (1.5)) in the online setting. Under i.i.d. assumption for data and with a convex loss function, an upper-bound on the regret of an online algorithm entails an upper-bound on the expected excess risk, using the average predictor, as explained by the following Proposition.

). Then E[R( Ȳn )] -inf f ∈F R(f ) 1 n E [Regret n ] , (1.6)
where the expectations are taken with respect to the joint distribution of

(Y 1 , • • • , Y n ).
Proof. For any time t = 1, . . . , n, since Y t only depends on (Y 1 , . . . , Y t-1 ) and is independent of

Y t , E[R( Y t )] = E E Yt∼π [ ( Y t , Y t )] | Y t = E[ ( Y t , Y t )]
where the expectations are taken with respect to the joint distribution of (Y

1 , • • • , Y t ).
Since the loss function is convex, the risk R is convex. Then, for any f ∈ F,

E[R( Ȳn )] -R(f ) 1 n n t=1 E[R( Y t )] -R(f ) = 1 n E n t=1 ( Y t , Y t ) -(f, Y t ) E [Regret n ] n
where the expectations are also taken with respect to the joint distribution of (Y 1 , • • • , Y n ); the first inequality uses the convexity of R, and the last inequality uses the definition of the regret (Equation (1.5)) on the sequence (Y 1 , . . . , Y n ).

Then we obtain the Equation (1.6) by taking the supremum over f ∈ F.

Proposition 1.1 gives a general procedure using the average predictor to convert an online algorithm into a batch algorithm, with guaranteed excess risk in expectation; the same average predictor is also showed with generalization ability with big probability, see for example Cesa-Bianchi et al. [2004]. Other forms of onlineto-batch conversion are also possible, for example a non-uniform average predictor was reported in [START_REF] Zhang | Solving large scale linear prediction problems using stochastic gradient descent algorithms[END_REF].

Besides, we remark that in Proposition 1.1, we can get rid of the convexity assumption. In that case, instead of taking the average predictor, we choose Ȳn = Y τ with τ uniform drawn from {1, . . . , n}, and it yields the same excess risk guarantee in expectation.

Online logistic regression, related works

Online binary logistic regression falls into the setting of supervised online learning, with X = R d , Y = {-1, +1}, and F corresponding to the logistic models

{f θ : θ ∈ R d }, such that p(1 | x) = 1 -p(-1 | x) = f θ (x) = σ( θ, x )
for any x, θ ∈ R d , where σ denotes the sigmoid function σ(u) := e u /(1 + e u ). The loss function for this problem is given by

( θ, x , y) = -log σ(y θ, x ) = -1 y=1 log σ( θ, x ) -1 y=-1 log σ(-θ, x ).
Usually, we study online binary logistic regression under an assumption on the bounded features x t R and bounded comparison class F = {x → θ, x , θ B}, with some R, B > 0.

As an instance of online convex optimization, gradient methods such as Online Gradient Descent (OGD) [Zinkevich, 2003] and Online Newton Steps (ONS) [Hazan et al., 2007] apply for online logistic regression. However, under the proper setting and by constructing adversary examples, Hazan et al. [2014] showed that in particular, any O(B log n) regret is impossible for proper algorithms.

However, better regret guarantees are possible with improper algorithms. The following studies indeed moved on to some improper Bayesian and non-Bayesian algorithms. Bayesian improper method Foster et al. [2018] enjoys an optimal regret guarantee but suffers from its computational complexity. Non-Bayesian improper method AIOLI [Jézéquel et al., 2020] is built upon a new quadratic approximation of the logistic regression and a regularization by samples, coming with an extra factor BR in its regret guarantee but with efficient computations. Recent works GAF [Jézéquel et al., 2021] and FOLKLORE [Agarwal et al., 2021] both extend the quadratic approximation first introduced in AIOLI [Jézéquel et al., 2020] into the multi-class case, respectively propose a Bayesian improper algorithm and a non-Bayesian improper algorithm for multi-class logistic regression, with regret guarantee similar to the AIOLI one. On the other hand, in the batch setting (in particular batch logistic regression), Mourtada and Gaïffas [2019] introduces a minmax estimator. This improper algorithm achieves an excess risk guaranteed in the optimal order.

A detailed literature review is proposed in Chapter 5 of the manuscript. Let us summarize here in Table 1.2 the regret upper-bounds and the computational costs of these algorithms for online binary logistic regression.

Table 1.2 -Regret upper-bounds and computational costs in O(•) of several algorithms for online binary logistic regression: OGD [Zinkevich, 2003], ONS [Hazan et al., 2007], Foster [Foster et al., 2018], GAF [Jézéquel et al., 2021], AIOLI [Jézéquel et al., 2020

nd 2 + nBR log(n) AOSMP Improper, non-Bayesian dBR log(n) + B 2 R 2 nd 2 + nC GD

Contribution: study of some non-Bayesian improper algorithms

The starting point of the work presented in Chapter 5 is the following question: with possibly some adaptation, could an online version of SMP [Mourtada and Gaïffas, 2019] achieve a regret guarantee comparable to SMP's excess risk? We present two candidate algorithms, OSMP and AOSMP, and their regret analysis. Let us start by considering the λ-ridge-penalized regret until time t for online logistic regression, which writes,

t s=1 ( y s , y s ) -inf θ∈Θ t s=1 (θ x s , y s ) + λ θ 2 .
We introduce the One-Step Minmax Predictor (OSMP), taking "best possible" choice of prediction y t against "the worst possible" ground-truth value of y t ∈ {-1, +1} and the adversary parameter θ, measure by the λ-ridge-penalized regret above. Namely,

y t = argmin ŷ∈R sup yt∈{-1,1} sup θ∈R d ( y, y t ) + L t-1 -(θ x t , y t ) + L λ,t-1 (θ) ,
with L t-1 := t-1 s=1 ( y s , y s ), and L λ,t-1 (θ) := t-1 s=1 (θ x s , y s ) + λ θ 2 . We show that equivalently (Lemma 5.4), OSMP predicts

y t = -L +1 λ,t + L -1 λ,t with L y λ,t := inf θ∈R d (θ x t , y) + L λ,t-1 (θ) for y ∈ {-1, +1}
. This gives the explicit Algorithm 1 for the computation of OSMP. On the regret analysis of OSMP, we first remark that

Regret n (θ) = n t=1 ( y t , y t ) - n t=1 (θ x t , y t ) λ θ 2 + n t=1 r t ,
where we denote r t := ( y t , y t ) -L λ,t + L λ,t-1 the instant regret of time t. We deduce the following upper-bound for the instant regret, whose proof relies on the generalized R-self-concordance [START_REF] Bach | Self-concordant analysis for logistic regression[END_REF] of the function L λ,t .

Algorithm 1 One-Step Minmax Predictor (OSMP) description.

1: Inputs: Parameters λ > 0, n, d 1 constants B, R > 0 2: Initialize: L λ,0 (θ) = λ θ 2 3: for t = 1, . . . , n do 4: Receive x t 5: Compute L y λ,t ← inf θ∈R d (θ x t , y) + L λ,t-1 (θ) for y = -1 and y = +1 6: Predict y t ← -L +1 λ,t + L -1 λ,t 7:
Receive y t

8:

Update function L λ,t (θ) = L λ,t-1 (θ) + (θ x, y t ) 9: end for Proposition (Proposition 5.9). Denote θ t := argmin θ∈R d L λ,t (θ), assume x t R for some R > 0. We run OSMP (Algorithm 1) with λ R 2 . Then the instant regret as defined above is upper-bounded,

r t e • σ ( θ t , x t ) • x t 2 (∇ 2 L λ,t (θt)) -1 .
However, this Proposition is as far as goes our analysis of OSMP. In particular, the value of Hessian ∇ 2 L λ,t (θ t ) depends on the point θ t where it is evaluated, and we are not aware of a way to control such Hessian matrix leading to a regret upperbound which is logarithmic in n and without an exponential factor in BR, see Section 5.5.3 for more details.

To get around this difficulty, we introduce the following Approximated One-Step Minmax Predictor (AOSMP), defined by

y t = argmin y∈R sup yt∈{-1,+1} sup θ∈R d L t-1 + ( y, y t ) -(θ x t , y t ) + Lλ,t-1 (θ)
with Lλ,t-1 (θ) := t-1 s=1 ˜ (θ x s , y s )+λ θ 2 . This formulation is in the same minmax spirit with OSMP. The difference is that in AOSMP, we replace the evaluation function on the adversary parameter L λ,t-1 by its quadratic surrogate Lλ,t-1 . We rely on the quadratic approximation of the logistic function first introduced in Jézéquel et al. [2020, Lemma 5], namely

˜ t (θ) := t ( θt ) + g t (θ -θt ) + 1 2 η t x t (θ -θt ) 2 , (1.7)
with g t := ∇ t ( θt ), η t := σ ( θt , x t )/(1 + BR) and we choose θt = argmin

θ∈R d Lyt λ,t (θ) with Ly λ,t (θ) := (θ x t , y) + Lλ,t-1 (θ).
For the same reasons for OSMP, AOSMP predicts

y t = -L+1 λ,t + L-1 λ,t
with Ly λ,t := inf θ∈R d (θ x t , y) + Lλ,t-1 (θ) for y ∈ {-1, +1}, and that gives Algorithm 2 for the computation of AOSMP. Then, we are able to apply the classical techniques with the quadratic forms to upper-bound the sum of the pseudo instant regrets, to obtain the following Theorem on the regret of AOSMP.

Algorithm 2 Approximated One-Step Minmax Predictor (AOSMP) description.

1: Inputs: Parameters λ > 0, n, d 1, constants B, R > 0 2: Initialize: Function Lλ,0 (θ) = λ θ 2 3: for t = 1, . . . , n do 4: Receive x t 5: Compute Ly λ,t ← inf θ∈R d (θ x t , y) + Lλ,t-1 (θ) for y = -1 and y = +1 6: Predict y t ← -L+1 λ,t + L-1 λ,t 7:
Receive y t

8:

Compute θt ← argmin θ∈R d (θ x t , y t ) + Lλ,t-1 (θ) , and function ˜ t (θ) as specified in Equation (1.7)

9:

Update function Lλ,t (θ) = Lλ,t-1 (θ) + ˜ t (θ) 10: end for Theorem (Theorem 5.19)

. Let R, B > 0 and d, n 1. Let (x 1 , y 1 ), . . . , (x n , y n ) ∈ [-R, R] d × {-1,
+1} be an arbitrary sequence of observations. We run AOSMP (Algorithm 2) with λ R 2 . Then its regret against any θ ∈ B(R d , B) satisfies the following upper-bound,

Regret n (θ) e • (1 + BR)d log 1 + nR 2 8d(1 + BR)λ + λ θ 2 .
In particular, choosing λ = R 2 yields

Regret n e • (1 + BR)d log 1 + n 8d(1 + BR) + B 2 R 2 .
We obtain for AOSMP a regret upper-bound similar to the one of AIOLI [Jézéquel et al., 2020], as we use the same quadratic approximation (1.7).

Overall, in this work, we propose and analyze two SMP-inspired [Mourtada and Gaïffas, 2019] algorithms for online logistic regression: for OSMP, we only found an upper-bound for instant regret

r t e • σ ( θ t , x t ) • x t 2 (∇ 2 L λ,t (θt)) -1
; for AOSMP, we proved a regret upper-bound in a similar order with the state-of-theart algorithm [Jézéquel et al., 2020] for binary online logistic regression to the best of our knowledge. The question of an efficient algorithm with better regret upperbound for online logistic regression remains open.

Abstract

This work considers the problems of modeling and predicting a long-term and "blurry" relapse that occurs after a medical procedure, such as a surgery.

We do not consider a short-term complication related to the act itself, but a long-term relapse that clinicians cannot explain easily, since it depends on unknown sets or sequences of past events that occurred before the act. The relapse is observed only indirectly, in a "blurry" fashion, through longitudinal prescriptions of drugs over a long period of time after the medical act. We introduce a new model, called ZiMM (Zero-inflated Mixture of Multinomial distributions) in order to capture long-term and blurry relapses. On top of it, we build an end-to-end deep-learning architecture called ZiMM Encoder-Decoder (ZiMM ED) that can learn from the complex, irregular, highly heterogeneous and sparse patterns of health events that are observed through a claims-only database. ZiMM ED is applied on a "non-clinical" claims database, that contains only timestamped reimbursement codes for drug purchases, medical procedures and hospital diagnoses, the only available clinical feature being the age of the patient. This setting is more challenging than a setting where bedside clinical signals are available. Our motivation for using such a non-clinical claims database is its exhaustivity population-wise, compared to clinical electronic health records coming from a single or a small set of hospitals. Indeed, we consider a dataset containing the claims of almost all French citizens who had surgery for prostatic problems, with a history between 1.5 and 5 years. We consider a long-term (18 months) relapse (urination problems still occur despite surgery), which is blurry since it is observed only through the reimbursement of a specific set of drugs for urination problems. Our experiments show that ZiMM ED improves several baselines, including non-deep learning and deep-learning approaches, and that it allows working on such a dataset with minimal preprocessing work.

Introduction

The increasing volume of Electronic Health Records (EHR) systems of national medical organizations in recent years allows to capture data from millions of individuals over many years. Each individual's EHR can link data from many sources and hence contain "concepts" such as diagnoses, interventions, lab tests, clinical narratives, and more. This provides great opportunities for data scientists to collaborate on different aspects of healthcare research by applying advanced analytics to these EHR clinical data [Rajkomar et al., 2018].. There are several challenges in processing EHR data [START_REF] Cheng | Risk Prediction with Electronic Health Records: A Deep Learning Approach[END_REF]: data quality, high-dimensionality, temporality which refers to the sequential nature of clinical events, sparsity in both medical codes representation and in timestamp representation, irregularly timed observations, biases such as systematic errors in data collection, and mixed data types with missing data. Representation learning can overcome these challenges [Rajkomar et al., 2018] and the choice of data representation or feature representation plays a significant role in the success of this approach.

Related works

Recently, much work has been done on developing compact and functional representations of medical records, including the use of deep learning over EHR data [Bajor et al., [START_REF] Bender | HL7 FHIR: An agile and RESTful approach to healthcare information exchange[END_REF]) to encode EHR information for several different sequence-oriented models.

RETAIN [Choi et al., 2016c] and GRAM [START_REF] Choi | GRAM: Graph-based attention model for healthcare representation learning[END_REF] are two state-ofthe-art models using RNNs for predicting future diagnoses. However, they cannot handle long sequences effectively. LSTM or bidirectional RNNs [START_REF] Ma | Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks[END_REF]] can be trained using all available input information in the past and future, and have been used to alleviate the effect of the long sequence problem and improve the predictive performance. The work of [START_REF] Lipton | Learning to diagnose with LSTM recurrent neural networks[END_REF] has been the first to successfully use LSTM to predict patients diagnosis, for multi-label classification. BEHRT [START_REF] Li | BEHRT: Transformer for Electronic Health Records[END_REF] is one of the most recent researches that provides the field with an accurate predictive model for the prediction of next diseases based on the transformer-based architecture in NLP [Devlin et al., 2018].

Although fixed timesteps are perfectly suitable for many RNN applications, EHRs often contain event-driven and asynchronously sampled samples, in particular for the application considered in the present paper. Recent works try to capture time irregularity using recurrent neural networks. Phased LSTM [START_REF] Neil | Phased LSTM: Accelerating recurrent network training for long or event-based sequences[END_REF] tries to model the time information by adding a time gate to LSTM, which controls the update of the cell state, the hidden state and thus the final output. Another attempt is [START_REF] Zhu | What to do next: Modeling user behaviors by time-lstm[END_REF] where time gates are added to the LSTM in order to model time intervals and specifically better capture both of short-term and long-term sequential events. Time-aware LSTM [START_REF] Baytas | Patient subtyping via time-aware lstm networks[END_REF] addresses time irregularity between two events in an LSTM architecture by decomposing the memory of the previous timesteps into short-term memory and long-term memory. Let us point out that, however, most of the works on EHR data cited above either ignores subsequence-level irregularity by partitioning the data into regular time windows and by aggregating the data within each window or handles this irregularity by simply adding the time-span as one coordinate of the features vector, see Choi et al. [2016a] for instance.

Aim of this chapter and contributions

This chapter aims at the construction of a predictive model for long-term and "blurry" relapses that occur after a medical act. We do not consider a short-term complication related to the act itself, but a long-term relapse which is observed only indirectly. In the example considered here, the medical act is a precise surgery, and the relapse is observed through longitudinal prescriptions of a specific set of drugs for urinary problems, over a period of 18 months after surgery. We use all the data available in SNIIRAM (French national health insurance information system, a huge database containing health reimbursements claims of almost all French citizens since 2015, see Section 2.3.1 for details) for patients on which such a surgery has been performed (TURP surgery, see Section 2.3.2).

The ZiMM model. The first natural idea is to cast this problem as a classification problem for "relapse" versus "no relapse" after the medical act. However, such a naive approach cannot work here. Indeed, the definition of a binary label would require threshold choices, both about time and dosage: after how many time and amount of drugs prescribed do we consider that there is a relapse? Such thresholds might depend on various clinical practices, habits of the patient, and many other exogenous factors. In this work, we propose to work directly on the data observed, by using all the "blurry" observations at once to train an end-to-end architecture. For this purpose, we introduce a new methodological contribution, namely the ZiMM model (Zero-inflated Multinomial Mixture) which is described in Section 2.2.1 below.

The ZiMM Encoder-Decoder (ZiMM ED).

We introduce an end-to-end Encoder-Decoder architecture trained with an objective, based on the negative loglikelihood of the ZiMM model. All available patients' claims before the medical act are first encoded into a single embedding vector by the Encoder. This Encoder combines embedding layers, self-attention layers and recurrent layers to output an embedding vector of the full patient pathway prior to the medical act. This embedding is used as input to a Decoder which is dedicated to the learning of the parameters of the ZiMM model. This architecture is described in Sections 2.2.3 to 2.2.5 below.

Specifics of this work.

The specificity of our approach lies in several points. First of all, it is performed on non-clinical EHR data, and we consider long-term predictions (18 months ahead). Our end-to-end architecture addresses several challenges, such as variable-size inputs, confounding interactions between medical codes, long-term predictions. Finally, we do not inject any prior knowledge: the embeddings of patient pathways are fully trained in an end-to-end fashion, we do not use any strong preprocessing or simplifying aggregations, the data is used almost in its raw form. We keep the smallest granularity possible both on the codes (we use the actual medical codes instead of encompassing categories) and on time, namely we work on the original 1-day time scale.

Organization of the chapter. The ZiMM model is described in Section 2.2.1 while the ZiMM ED architecture is described in Sections 2.2.3 to 2.2.5. Some details concerning data preprocessing are provided in Section 2.2.2. Section 2.3 provides results and a comparison with strong baselines, together with some variations of the architecture to fine-tune the final performance and we conclude in Section 2.4.

Proposed architecture

The ZiMM ED architecture has three main building blocks: the ZiMM (Zero-Inflated Multinomial Mixture) model (see Section 2.2.1), an Encoder (see Section 2.2.3) and a Decoder (see Section 2.2.4), and is described in Figure 2.1 below. Given a patient i ∈ {1, . . . , n} (among n patients), with medical act of interest occurring at time T i , the steps through this architecture are roughly as follows:

• All the medical codes (drugs, diagnosis, medical acts) observed longitudinally in the life of patient i before T i are embedded, then aggregated in a time window using a self-attention mechanism. The time distance (in days) to T i of each day with a non-empty set of codes is embedded as well, and we embed as well hospital stay durations. This sequence of vectors is then fed to a (or several) recurrent layers. This corresponds to Steps 1,2 and 3 in Figure 2.1. This leads to an embedding vector x i ∈ R d of the full pathway of patient i prior to T i .

• The vector x i ∈ R d is concatenated with other static features of the patient, and used as the input of the Decoder, which is based on several recurrent layers, in order to produce the parameters of the ZiMM model. This corresponds to

Step 4 in Figure 2.1.

Precise descriptions of all steps are provided below, starting with the ZiMM model, followed by some details concerning data preprocessing, and the Encoder-Decoder architecture.

ZiMM: Zero-inflated Multinomial Mixture model

Assume for now that we have an embedding vector x i ∈ R d that encodes the full health pathway of patient i ∈ {1, . . . , n} prior to T i . This vector is the output of the ZiMM-Encoder described in Section 2.2.3 below. Moreover, we observe a vector of labels y i = [y i,1 , . . . , y i,B ] ∈ N B , which corresponds to the blurry observation of the relapse after T i . Here, B corresponds to the number of time intervals considered after T i and y i,b 1 is the number of blurry relapses observed in time bucket b, so that y i,b = 0 means that no blurry relapse is observed in time bucket b. In the example considered Section 2.3 below, B = 18 corresponds to a 18-months period and y i,b is the number of drugs (among a set of drugs for urinary problems, see Section 2.3.3), purchased by patient i in time bucket b.

We introduce n i = B b=1 y i,b ∈ N, the overall number of blurry relapses of patient i. Let us point out that a binary classification problem (n i > 0 versus , B -1). The first case can be due to exogenous factors, such as the patient simply kept buying the drug after the surgery just out of a pure habit (prescriptions can run for a long period), while in the second example, there is no doubt that a relapse is occurring. Therefore, we need to find a way to model the whole vector y i , so that n i is not fixed and includes zero-inflation, namely a parametrized likelihood for n i = 0. For that purpose, we introduce the following ZiMM model, where we suppose that n i ∈ {0, 1, . . . , B} is distributed (conditionally to x i ) as

n i ∼ Categorical(π 0 (x i ), π 1 (x i ), . . . , π B (x i )),
which means that

P(n i = k|x i ) = π k (x i ) for k ∈ {0, . . . , B}, where π b (x i ) are such that B b=0 π b (x i ) = 1 and π b (x i )
0 (coming out of a softmax activation for instance). These parameters correspond to the categorical distribution specific to patient i, and are constructed by the ZiMM-Decoder (see Section 2.2.4 below) from x i . Then, we assume that the distribution of y i conditional to n i = b and x i follows either a Dirac distribution on vector (of size B) [0, . . . , 0] whenever b = 0, or a multinomial distribution of parameters b and p b,1 (x i ), . . . , p b,B (x i ), namely

y i |(x i , n i = b) ∼ δ [0,...,0] if b = 0, Multinomial(b, p b,1 (x i ), . . . , p b,B (x i )) otherwise,
where p b,1 (x i ), . . . , p b,B (x i ) are the parameters of a multinomial distribution whenever n i = b, for each b = 1, . . . , B. Once again, these parameters are specific to patient i thanks to the embedding vector x i ∈ R d , and are constructed by the ZiMM-Decoder. These parameters must satisfy B b =1 p b,b (x i ) = 1 for b = 1, . . . , B, and p b,b (x i ) 0 for any b, b = 1, . . . , B, which is easily achieved with a softmax activation applied row-wise. The distribution of y i conditionally on x i is therefore given by the following mixture model:

y i |x i ∼ π 0 (x i )δ [0,...,0] + B b=1 π b (x i )Multinomial(b, p b,1 (x i ), . . . , p b,B (x i )).
Zero-inflation corresponds to the case where n i = 0 and has likelihood measured by π 0 (x i ). The ZiMM model has two sets of parameters that are functions of the feature vector x, namely {π b (x)} b∈{1,...,B} that parametrizes the distribution of the total number of blurry relapses and {p b,b (x)} b,b ∈{1,...,B} 2 that parametrizes the dynamics of these relapses. As explained in Section 2.2.4 below, the dynamics of {p b,b (x)} b ∈{1,...,B} for each b are learned by dedicated recurrent layers in ZiMM-Decoder.

Preprocessing of the data

After a preliminary preprocessing based on our SCALPEL3 library described in Bacry et al. [2019], the data considered in this paper comes in the form of a table, where each row represents an event with the patient ID, the type of event (drug, diagnosis or medical procedure), its corresponding code, the start date and the end date of the event. Figure 2.2 provides an illustration of the sequence of claims observed for a single patient. Through claims, we observe three types of events: drug purchases (blue), medical procedures (red) and diagnosis (yellow) before the medical act (prostate surgery in the example) that happens at time T i for patient i. All events are timestamped, and the time delta is a day long. Several events can occur the same day and some days have no event: events are typically very irregularly sampled. All these events, observed before T i , are used to learn the embedding vector x i ∈ R d of patient i. After T i , we only keep the events corresponding to the blurry relapses considered (drug purchases among a set of drugs for urinary problems). These blurry relapses are used to build the label vector

y i ∈ N B of patient i.
If an event is related to a hospital stay, the start date is the first day of hospitalization while the end is the exit date from the hospital. Otherwise, the start and end dates are the same. The time delta is a day-long, and no aggregation is performed to keep the data as raw as possible.

Multiple events can occur within the same day, and the precise ordering of such events does not carry any information. Thus, we shuffle at random within-day events, to avoid any bias that could occur from the data collection mechanism (a similar strategy can be found in Choi et al. [2016d]).

All the events observed through claims before T i are used as inputs to the Encoder described in Section 2.2.3 below. The output of the Encoder is the embedding vector x i ∈ R d of patient i. The blurry relapses observed after T i are used to build the label vector y i ∈ N B of patient i.

The ZiMM Encoder

The ZiMM Encoder corresponds to Steps (2) to (4) in Figure 2.1, a more detailed illustration is shown in Figure 2.3. Let us provide now details about each step, following the flow of the data.

Medical codes and timestamps embeddings.

The following is applied separately for each type of event code (drugs, medical procedures and diagnoses). Codes are first tokenized, and each unique token is individually mapped to an embedding vector in R d E which is learned during training, where d E is a hyper-parameter to be tuned (see Section 2.3.7). We consider only tokens that occur at least 50 times in the training dataset. For any event occurring at time t T i in the observation period of patient i, we compute the "time horizon" as T i -t, namely the distance (in days) between the event and medical act. Moreover, whenever it makes sense, we compute endstart, the duration of the event, which corresponds to the duration of an hospital stay defined as the time between hospital admission and discharge. These two integers (in number of days) are also tokenized and replaced by a learned embedding vector. This allows the encoder to learn to put more or less emphasis on events that are close or far from T i , and to exploit the duration of events as a proxy for severity of medical procedures and diagnoses. The patient age in years at T i is also similarly bucketized and embedded. The embedding of medical codes corresponds to Step (2) of both Figures 2.1 and 2.3. In our experiments, the default model uses d E = 64 for the three types of codes and several regularization strategies are applied to avoid overfitting: weights decay ( 2 regularization with strength 10 -2 ), multiplicative Gaussian noise, and layer normalization [START_REF] Ba | Layer Normalization[END_REF]]. Since we observed empirically that these embeddings are prone to overfitting for the application considered in the paper, a comparison of the different regularization strategies is provided in Section 2.3.7.

Embeddings aggregation through self-attention. Several events can occur at the same time (within the same day), so that the number of codes observed within a day is highly heterogeneous. Moreover, such codes are not likely to contribute equally to the vector representation of the day, and their order is not informative. Therefore, we need to perform a trainable aggregation of these codes in order to produce a representation of the patient history at each timestamp. An approach can consist of using a hierarchy relationship between different diagnosis and treatment inside each patient visit [START_REF] Choi | Graph Convolutional Transformer: Learning the Graphical Structure of Electronic Health Records[END_REF]. However, in the data considered here, diagnosis codes are not explicit each time there is treatment; in particular, drugs ... purchases are almost never related to a diagnosis code (unless they are related to a hospitalization). Hence, we use a self-attention mechanism [START_REF] Lin | A structured self-attentive sentence embedding[END_REF]Vaswani et al., 2017] using a bag-of-features approach to learn how to combine embedding vectors within the same day, following previous successful applications of self-attention for fusing disease embeddings [Luo et al., 2019].

Let C be a set of codes with cardinality |E| (for drugs, medical procedures or diagnoses) and let E C ∈ R d E ×|C| be the corresponding matrix of concatenated embedding vectors. The multi-headed self-attention aggregation mechanism contains two layers, the first with K heads, each of which is a self-attention function that generates a probability vector of size |E| from E C :

w k = softmax(α k tanh(A k E C )),
(2.1) We use again dropout on the inputs and on the recurrent units to prevent overfitting. Finally, the output vector of the encoder is concatenated with an embedding vector of the age of the patient, leading to the final vector x i ∈ R d that encodes the full pathway of patient i before T i . This vector is the input of the ZiMM Decoder described in the next Section.

for k = 1, . . . , K, where α k ∈ R 1×d E and A k ∈ R d E ×d E are

The ZiMM Decoder

The decoder uses the input vector x i ∈ R d of patient i to construct the parameters of the ZiMM model, and the whole architecture is trained against the negative log-likelihood of the ZiMM model from Section 2.2.1 computed at the label vector y i = [y i,1 , . . . , y i,B ] containing the blurry relapses. The parameters {π b (x i )} b=1...,B and {p b,b (x i )} b,b ∈{1,...,B} 2 are highly dependent and are time-ordered, so that a specific architecture is used to model these dependencies. The mixture probabilities are learned through a fully connected feed-forward layer (FFN) that use as input x i , namely

π 0 (x i ), . . . , π B (x i ) = softmax(FFN(x i )),
and another recurrent layer (RNN) uses as well x i in order to produce hidden states given by

h t = RNN(h t-1 , x i ) (2.3) for t = 1, . . . , B.
Then, we use different recurrent layers (RNN b ) in parallel for each value n i = b ∈ {1, . . . , B} using

h b t = RNN b (h b t-1 , h t )
for t = 1, . . . , B, since the multinomial distributions vary strongly conditionally on n i = b. Finally, a softmax activation is applied on h b t along t = 1, . . . , B to produce the parameters p b,1 (x i ), . . . , p b,B (x i ). Details on the type of layers used and the tuning of hyperparameters is provided in Section 2.3 below.

Training

The full ZiMM Encoder-Decoder architecture is trained in an end-to-end fashion by minimizing the average negative-log likelihood over all patients i = 1, . . . , n:

i (Θ) = log(π 0 (x i ))1 n i =0 + B b=1 log(π b (x i )) + log b! Π B b =1 y i,b ! + B b =1 y i,b log(p b,b (x i )) 1 n i =b ,
where Θ stands for the concatenation of all the trainable parameters involved in the layers described in Sections 2.2.3 and 2.2.4. The choices of optimizer, learning rate schedule and other hyperparameters are described in Section 2.3.

Application: prediction of post-surgical relapse of urinary problems

In this section, we apply the ZIMM ED architecture to predict the blurry relapse of urinary problems after a TURP surgery for patients with benign prostatic hyperplasia (see Section 2.3.2 below) using a cohort constructed from the French SNIIRAM database (see Section 2.3.1 below). Then, we explain in details the way the labels are built (Section 2.3.3), the different steps involved in the cohort construction (Section 2.3.4), the evaluation metric and implementation details (Section 2.3.5).

Then, the remaining of this section describes the baselines and a comparison with ZiMM ED, followed by an ablation study. Let us stress that while this data is extremely rich and almost population-wide over France, it is not clinical data, but only claims data, clinical information being only latent in the codes appearing in reimbursement information. No vital bedside information, lab tests results or natural language notes from clinicians is available. An important amount of data preprocessing and domain expertise is required for data extraction, in particular in order to clean existing inaccuracies in codes and timestamps. This is achieved through our SCALPEL3 library [START_REF] Bacry | SCALPEL3: a scalable open-source library for healthcare claims databases[END_REF], which is a separate topic of research that this paper builds upon. Furthermore, the identification of a disease state (indication, concomitant disease or outcome) does not rely only on a single source of information but on the convergence of information from different sources. This might include, for instance, the presence of a chronic disease registration, hospital diagnoses, tracer drugs, and procedures or lab tests [START_REF] Bezin | The national healthcare system claims databases in France, SNIIRAM and EGB: Powerful tools for pharmacoepidemiology[END_REF], that are all observed in the data through claims.

SNIIRAM: A non-clinical claims dataset

We believe that these downsides are counter-balanced by the exhaustive nature of this data. It is exhaustive both in terms of population (it includes nearly all the population living in France) and in terms of healthcare events. Indeed, in France, almost all healthcare spendings are at least partially reimbursed by the government, so almost each healthcare of each individual is associated to at least one event in the database. This makes it a very rich and powerful database, with a relatively small bias, and many works have already proved successful in improving population-level health [Atramont et Let us note that, since 2016, the access to this database, for public interest research, is possible through the SNDS access pipeline [SNDS, 2019]. It mainly offers access through the SAS software. An access using classical open source big data and AI frameworks (e.g. R, Python, Spark) including the SCALPEL3 library will be possible, by the end of 2020, through the Health Data Hub Hub [2019], a 80M$-funded national project which aims to be the national unique gateway to most French health data for operating public interest research (operated by both public or private entities) on modern infrastructures.

Benign prostatic hyperplasia (BPH)

BPH is a common urological disease that affects aging men all over the world [START_REF] Dahm | Comparative Effectiveness of Newer Medications for Lower Urinary Tract Symptoms Attributed to Benign Prostatic Hyperplasia: A Systematic Review and Meta-analysis[END_REF][START_REF] Silva | Current medical treatment of lower urinary tract symptoms/BPH: do we have a standard?[END_REF]. It causes urinary tract obstruction due to the unregulated growth of the prostate gland, causing lower urinary tract symptoms (LUTS) [START_REF] Kim | Management of Benign Prostatic Hyperplasia[END_REF]. The options for management of BPH include watchful waiting, pharmacotherapy, transurethral resection of the prostate (TURP), and other minimally invasive surgical treatments (MISTs) and open prostatectomy. Most studies consider TURP as the gold standard for surgical management of BPH [START_REF] Hashim | Transurethral resection of the prostate for benign prostatic obstruction: Will it remain the gold standard?[END_REF]. Patients suffering from prostatic hyperplasia regularly take drugs for urination problems. Successful surgery should cure such problems so that patients should not need to continue their drugs for urination problems. However, it is often observed that patients retake such drugs after surgery. In some cases, it merely comes from a habit of taking routine drugs, but in many cases, it is related to a persisting urination problem [START_REF] Chung | Update on minimally invasive surgery and benign prostatic hyperplasia[END_REF][START_REF] Macey | Medical and Surgical Treatment Modalities for Lower Urinary Tract Symptoms in the Male Patient Secondary to Benign Prostatic Hyperplasia: A Review[END_REF]].

An important problem, from a clinical point of view, is, therefore, to predict the outcome of such a surgery, on a long-time period following it (18 months is considered here), to improve the decision making of the clinicians, in particular, to help decide when surgery should be performed. Use of long-term data after TURP surgery is very scarce in the literature, with only a few studies available [START_REF] Cornu | A systematic review and meta-analysis of functional outcomes and complications following transurethral procedures for lower urinary tract symptoms resulting from benign prostatic obstruction: An update[END_REF]. A recent analysis of 20 contemporary randomized clinical trials with a maximum follow-up of five years reports that TURP resulted in a significant improvement of maximum flow-rate and quality of life [START_REF] Cornu | EAU Guidelines: Management of Nonneurogenic Male LUTS | Uroweb[END_REF][START_REF] Lourenco | The clinical effectiveness of transurethral incision of the prostate: a systematic review of randomised controlled trials[END_REF]. A second prostatic operation (re-TURP) has been reported at a constant annual rate of approximately 1-2%. A review analyzing 29 RCTs found a re-treatment rate of 2.6% after a mean follow-up of sixteen months [START_REF] Cornu | EAU Guidelines: Management of Nonneurogenic Male LUTS | Uroweb[END_REF]. In a large-scale study of 20,671 men, the overall re-treatment rates (i.e., either re-TURP, urethrotomy or bladder neck incision) were 5.8%, 12.3%, and 14.7%, respectively at one, five, and eight years follow-up. More specifically, the respective incidence of re-TURP was 2.9%, 5.8% and 7.4% [START_REF] Madersbacher | Prostate Study Group of the Austrian Society of Urology. Reoperation, myocardial infarction and mortality after transurethral and open prostatectomy: a nation-wide, long-term analysis of 23,123 cases[END_REF]. However, urology guidelines highlight the lack of extended follow-up after the surgery and no clear evidence explaining re-treatment.

Building a model with the ability to predict the outcome of this surgery is of primary importance for improving population-level health, especially since prostatic problems are a common condition for aging men. In this section, we use ZiMM ED to train such a model using a cohort based on SNIIRAM. The outcome of this surgery is evaluated by the distribution of the blurry relapses, observed through the use of specific drugs related to urination problems.

Surgery identification and labels construction

The identification of a TURP surgery in SNIIRAM is made through some specific CCAM codes provided by clinicians2 . However, it happens that two TURP surgeries can occur in a pretty small amount of time. This is likely to correspond to a case where a surgery has clearly failed and that a second surgery is required quickly, which is not the type of complication we are interested in. If the amount of time between the two surgeries is small, it would be improper to say that it corresponds to a relapse. We thus define, following clinicians recommendations, a six week period after the first surgery, as a single surgery block. If another TURP surgery occurs inside the same block, we consider that it is part of the same medical act. The timestamp of the event corresponds to the last surgery within the block, and provides the value of T i for any patient i.

As far as the labels are concerned, a simple but efficient way to identify whether the urinary problems have not ceased or reappeared after a while is to see if the patient, at some point after the surgery, needs to take medications for these urination problems again3 . For that purpose, we use a list (provided by clinicians) of 136 CIP-13 drugs that are mainly related to urination problems. We choose to drive the prediction on a 18-months period after T i . So, using the notation of Section 2.2.1, we chose the number of buckets B = 18 and bucket size of 30 days (540 days total).

Cohort construction and exploration

The construction of the cohort follows the flowchart illustrated in Figure 2.4. We start by selecting all SNIIRAM patients alive, with at least one medical event re-lated to urination problem (observed through medication or surgery codes) between 2010/01/01 and 2015/12/31. This selection gathers a little bit more than 5 million patients. Then, we keep only male patients over 18 years old on 2014/12/31. This reduces the size of the cohort by roughly 1 million patients. Keeping only patients with a TURP surgery between 2010/01/01 and 2013/06/30 and with at least 2 events (whatever the types) occurring in the follow-up 18-months period leads to a cohort containing 231 747 patients.

As explained in Section 2.3.3, we compute surgery blocks (any surgery that occurs within a six week period after the first surgery is in the same block), that are considered as a single event of surgery. We remove all the patients that have another surgery out of this first block. We choose a follow-up period of 540 days, corresponding to 18 buckets of 30 days, so that the follow-up period is of the order of 18 months. Indeed, clinicians expect that such a relapse should occur not long after the first year following surgery. As shown in Figure 2.4, the final cohort has 138 976 patients. The numbers of medical codes, namely medications, procedures and diagnoses observed in the final cohort, are displayed in Table 2.1. We count the number of unique codes, the number of codes observed at least 50 times and the number of codes remaining when keeping only the ones prior to T i for each patient i. The statistics reported in Table 2.1 below and in all figures from this section use the raw coding scheme (that use CIP-13 for drugs), in order to better describe the statistics of the data, while for training ZiMM ED we replace CIP-13 by ATC (for drugs) since, as illustrated in Section 2.3.7 below, it allows to reduce the vocabulary size of drugs while keeping the same predictive performance.

On the left-hand side of Figure 2.5, we show, for this final cohort, the distri- bution of n i , namely the number of patients with a given total number of drugs prescriptions (related to urinary problems) during the follow-up 18 months period. Namely, the y-axis of Figure 2.5 (left-hand side) displays #{i : n i = b} where b = 1, . . . , B is given by the x-axis. We observe a sharp decrease for b = 1, . . . , 5 from the mode b = 1, which corresponds to patients who continue to buy their drugs and just keep doing it (for several months) after their surgery. However, we observe a second mode around b = 15 which certainly corresponds to the relapse we are interested in. This plot shows that the considered problem is in a "weak signal" setting, since the second mode is very small compared to the first (at b = 1), and that the prediction of this relapse requires a dedicated methodology indeed. This observation is corroborated by the right-hand side of Figure 2.5, which displays the number of patients having at least one drug prescription (related to urinary problems) on a specified bucket, namely the y-axis displays #{i, y i,b ≥ 1} as a function of b given by the x-axis. Once again, the function is decreasing from the mode b = 1 and plateaus between b = 7 and b = 18, because of patients who continue to purchase the drug after the surgery. We do not observe a second mode as we did in the left-hand side, since the relapse we are interested in is blurry: the drug purchases that define this blurry relapse are heterogeneously distributed among patients and end up flattened out when aggregated in the display of the right-hand side. This motivates, once again, the dedicated methodology proposed by the ZiMM model. In the left-hand side of Figure 2.6, we show the distribution of the observation period of each patient before T i . We observe that it is highly heterogeneous: the maximum is reached at 1 274 days while the average is 486 days, and some patients have a very short observation period. This variability is related to the variability of the medical practice itself: some patients are treated for urological problems with drugs for a long time before the surgery, while other patients have surgery sooner. On the right-hand side of Figure 2.6, we show the distribution of the durations of all hospitalizations before T i . We observe that most hospitalizations are one-day short, and that the distribution is heavy-tailed, since these hospitalizations can be related to a large set of possible health problems, leading to very heterogeneous durations. Let us stress that we are considering all hospitalizations that occur before T i , and not only the ones related to a surgery or a urological problem, since all the health events of a patient i are kept before T i . The strong variability of the observation period displayed here is related to the variability of the medical practice itself: some patients are treated for urological problems with drugs for a long time before the surgery, while other patients have surgery sooner. Right-hand side. Distribution of the durations of all hospitalizations before T i . Most hospitalizations are one-day short, and the distribution is heavy-tailed, since these hospitalizations can be related to a large set of possible health problems.

In the left-hand side of Figure 2.7, we display the distribution of the number of unique events per patient. We observe that the bulk of the distribution is between 5 and 200 unique events, with an average of 10 unique events. The right-hand side of Figure 2.7 shows the number of days with at least one medical code in the history of patients before T i . We observe that most patients have more than 10 days in their history with medical codes. Both displays from Figure 2.7 show that the health pathways of most patients before T i contain enough variability when assessed by the number of distinct codes, and the number of distinct days with at least one medical code, corresponding to a sufficient amount of variability to carry information for the prediction of the blurry relapse after T i . The bulk of the distribution is between 5 and 200 unique events, with an average of 117 events per patient. Right-hand side. Distribution of the number of days with at least one medical code in the history of patients before T i . We observe that most patients have more than 10 days in their history with medical codes. Both figures show that most patients have a significant amount of medical code variability and time variability before T i .

Implementation and evaluation metrics

Our models are implemented with TensorFlow 2 [START_REF] Abadi | Ten-sorFlow: Large-scale machine learning on heterogeneous systems[END_REF]. The ZiMM ED architecture is open-source in GitHub4 . All the models are trained on a machine equipped with 4 GeForce GTX 1080Ti GPUs and another machine with 3 Tesla V100 GPUs. All the models are trained with the Nadam optimizer [START_REF] Dozat | Incorporating Nesterov Momentum into Adam[END_REF][START_REF] Kingma | A method for stochastic optimization[END_REF] with learning rate 0.001. The hyper-parameters of all the models are selected through an extensive random grid search, whereas in order to reduce computation time, some hyper-parameters are fixed "by hand". An ablation study showing the sensitivity to hyper-parameters is provided in Section 2.3.7 below.

Metrics: mean-AP, AUC-PR and AUC-ROC. In order to evaluate the quality of the prediction of y i = [y i,1 , . . . , y i,B ] (when using ZIMM ED architecture or any other benchmark models) we use mean-AP, defined as the average of the area under the precision-recall curve (AUC-PR), namely we compute the average over the buckets b = 1, . . . , B of the AUC-PR for each bucket b, i.e., the AUC-PR of the prediction of y i,b . Moreover, we report also the AUC-PR and the AUC-ROC (area under the ROC curve) for the binary classification problem n i > 0 against n i = 0.

Train, validation and test sets. All our experiments use the same random data splitting into 70% of patients for training, 15% of patients for validation and 15% for testing. We checked the stationarity across the three splits of labels distribution and the main drugs, diagnoses and medical procedures. We report performances on both the validation and test sets.

Baselines

The prediction performances of ZIMM-ED is compared with several baselines, involving several featuring strategies and different predictive models, both for binary prediction (n i > 0 versus n i = 0) using the AUC-PR and AUC-ROC metrics and for the prediction of y i = [y i,1 , . . . , y i,B ] (the mean-AP score defined in Section 2.3.5).

Results are reported in Table 2.2 below, where we observe that ZIMM ED improves all the considered baselines, in particular for the prediction of y i , since the ZiMM model is dedicated to this task. We consider the following baseline models, more details are provided below:

• LRl2: Logistic regression with 2 penalization using the scikit-learn library [Pedregosa et al., 2011];

• GBDT: Gradient boosting using XGBoost [Chen et al., 2015];

• MLP: Multilayer Perceptron model with 128 hidden units;

• Word2vec: we first train embeddings of all medical codes following [Mikolov et al., 2013], and use these pre-trained embeddings in a MLP with 128 units;

• LSTM: a single embedding layer and one forward LSTM layer with 128 hidden units;

• Patient2Vec [START_REF] Zhang | Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record[END_REF].

We evaluated them using 3 types of input features described below.

Static features (SF).

This featuring is used for LRl2, GBDT and MLP models. Inputs correspond to aggregated counts of grouped medical codes over the entire observation period of a patient. Number of occurrences of each code is then multiplied by the corresponding one-hot encoding. Hence, the input is a N -dimensional vector representing a patient's medical history. One logistic regression (LRl2) and one gradient boosting decision tree (GBDT) is trained for each bucket b. This input is also used for the multi-layer perceptron (MLP).

Dynamic features (DF).

This featuring is used for LRl2, GBDT and MLP models. We split the sequence into subsequences of 60 days, on which we compute the same features as with SF, but within each interval, in order to incorporate longitudinal information into LRl2, GBDT and MLP.

Irregularly-spaced sequence (ISS). This featuring is used for Word2vec, LSTM and Patient2Vec models. In this case, we consider the original patient sequence.

For both the Word2vec model and the LSTM one, the events are just gathered in a sequence in which the time-stamps or duration of the events are not used.

In the Word2vec case, the codes embeddings are trained using Word2vec [Mikolov et al., 2013] and prediction is performed with a MLP with 128 hidden nodes. In the LSTM case, an embedding layer and one forward LSTM layer with 128 hidden units are trained in an end-to-end fashion and is applied to the sequence. The Pa-tient2vec uses the same input features as the ZIMM ED architecture (i.e., sequence, including time-stamps and duration of events). It has been only used for the binary classification problem (i.e., n i > 0 versus n i = 0).

The predictive performance of all benchmarks and of ZiMM ED are presented in Table 2.2, where we report mean-AP scores on the test set, as well as AUC-PR and AUC-ROC scores for the binary classification problem (see Section 2.3.5 for definition of these scores). According to this table, GBDT-based models on dynamic features (GBDT-DF) performs the best among all benchmark models, however the ZIMM ED architecture outperforms all the benchmark models both for the multi-output y i prediction and for the binary prediction.

Ablation study: model performances and variations

The performance reported in Table 2.2 for the ZiMM ED architecture relies on a careful tuning of several hyper-parameters. This corresponds to the so-called ZiMM ED default architecture, where the hyperparameters used are described in The hyper-parameters described in Table 2.3 have been selected using an extensive random search for the best mean-AP metric on the validation set. In Figure 2.8 below, we illustrate the value of this metric (y-axis) for some models that were evaluated during the random search. Each point corresponds to a single model with fixed hyper-parameters, and the set of models is the same on all four displays of Figure 2.8. In each of these displays we "align" all the models that share a specific hyper-parameter, namely, respectively from left to right: the number of heads used in the self-attention layer, the number of hidden units used in the recurrent layer of the ZiMM Encoder, the number of stacked recurrent layers in ZiMM Encoder and finally the number of hidden units used in the ZiMM Decoder. The red point on all four Figures corresponds to the best model overall, that leads to the ZiMM ED default architecture described in Table 2.3. Finally, note that the random search included many other hyper-parameters, such as several learning-rate scheduling strategies, dropout rates, types of dropout regularization including input, output, embedding, cell-state and multiplicative Gaussian, that we do not report for the sake of conciseness. where each point corresponds to a single model. The set of models is the same on all four displays and the best one (ZiMM ED default) is indicated by a red point. Each display aligns all the models that share a specific hyper-parameter, being, respectively from left to right, the number of heads used in the self-attention layer, the number of hidden units used in the recurrent layer of the ZiMM Encoder, the number of stacked recurrent layers in ZiMM Encoder and finally the number of hidden units used in the ZiMM Decoder.

The remaining of this section proposes an ablation study: we modify some hyperparameters or change some components of ZiMM ED default, and report the impacts on performances in Table 2.4 below, where the first line reports the performances of ZiMM ED default. A discussion around these ablations is given below. 2.4 shows results obtained with variations of the maximum sequence lengths and the classification system used for drugs encoding. The ZiMM ED default uses ATC encoding instead of the raw CIP-13 encoding, which allows to reduce the vocabulary size from 10 664 for CIP-13 to 1 105 for ATC. As observed in Table 2.4, using CIP-13 (which is a much larger vocabulary for drugs) instead of ATC actually hurts strongly the performances. We observe also that considering longer sequences (100 days) instead of what we do in ZiMM ED default (50 days) deteriorates the performances as well.

Data preprocessing. The first part (PP) of Table

Embedding of medical codes and time.

Variations around the way embeddings are produced for medical codes and time are reported in the second part (Embedding) of Table 2.4. We observe that increasing the embedding dimensions does not offer performance increase (using E dim = 128 instead of the default E dim = 64), as well as using the same embedding space for all medical codes. In the ZiMM ED default model, tokenized durations and distance to T i (see Section 2.2.3 above) are embedded with trainable embedding vectors. We observe in Table 2.4 that without these time embeddings, the model performance deteriorates. Also, we find that using trigonometric functions for "positional encoding" as proposed in Vaswani et al.

[2017] instead of learned embeddings does not help in our setting.

Embeddings aggregation.

Results obtained through different embeddings aggregation techniques are presented in the third part (Aggregation) of Table 2.4. We applied penalization and dropout on attention weights as proposed in [START_REF] Lin | A structured self-attentive sentence embedding[END_REF], as well as DropConnect [START_REF] Wan | Regularization of neural networks using dropconnect[END_REF] for regularizing weights in large fully-connected layers. As explained in Vaswani et al. [2017], multi-head attention allows the model to jointly attend information from different representation subspaces at different positions, however in our setting, increasing N heads (number of heads) in the self-attention layer does not clearly improve the performance. We tested N heads = 8, as reported in Luo et al. [2019], but the best performance in our case was achieved with N heads = 3. ZiMM Encoder. We report in the fourth part (Encoder) of Table 2.4 the results of extensive experiments performed in order to identify the best combination of hyperparameters for both recurrent and convolutional layers (CNN): number of hidden units, types of recurrent layers and different dropout rates used in the recurrent layers. Only the best results for each type of layer are reported in Table 2.4. We observe that GRU layers perform generally as well as LSTM layers. Increasing the number of hidden units in recurrent layers to N units = 128 instead of the default choice N units = 64 does not significantly improve performance, as well as using two stacked LSTM layers N layers = 2 instead of the default N layers = 1. We observed also that in our setting, CNN layers badly under-perform compared to the other types of layers. Finally, we tested "Multi-head transformer" which corresponds to the transformer encoder [Vaswani et al., 2017] which takes as input all patient sequence of events. Clinical codes are embedded in the same way as for ZiMM model, which is then passed to the encoder block with a 4-head self-attention mechanism through the entire patient sequence. ZiMM Decoder. Finally, we consider some variations around the ZiMM Decoder, and report the results in the fifth part (Decoder) of Table 2.4. While the ZiMM Decoder described in Section 2.2.4 uses a single feed-forward layer to predict the mixture probabilities, a GRU layer to learn hidden states and GRU layers to predict the parameters of each multinomial distributions, one could use instead fully-connected layers to predict all the parameters of the ZiMM distribution, or alternatively simple RNN layers or LSTM layers. The fully-connected layers deteriorates the most the performance, while replacing the GRU layers by RNN or LSTM layers only deteriorates it mildly. We also report the performance obtained with two stacked GRU N layers = 2 instead of the default one N layers = 1 to produce the hidden states (see Equation (2.3)) and with a smaller hidden size N units = 18 instead of N units = 32.

Visualization of the embeddings produced for diagnoses and drugs codes

We explore the embeddings produced for diagnoses and drugs as a by-product of the ZiMM ED architecture (see Section 2. On the left-hand side of Figure 2.9, we can observe the UMAP projections of the embedding vectors of each ICD10 code, colored by the corresponding PheWAS phenotype. The hyper-parameters used here for UMAP are n-neighbors=20 and min-dist=0.1. Many points share the same color since many ICD10 codes have the same PheWAS. We observe here that diagnoses that are known to co-occur and belong to the same clinical groups are projected close to each other. The redorange-highlighted points are the phenotypes the most related to the urogenital system such as Hyperplasia of prostate, Retention of urine, Benign neoplasm of male genital organs, and Cancer of prostate. We can also observe a visually strong association of these phenotypes with Essential hypertension. This is confirmed by [START_REF] Michel | Association of hypertension with symptoms of benign prostatic hyperplasia[END_REF], where the association of hypertension and symptoms of benign prostatic hyperplasia is well-studied and understood. Other clinical concepts that are projected close to each other are Hearing loss and Chemotherapy, Urinary tract infection and Hematuria (blue-highlighted points), which is also confirmed by [START_REF] Lin | Association of Dialysis with the Risks of Cancers[END_REF]; Taborelli et al. [2019], where it is shown that renal dialysis is associated with a higher risk of cancer.

On the right-hand side of Figure 2.9, we observe the UMAP projections of drugs embeddings, colored by the first level of the ATC classification (first letter). The UMAP hyper-parameters used here are n-neighbors=50 and min-dist=0.2. The dark blue points correspond to Various ATC class (V), that mainly correspond to contrast agents for MRI. We observe that these points do not form any cluster but are dispersed over the whole space. This can be explained by the fact that the considered cohort contain only men with BPH (see Section 2.3.4), that often have an MRI of the prostate. Furthermore, we observe that the drugs from the class Genitourinary system and sex hormones (G), that include drugs for urination problems, form one cluster with other age-related medications from the class Nervous system (N), namely analgesics, anti-thrombotics, psycholeptics, psychotropic, and antiparkinsonian drugs.

Conclusion and future works

In this work, we propose ZiMM Encoder Decoder (ZiMM ED), an end-to-end deep learning model trained against the negative log-likelihood of the new ZiMM (Zero-Inflated Mixture of Multinomial) model, for the modeling of long-term and blurry relapses. This deep learning model is trained on a cohort based on a large electronic health record database, that contains only claims and no clinical data, from the whole French population. We show that it improves the performances of a large number of baselines, including the state-of-the-art, for the considered predictive task. ZiMM ED allows to represent the full health pathways of patients, using all available information, with minimal preprocessing. Therefore, this end-to-end architecture can be used for other tasks as well, through transfer learning or fine tuning of the model. Future works will consider a multi-task version of ZiMM ED (several types of relapses, or other types of events), and other improvements using alternative architectures for attention modeling [Dai et This work addresses the problem of predicting the blurry relapses of the TURP surgery, which is the first step towards an evidence-based approach using machinelearning to help the clinical decision. The next step is to exploit these predictions to help to decide the timing of the surgery: given the current health pathway of the patient, what is his probability of a relapse, so that it can help the clinician to decide when to perform the surgery, or to consider alternative treatments.

Abstract

Contrastive representation learning has been recently proved to be very efficient for self-supervised training. These methods have been successfully used to train encoders which perform comparably to supervised training on downstream classification tasks. A few works have started to build a theoretical framework around contrastive learning in which guarantees for its performance can be proven. We provide extensions of these results to training with multiple negative samples and for multiway classification. Furthermore, we provide convergence guarantees for the minimization of the contrastive training error with gradient descent of an overparametrized deep neural encoder, and provide some numerical experiments that complement our theoretical findings.

Introduction

The where we can observe that, quite surprisingly, the gap between unsupervised and supervised performance has been closed for tasks such as image classification: the use of a pretrained image encoder on top of simple classification layers, that are trained on a fraction of the labels available, allows to achieve an accuracy comparable to that of a fully supervised end-to-end training [Grill et Although the papers cited above introduce methods with considerable variations, they mostly agree on the following basic pretraining approach: provided a dataset, an encoder is trained using a contrastive loss whose minimization allows to learn embeddings that are similar for pairs of samples (called the positives) that are close to each other (such as pairs of random data augmentations of the same image, see Chen et However, despite growing efforts [Saunshi et al., 2019; Wang and Isola, 2020], as of today, few theoretical results have been obtained. For instance, there is still no clear theoretical explanation of how a supervised task could benefit from an upstream unsupervised pretraining phase, or of what could be the theoretical guarantees for the convergence of the minimization procedure of the contrastive loss during this pretraining phase. Getting some answers to these questions would undoubtedly be a step towards a better theoretical understanding of contrastive representation learning.

Our contributions in this paper are twofold. In Section 3.3, we provide new theoretical guarantees for the classification performance of contrastively trained models in the case of multiway classification tasks, using multiple negative samples. We extend results from Saunshi et al. [2019] to show that unsupervised training performance reflects on a subsequent classification task in the case of multiple tasks and when a high number of negative samples is used. In Section 3.4, we prove a convergence result for an explicit algorithm (gradient descent), when training overparametrized deep neural network for unsupervised contrastive representation learning. We explain how results from Allen-Zhu et al. [2019] about training convergence of overparametrized deep neural networks can be applied to a contrastive learning objective. The results and major assumptions of both Sections 3.3 and 3.4 are illustrated in Section 2.3 through experiments on a few simple datasets.

Related work

A growing literature attempts to build a theoretical framework around contrastive learning and to provide justifications for its success beyond intuitive ideas. In Saunshi et al. [2019] a formalism is proposed together with results on classification performance based on unsupervisedly learned representation. However, these results do not explain the performance gain that is observed empirically [Chen et al., 2020a; He et al., 2020] when a high number of negative samples are used, while the results proposed in Section 3.3 below hold for an arbitrary large number of negatives (and decoupled from the number of classification tasks). A more recent work [Wang and Isola, 2020] emphasizes the two tendencies encouraged by the contrastive loss: the encoder's outputs are incentivized to spread evenly on the unit hypersphere, and encodings of same-class samples are driven close to each other while those of different classes are driven apart. Interestingly, this work also shows how the trade-off between these two aspects can be controlled, by introducing weight factors in the loss leading to improved performance. [START_REF] Chuang | Debiased contrastive learning[END_REF] considers the same setting as Saunshi et al. [2019] and addresses the bias problem that comes from collisions between positive and negative sampling in the unsupervised constrastive loss. They propose to simulate unbiased negative sampling by assuming, among other things, extra access to positive sampling. However, one has to keep in mind that excessive access to positive sampling gets the setting closer to that of supervised learning.

In a direction that is closer to the result proposed in Section 3.4 below, Wen [2020] provides a theoretical guarantee on the training convergence of gradient descent for an overparametrized model that is trained with an unsupervised contrastive loss, using earlier works by Allen-Zhu et al. [2019]. However, two separate encoders are considered instead of a single one: one for the query, which corresponds to a sample from the dataset, and one for the (positive and negative) samples to compare the query to. In this setting, it is rather unclear how the two resulting encoders are to be used for downstream classification. In Section 3.4 below, we explain how the results from Allen-Zhu et al. [2019] can be used for the more realistic setting of a single encoder, by introducing a reasonable assumption on the encoder outputs.

Unsupervised training improves supervised performance

In this section, we provide new results in the setting previously considered in Saunshi et al. [2019]. We assume that data are distributed according to a finite set C of latent classes, and denote N C = card(C) its cardinality. Let ρ be a discrete distribution over C that is such that c∈C ρ(c) = 1 and ρ(c) > 0 for all c ∈ C. We denote D c a distribution over the feature space X from a class c ∈ C. In order to perform unsupervised contrastive training, on the one hand we assume that we can sample positive pairs (x, x + ) from the distribution

D sim (x, x + ) = c∈C ρ(c)D c (x)D c (x + ), (3.1)
namely, (x, x + ) is sampled as a mixture of independent pairs conditionally to a shared latent class, sampled according to ρ. On the other hand, we assume that we can sample negative samples x -from the distribution In particular, we denote as C, whenever there is no ambiguity, the N C -way classification task where the labels are sampled from ρ, namely D C (x, c) = ρ(c)D c (x).

D neg (x -) = c∈C ρ(c)D c (x -). (3.2) Given k N C -1, a (k + 1)

Supervised loss, mean classifier

For an encoder function f : X → R d , we define a supervised loss (cross-entropy with the best possible linear classifier on top of the representation) over task T as

L sup (f, T ) = inf W ∈R |T |×d E (x,c)∼D T -log exp (W f (x)) c c ∈T exp (W f (x)) c . (3.3)
Then, it is natural to consider the mean or discriminant classifier with weights W µ which stacks, for c ∈ T , the vectors

W µ c,: = E x∼Dc [f (x)] (3.4)
and whose corresponding (supervised) loss is given by

L µ sup (f, T ) = E (x,c)∼D T -log exp (W µ f (x)) c c ∈T exp (W µ f (x)) c . (3.5)
Note that, obviously, one has L sup (f, T ) L µ sup (f, T ).

Unsupervised contrastive loss

We consider the unsupervised contrastive loss with N negative samples given by

L N un (f ) = E (x,x + )∼D sim X -∼D ⊗N neg   -log   exp f (x) f (x + ) exp (f (x) f (x + )) + x -∈X -exp (f (x) f (x -))     ,
(3.6) where D sim is given by Equation (3.1) and where D ⊗N neg stands for the N tensor product of the D neg distribution given by Equation (3.2). When a single negative sample is used (N = 1), we will use the notation L un (f ) = L 1 un (f ). In the rest of the chapter, N will stand for the number of negatives used in the unsupervised loss (3.6).

Inequalities for unsupervised training with multiple classes

The following Lemma states that the unsupervised objective with a single negative sample can be related to the supervised loss for which the target task is classification over the whole set of latent classes C. Lemma 3.1. For any encoder f : X → R d , one has

L sup (f, C) L µ sup (f, C) 1 p ρ min L un (f ) + log N C , (3.7)
where p ρ min = min c ρ(c). The proof of Lemma 3.1 is given in the appendix, and uses a trick from Lemma 4.3 in Saunshi et al. [2019] relying on Jensen's inequality. This Lemma relates the unsupervised and the supervised losses, a shortcoming being the introduction of p ρ min , which is small for a large N C since obviously p ρ min 1/N C . The analysis becomes more difficult with a larger number of negative samples. Indeed, in this case, one needs to carefully keep track of how many distinct classes will be represented by each draw. This is handled by Theorem B.1 of Saunshi et al.

[2019], but the bound given therein only estimates an expectation of the supervised loss w.r.t. the random subset of classes considered (so called tasks). For multiple negative samples, the approach adopted in the proof of Lemma 3.1 above further degrades, since p ρ min would be replaced by the minimum probability among tuple draws, an even much smaller quantity.

We propose the following Lemma, which assumes that the number of negative samples is large enough compared to the number of latent classes.

Lemma 3.2. Consider the unsupervised objective with N negative samples as defined in Equation (3.6) and assume that

N satisfies N = Ω(N C log N C ). Then, we have L sup (f, C) L µ sup (f, C) 1 p ρ cc (N ) L N un (f ), (3.8)
where p ρ cc (N ) is the probability to have all coupons after N draws in an N C -coupon collector problem with draws from ρ.

The proof of Lemma 3.2 is given in the appendix. In this result, p ρ cc (N ) is related to the following coupon collector problem. Assume that ρ is the uniform distribution over C and let T be the random number of necessary draws until each c ∈ C is drawn at least once. It is known (see for instance [START_REF] Motwani | Randomized Algorithms[END_REF]) that the expectation and variance of T are respectively given by N C H N C and (N C π) 2 /6, where H n is the n-th harmonic number H n = n i=1 1/i. This entails using Chebyshev's inequality that

P (|T -N C H N C | βN C ) π 2 6β 2
for any β > 0, so that whenever ρ is sufficiently close to a uniform distribution and N = Ω(N C log N C ), the probability p ρ cc is reasonably high. Due to the randomness of the classes sampled during training, it is difficult to obtain a better inequality than Lemma 3.2 if we want to upper bound L N un (f ) by the supervised L sup (f, C) on all classes. However, the result can be improved by considering the average loss over tasks L sup,k (f ), as explained in the next Section.

Guarantees on the average supervised loss

In this Section, we bound the average of the supervised classification loss on tasks that are subsets of C. Towards this end, we need to assume (only in this Section) that ρ is uniform. We consider supervised tasks consisting in distinguishing one latent class from k other classes, given that they are distinct and uniformly sampled from C. We define the average supervised loss of f for (k + 1)-way classification as

L sup,k (f ) = E T ∼D k+1 [L sup (f, T )] ,
(3.9)

where D k+1 is the uniform distribution over (k + 1)-way tasks, which means uniform sampling of {c 1 , • • • , c k+1 } distinct classes in C. We define also the average supervised loss of the mean classifier

L µ sup,k (f ) = E T ∼D k+1 L µ sup (f, T ) , ( 3
: X → R d satisfies L sup,k (f ) L µ sup,k (f ) k 1 -τ + N L N un (f ) -τ + N log(N + 1) with τ + N = P c i = c, ∀i | (c, c 1 , • • • , c N ) ∼ ρ ⊗N +1
. The proof of Proposition 3.3 is given in the appendix. This Proposition states that, in a setting similar to that of Saunshi et al. [2019], on average, the (k + 1)-way supervised classification loss is upper-bounded by the unsupervised loss (both with N = 1 negative or N > 1 negatives), that contrastive learning algorithms actually minimize. Therefore, these results give hints for the performances of the learned representation for downstream tasks. Also, while Saunshi et al. [2019] only considers an unsupervised loss with N = k negatives along with (k + 1)-way tasks for evaluation, the quantities N and k are decoupled in Proposition 3.3. Furthermore, whenever ρ is uniform, one has

τ + N = c∈C ρ(c) N +1 = N -N
C , which decreases to 0 as N → +∞, so that a larger number of negatives N makes k/(1 -τ + N ) smaller and closer to k. This provides a step towards a better understanding of what is actually done in practice with unsupervised contrastive learning. For instance, N = 65536 negatives are used in He et al. [2020].

While we considered a generic encoder f and a generic setting in this Section, the next Section 3.4 considers a more realistic setting of an unsupervised objective with a fixed available dataset, and the study of an explicit algorithm for the training of f .

Convergence of gradient descent for contrastive unsupervised learning

This section leverages results from Allen-Zhu et al. [2019] to provide convergence guarantees for gradient-descent based minimization of the contrastive training error, where the unsupervisedly trained encoder is an overparametrized deep neural network.

Deep neural network encoder

We consider a family of encoders f defined as a deep feed-forward neural network following Allen-Zhu et al. [2019]. We quickly restate its structure here for the sake of completeness. A deep neural encoder f is parametrized by matrices A ∈ R m×dx , B ∈ R d×m and W 1 , . . . , W L ∈ R m×m for some depth L. For an input x ∈ R dx , the feed-forward output y ∈ R d is given by

g 0 = Ax, h 0 = φ(g 0 ), g l = W l h l-1 , h l = φ(g l ) for l = 1, . . . , L, y = Bh L ,
where φ is the ReLU activation function. Note that the architecture can also include residual connections and convolutions, as explained in Allen-Zhu et al. [2019]. We know from Allen-Zhu et al.

[2019] that, provided a δ-separation condition on the dataset (x i , y i ) for i = 1, . . . , n with δ > 0 and sufficient overparametrization of the model (m = Ω poly(n, L, δ -1 ) • d ), the optimisation of the least-squares error 1 2 n i=1 y i -y i 2 2 using gradient descent provably converges to an arbitrarily low value > 0, where y i = f (x i ) are the network outputs. Moreover, the convergence is linear i.e. the number of required epochs is T = O(log(1/ε)), although involving a constant of order poly(n, L, δ -1 ). Although this result does not directly apply to contrastive unsupervised learning, we explain below how it can be adapted provided a few additional assumptions.

Ideally, we would like to prove a convergence result on the unsupervised objective defined in Equation (3.6). However, we need to define an objective through an explicitly given dataset so that it falls within the scope of Allen-Zhu et al. [2019]. Regarding this issue, we assume in what follows that we dispose of a set of fixed triplets (x, x + , x -) ∈ (R dx ) 3 we train on.

Objective function

Let us denote this fixed training set {(x

i , x + i , x - i )} n i=1 . Each element leads to an output z i = (f (x i ), f (x + i ), f (x - i )
) by the encoder and we optimize the empirical objective

L un (f ) = n i=1 ζ f (x i ) T (f (x - i ) -f (x + i )) = n i=1 (z i ), (3.11)
where we introduced the loss function

(z i ) = (z i,1 , z i,2 , z i,3 ) = ζ(z T i,1 (z i,3 -z i,2
)) with ζ(x) = log (1 + e x ). Note that L un (f )/n is the empirical counterpart of the unsupervised loss (3.6). Our management of the set of training triplets can be compared to that of [START_REF] Wen | Convergence of end-to-end training in deep unsupervised contrasitive learning[END_REF] who similarly fixes them in advance but uses multiple negatives and the same x i as a positive. However, two distinct encoders are trained therein, one for the reference sample x i and another for the rest. We consider here the more realistic case where a single encoder is trained. Our approach also applies to multiple negatives, but we only use a single one here for simplicity. We need the following data separation assumption from Allen-Zhu et al. [2019].

Assumption 3.1. We assume that all the samples x ∈ X data = n i=1 {x i , x + i , x - i } are normalized x = 1 and that there exists δ > 0 such that x -x 2 δ for any x, x ∈ X data .

Note that sampling the positives and negatives x + i , x - i need not to be made through simple draws from the dataset. A common practice in contrastive learning [Chen et al., 2020a] is to use data augmentations, where we replace x ± i by ψ(x ± i ) for an augmentation function ψ also drawn at random. Such an augmentation can include, whenever inputs are color images, Gaussian noise, cropping, resizing, color distortion, rotation or a combination thereof, with parameters sampled at random in prescribed intervals. The setting considered here allows the case where x ± i are actually augmentations (we won't write ψ(x ± i ) but simply x ± i to simplify notations), provided that Assumption 3.1 is satisfied and that such augmentations are performed and fixed before training. Note that, in practice, the augmentations are themselves randomly sampled at each training iteration [Chen et al., 2020a]. Unfortunately, this would make the objective intractable and the convergence result we are about to derive does not apply in that case.

In order to apply the convergence result from Allen-Zhu et al.

[2019], we need to prove that the following gradient-Lipschitz condition

(z + z ) (z) + ∇ (z), z + L smooth 2 z 2 (3.12)
holds for any z, z ∈ R 3d , for some constant L smooth > 0, where is the loss given by (3.11). However, as defined previously, does not satisfy (3.12) without extra assumptions. We propose to bypass this problem by making the following additional assumption on the norms of the outputs of the encoder.

Assumption 3.2. For each element x ∈ X data , the output z = f (x) ∈ R d satisfies η < z < C
during and at the end of the training of the encoder f , for some constants 0 < η < C < +∞.

In Section 2.3, we check experimentally on three datasets (see Figure 3.3 herein) that this assumption is rather realistic. The lower bound η > 0 is necessary and used in Lemma 3.6 below, while the upper bound C is used in the next Lemma 3.4, which establishes the gradient-Lipschitz smoothness of the unsupervised loss and provides an estimation of L smooth . Lemma 3.4. Consider the unsupervised loss given by (3.11), grant Assumption 3.2 and define the set

B 3 = z = (z 1 , z 2 , z 3 ) ∈ (R d ) 3 : max j=1,2,3 z j 2 2 C 2
where C > 0 is defined in Assumption 3.2. Then, the restriction of to B 3 satisfies (3.12) with a constant L smooth 2 + 8C 2 .

The proof of Lemma 3.4 is given in the appendix. Now, we can state the main result of this Section. Theorem 3.5. Grant both Assumptions 3.1 and 3.2, let > 0 and let L un (f ) be the loss given by (3.11). Then, assuming that

m Ω poly(n, L, δ -1 ) • d ,
the gradient descent algorithm with a learning rate ν and a number of steps T such that

ν = Θ dδ poly(n, L) • m and T = O poly(n, L) δ 2 2 ,
finds a parametrization of the encoder f satisfying

L un (f ) .
The proof of Theorem 3.5 is given in the appendix. Although it uses Theorem 6 from Allen-Zhu et al. [2019], it is actually not an immediate consequence of it. Indeed, in our case, the Theorem 6 therein only allows us to conclude that ∇ L un (f )

, where the gradient is taken w.r.t. the outputs of f . The convergence of the objective itself is obtained thanks to the following Lemma whose proof is given in the appendix. Lemma 3.6. Grant Assumption 3.2 and assume that the parameters of the encoder f are optimized so that ∇ L un (f ) with < η/2, where η is defined in Assumption 3.2. Then, for any i = 1, . . . , n, we have (z i ) 2 /η where

z i = (f (x i ), f (x + i ), f (x - i )
). This Lemma is crucial for proving Theorem 3.5 as it allows to show, in this setting, that the reached critical point is in fact a global minimum.

A natural idea would be then to combine Theorem 3.5 with Proposition 3.3 in order to prove that gradient descent training of the encoder using the unsupervised contrastive loss helps to minimize the supervised loss. This paper makes a step towards such a result, but let us stress that it requires much more work, to be considered in future papers, the technical problems to be addressed being as follows. Firstly, the result of Theorem 3.5 applies to L un (f ) and cannot be directly extrapolated on L un (f ). Doing so would require a sharp control of the generalization error, while Theorem 3.5 is about the training error only. Secondly, Assumption 3.1 requires that all samples are separated and, in particular, distinct. This cannot hold when the objective is defined through an expectation as we did in Section 3.3. Indeed, it would be invalidated simply by reusing a sample in two different triples.

Experiments

In this section, we report experiments that illustrate our theoretical findings.

Datasets and Experiments

We use a small convolutional network as encoder on MNIST [LeCun and Cortes, 2010] and FashionMNIST [START_REF] Xiao | Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms[END_REF], and VGG-16 [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF]] on CIFAR-10 [ [START_REF] Krizhevsky | Learning multiple layers of features from tiny images[END_REF]. Experiments are performed with PyTorch [Paszke et al., 2019].

Results

Figure 3.1 provides an illustration of Lemma 3.1, where we display the values of L un (i.e., L N un with N = 1) and L µ sup (f, C) along training iterations over 5 separate runs (and their average). We observe that Inequality (3.7) is satisfied on these experiments, even when the log N C term is discarded. Moreover, both losses follow a similar trend. Figure 3.2 illustrates Lemma 3.2 for several values of N . Once again, we observe that both losses behave similarly, and that Inequality (3.8) seems to hold even without the 1/p ρ cc term (removed for these displays).

Conclusion

This work provides extensions to previous results on contrastive unsupervised learning, in order to somewhat improve the theoretical understanding of the performance that is empirically observed with pre-trained encoders used for subsequent supervised task. The main hindrance to tighter bounds in Section 3.3 is the blind randomness of negative sampling, which is unavoidable in the unsupervised setting. Section 3.4 explains how recent theoretical results about gradient descent training of overparametrized deep neural networks can be used for unsupervised contrastive learning, and concludes with an explanation of why combining the results from Sections 3.3 and 3.4 requires many extra technicalities to be considered in future works.

Let us conclude by stressing, once again, our motivations for doing this: unsupervised learning theory is much less developed than supervised learning theory, and recent empirical results (see Section 2.1) indicate that some forms of contrastive learning enable the learning of powerful representations without supervision. In many fields of application, labels are too difficult, too expensive or too invasive to obtain (in medical applications, see for instance [START_REF] Ching | Opportunities and obstacles for deep learning in biology and medicine[END_REF]). We believe that a better understanding of unsupervised learning is therefore of utmost importance.

Appendix: technical proofs

Proofs for Section 3.3

Apart from the similarity between the unsupervised and supervised loss, the proof of Lemma 3.1 uses properties of log-sum-exp.

Proof of Lemma 3.1. We first rewrite the unsupervised loss as:

L un (f ) = E (x,x + )∼D sim ,x -∼Dneg log 1 + exp f (x) T (f (x -) -f (x + ))
where we recognize the ζ function ζ(x) = log(1 + e x ). We start by using Jensen's inequality

L un (f ) = E (x,x + )∼D sim x -∼Dneg ζ f (x) T (f (x -) -f (x + )) E c,c -∼ρ,x∼Dc ζ f (x) T (µ c --µ c ) p ρ min E c∼ρ,x∼Dc max c - ζ f (x) T (µ c --µ c ) = p ρ min E c∼ρ,x∼Dc max c - LSE 0, f (x) T (µ c --µ c ) p ρ min E c∼ρ,x∼Dc LSE f (x) T (µ c 1 -µ c ), . . . , f (x) T (µ c N C -µ c ) -log N C = p ρ min L µ sup (f, C) -log N C
where we have used properties of the log-sum-exp function

max(x 1 , . . . , x n ) LSE(x 1 , . . . , x n ) max(x 1 , . . . , x n ) + log n,
the fact that LSE is non-negative whenever one of its arguments is, and for x ∈ R 2n we have

LSE(x) = LSE(LSE(x 1 , x 2 ), . . . , LSE(x 2n-1 , x 2n )) max j=1,...,n LSE(x 2j-1 , x 2j ) + log n.
The proof of Lemma 3.2 considers the sample draws where all classes are represented.

Proof of Lemma 3.2. Let I ∈ [N C ] N the random vector of classes for each negative sample (I ∼ ρ ⊗N ) and let J be the set of represented classes i.e. J = {I j | j ∈ [N ]}. We have, again with Jensen's inequality

L N un (f ) = E x,x + ,x - 1 ,...,x - N LSE 0, f (x) T (f (x - 1 ) -f (x + )), . . . , f (x) T (f (x - N ) -f (x + )) E c∼ρ,I∼ρ ⊗N ,x∼Dc LSE 0, f (x) T (µ I 1 -µ c ), . . . , f (x) T (µ I N -µ c ) P (|J| = N C ) E c∼ρ I∼ρ ⊗N x∼Dc LSE 0, f (x) T (µ I 1 -µ c ), . . . , f (x) T (µ I N -µ c ) | |J| = N C P (|J| = N C ) L µ sup (f, C),
where we used that for S ⊂ [n] and x ∈ R n we have LSE(x S ) LSE(x) with x S the restriction of x to the indices in S. Finally, we have P (|J| = N C ) = p ρ cc (N ). We restate Proposition 3.3 for cases N = 1 and N > 1. The proof uses Jensen's inequality and the uniformity of ρ.

Proposition 3.3 (restated). Consider the unsupervised loss L N

un (f ) from Equation (3.6) with N negative samples. Assume that ρ is uniform over C and that

2 k + 1 N C . Then, (1) any encoder function f : X → R d satisfies L sup,k (f ) L µ sup,k (f ) k 1 -τ + L un (f ) -τ +
with τ + = P c,c ∼ρ 2 (c = c ), where L un (f ) is the unsupervised loss from Equation (3.6) with N = 1 negative sample;

(2) more generally,

L sup,k (f ) L µ sup,k (f ) k 1 -τ + N L N un (f ) -τ + N log(N + 1) with τ + N = P(c i = c, ∀i | c ∼ ρ, (c 1 , • • • , c N ) ∼ ρ N )
, and where L N un (f ) is the unsupervised loss from Equation (3.6).

Proof of Proposition 3.3. Let's start with (1). By Jensen's inequality, then use log = log 2 without loss of generality, and split the expectation into cases c -= c and c -= c,

L un (f ) = E (c,c -)∼ρ 2 E x,x + ∼Dc,x -∼D c -log 1 + exp f (x) T f (x -) -f (x + ) E (c,c -)∼ρ 2 ,x∼Dc log 1 + exp f (x) T (µ c --µ c ) = (1 -τ + )E c∼ρ,x∼Dc E c -∼ρ log 1 + exp f (x) T (µ c --µ c ) c -= c + τ + .
Let us write explicitly the uniform distribution ρ on C. On the one hand,

E c -∼ρ log 1 + exp f (x) T (µ c --µ c ) c -= c = c -∈C\{c} 1 N C -1 log 1 + exp f (x) T (µ c --µ c ) ,
on the other hand, And this is for every c -∈ C\{c}. We rearrange the double sum according to c - Hence, using the uniformity of ρ,

E c -∼ρ log 1 + exp f (x) T (µ c --µ c ) c -= c = 1 k E c 1 ,...,c k ∼ρ ⊗k k i=1 log 1 + exp f (x) T (µ c i -µ c ) {c, c 1 , . . . , c k } distinct 1 k E c 1 ,...,c k ∼ρ ⊗k log 1 + k i=1 exp f (x) T (µ c i -µ c ) {c, c 1 , . . . , c k } distinct .
That means we have

L un (f ) 1 -τ + k E c∼ρ,x∼Dc c 1 ,...,c k ∼ρ ⊗k log 1 + k i=1 exp f (x) T (µ c i -µ c ) {c, c 1 , . . . , c k } distinct + τ + = 1 -τ + k E T ∼D k+1 E (x,c)∼D T   -log    exp(f (x) T µ c ) exp(f (x) T µ c ) + c -∈T c -=c exp (f (x) T µ c -)       + τ + = 1 -τ + k L µ sup,k (f ) + τ + .
As for (2), again by Jensen's inequality, and split the expectation into cases

c - i = c, ∀i and ∃c - i = c, L N un (f ) = E (c,c - i )∼ρ N +1 E x,x + ∼Dc,x - i ∼D c - i log 1 + N i=1 exp f (x) T f (x - i ) -f (x + ) E (c,c - i )∼ρ N +1 ,x∼Dc log 1 + N i=1 exp f (x) T µ c - i -µ c = (1 -τ + N )E c∼ρ x∼Dc E c - i ∼ρ N log 1 + N i=1 exp f (x) T (µ c --µ c ) ∃c - i = c + τ + N log(N + 1) with τ + N = P(c i = c, ∀i | c ∼ ρ, c i ∼ ρ N ) = c∈C ρ(c) N +1 = N -N C .
Considering the fact that

E c - i ∼ρ N log 1 + N i=1 exp f (x) T (µ c --µ c ) ∃c - i = c E c -∼ρ log 1 + exp f (x) T (µ c --µ c ) c -= c ,
then by similar computations as in (1), we have

L N un (f ) 1 -τ + N k L µ sup,k (f ) + τ + N log(N + 1).

Proofs for Section 3.4

Let us first prove that under Assumption 3.2, the objective is gradient-Lipschitz w.r.t. the network outputs.

Lemma 4.1. Consider the unsupervised loss given by (3.11), grant Assumption 3.2 and define the set

B 3 = z = (z 1 , z 2 , z 3 ) ∈ (R d ) 3 : max j=1,2,3 z j 2 2 C 2
where C > 0 is defined in Assumption 3.2. Then, the restriction of to B 3 satisfies (3.12) with a constant L smooth 2 + 8C 2 .

Proof. We will prove this result by bounding the norm of the Hessian matrix. Let us write the gradient of (z) with respect to z first. We have z ∈ R 3d . For ease of writing, we define the matrices A 1 , A 2 , A 3 ∈ R 3d×d as

A 1 =    I d 0 d 0 d    A 2 =    0 d I d 0 d    A 3 =    0 d 0 d I d   
where I d , 0 d ∈ R d×d are the identity and zero matrix respectively. With this notation, we have z i = A T i z for i = 1, 2, 3 the three contiguous thirds of z's coordinates.

Our purpose is to compute

∂ ∂z (z) = ∂ ∂z   -log   exp z T 1 z 2 exp z T 1 z 2 + exp z T 1 z 3     .
Denote cos i,j = z T i z j , we can now compute for i, j ∈ {1, 2, 3} (i = j)

∂ ∂z cos i,j = A i A T j + A j A T i z =: ∂ cos i,j ∈ R 3d . Now, denote v = softmax (cos 1,2 , cos 1,3 ) ∈ R 2 , we can write ∂ ∂z (z) = (v 1 -1)∂ cos 1,2 +v 2 ∂ cos 1,3 .
We proceed with the following computation

∂ 2 ∂z 2 cos i,j = A i A T j + A j A T i ,
which we will denote simply as ∂ 2 cos i,j . Before we get the Hessian of loss, we still need to compute

∂v := ∂v ∂z = (diag(v) -vv T ) ∂ cos T 1,2 ∂ cos T 1,3 ∈ R 2×3d .

Now we can write

∂ 2 ∂z 2 (z) = (v 1 -1)∂ 2 cos 1,2 +v 2 ∂ 2 cos 1,3 + ∂ cos 1,2 ∂ cos 1,3 ∂v.
We can now estimate the norm of this matrix which will provide an estimation for the Lipschitz constant.

We find that ∂ cos i,j 2 max ( z i , z j ) , keeping in mind that the matrix diag(v) -vv T has norm at most 1/2, this leads to

∂ cos 1,2 ∂ cos 1,3 ∂v = 8 max i,j ( z i z j ) .
We have also that ∂ 2 cos i,j = 1. All in all, we have ∂ 2 ∂z 2 (z) = 2 + 8 max i,j ( z i z j ). Recalling that we restricted R 3d so that we have max i z i C the result follows.

Theorem 3.5 is actually obtained in two steps. First, Theorem 6 from Allen-Zhu et al. [2019] allows us to obtain that the gradient of the objective ∇ L un (f ) with respect to the network outputs reaches arbitrarily low values. Then, combining this with Assumption 3.2, this result can be extended into the objective itself.

Following appendix A of Allen-Zhu et al.

[2019], we need to define the vectors for our model. These are originally defined as i = y i -y * i (y i and y * i are respectively the output and label corresponding to an input x i from the dataset) for the 2 loss. More generally, for a network output z i , they are defined as Following the unsupervised training protocol, samples are fed into the network three at a time x, x + and x -. Let us denote θ the parameters of the network f , for a triplet (x i , x + i , x - i ), the trick is to write:

∂ ∂θ (z i ) = ∂z ∂θ ∂ ∂z (z i ) with z i the concatenation of f (x i ), f (x + i ), f (x - i ). By denoting (x 1 , x 2 , x 3 ) = (x i , x + i , x - i )
, the previous writing is equivalent to

3 j=1 ∂f (x j ) ∂θ A T j ∂ ∂z (z i )
and by letting i,j = A T j ∂ ∂z (z i ), we obtain a triplet of loss vectors for each data triple (matrices A j defined in the previous proof).

Lemma 3.7. Grant Assumption 3.1 and let L un (f ) be the loss incurred by f :

L un (f ) = n i=1 (f (x i ), f (x + i ), f (x - i ))
and let > 0 be the desired precision. Then, assuming m Ω poly(n, L, δ -1 ) • d -1 , the gradient descent with learning rate ν = Θ

dδ poly(n,L)•m finds parameters such that ∇ L un (f )
after a number of steps T = O poly(n,L)

δ 2 2 .
Proof. This result follows from Allen-Zhu et al. [2019] (see Theorem 6 and appendix A). It corresponds to the case of a non-convex bounded loss function. We only need to check the used loss function is bounded and gradient-Lipschitz smooth. The latter condition is verified due to Lemma 4.1 and Assumption 3.2. As for the boundedness, it is also a consequence of Assumption 3.2 and the fact that the softplus function satisfies

ζ(x) ∼ x→+∞ x and lim x→-∞ ζ(x) = 0.
From here, we can derive a result for the objective itself (Theorem 3.5) thanks to the following Lemma. Lemma 4.2. Grant Assumption 3.2 and assume that the parameters of the encoder f are optimized so that ∇ L un (f ) with < η/2, where η is defined in Assumption 3.2. Then, for any i = 1, . . . , n, we have

(z i ) 2 /η where z i = (f (x i ), f (x + i ), f (x - i )). Proof. Since we assume ∇ L un (f )
, this also implies that max i,j i,j (see Theorem 3 of Allen-Zhu et al. [2019] and its variant in appendix A).

We can write the norms i,j as:

i,1 = (v 1 -1)z i,2 + v 2 z i,3 i,2 = |v 1 -1| z i,1 i,3 = v 2 z i,1
where we defined v = softmax(z T 1 z 2 , z T 1 z 3 ). Thanks to Assumption 3.2, we can argue that z i,j η. These quantities can be small for v 1 → 1 and v 2 → 0. Since we have max i,j i,j , this implies in particular that for all i we get i,3 which means v 2 /η, and we have v 2 = σ(z T 1 (z 3 -z 2 )). So for an instance i ∈ [n] the loss term in the objective is:

ζ(z T i,1 (z i,3 -z i,2 )) = log(1 + exp(z T i,1 (z i,3 -z i,2 ))) = -log(σ(-z T i,1 (z i,3 -z i,2 ))) = -log(1 -σ(z T i,1 (z i,3 -z i,2 ))) = -log(1 -v 2 ) v 2 1 -v 2 2v 2 2 /η,
where we used the inequalitylog(1 -x)

x 1-x for 0 x < 1, and the assumption that < η/2. Lemma 4.2 allows us to deduce that the objective is well optimized (we treated the loss term for a single triplet here but the same methods can be applied to the whole objective with a number of gradient steps which is still polynomial).

Proof of Theorem 3.5. Theorem 3.5 is the consequence of combining Lemma 3.7 applied using η 2n instead of and Lemma 4.2 (the 1/n factor can be absorbed by the poly(n, L) factors in the bounds of Lemma 3.7). .2 -Mean test AUC and standard-deviations (y-axis) using 10 train/test splits for WW and scikit-learn's implementations of RF [START_REF] Louppe | Understanding random forests: From theory to practice[END_REF] and Extra Trees [Geurts et al., 2006], using default hyperparameters, on several datasets. Thanks to subtrees aggregation, WW improves these baselines, even with few trees (x-axis is the number of trees).

Secondly, WW uses feature binning ("histogram" strategy), similarly to what is done in extreme gradient boosting (EGB) libraries such as XGBoost [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF], LightGBM [START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF] and CatBoost [START_REF] Dorogush | Catboost: gradient boosting with categorical features support[END_REF]Prokhorenkova et al., 2017]. This strategy helps to accelerate computations in WW compared with standard RF algorithms, that typically require to sort features locally in nodes and try a larger number of splits [START_REF] Louppe | Understanding random forests: From theory to practice[END_REF]. This combination of subtrees aggregation and of the histogram strategy makes WW comparable with state-of-the-art implementations of EGB libraries, as illustrated in Figure 4.3. Moreover, WW supports optimal split finding for categorical features and missing values, with no need for particular pre-processing (such as one-hot encoding [Chen and Guestrin, 2016] or target encoding [START_REF] Dorogush | Catboost: gradient boosting with categorical features support[END_REF]Prokhorenkova et al., 2017]). Finally, WW is supported by some theoretical evidence, since we prove that for a general loss function, the subtrees aggregation considered in WW leads indeed to a performance close to that of the best subtree.

Related works

Since their introduction [START_REF] Breiman | Random forests[END_REF], RF algorithms have become one of the most popular supervised learning algorithm thanks to their ease of use, robustness to hyperparameters [START_REF] Biau | A random forest guided tour[END_REF] A recent empirical study Zhou and Mentch [2021] suggests that tree depth limitation in RF is an effective regularization mechanism which improves performance on low signal-to-noise ratio datasets. In [START_REF] Mourtada | AMF: Aggregated Mondrian forests for online learning[END_REF], an improvement of Mondrian Forests [Lakshminarayanan et al., 2014] is introduced for online learning, using subtrees aggregation with exponential weights, which is particularly convenient in the online learning setting. However, this paper considers only the online setting, with purely random trees (splits are not optimized using training data), leading to poor performances compared with realistic decision trees. In WW, we use a similar subtrees aggregation mechanism for batch learning differently: we make the most of the bootstrap, one of the key ingredients of RF, which provides in-the-bag and out-of-bag samples, to perform aggregation with exponential weights, together with efficient decision trees grown using the histogram strategy.

Extreme boosting algorithms are another type of ensemble methods. XG-Boost [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF] provides an extremely popular scalable tree boosting system which has been widely adopted in industry. LightGBM [START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF] introduced the "histogram strategy" for faster split finding, together with clever downsampling and features grouping algorithms in order to achieve high performance in reduced computation times. CatBoost [START_REF] Probst | Hyperparameters and tuning strategies for random forest[END_REF] is another boosting library which pays particular attention to categorical features using target encoding, while addressing the potential bias issues associated to such an encoding.

Limitations.

Our implementation of WW is still evolving and is not yet at the level of maturity of state-of-the-art EGB libraries such as [Chen and Guestrin, 2016; [START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF][START_REF] Probst | Hyperparameters and tuning strategies for random forest[END_REF]. It does not outperform such strong baselines, but proposes an improvement of RF algorithms, and gives an interesting balance between performance and computational efficiency.

WildWood: a new Random Forest algorithm

We consider batch supervised learning, where data comes as a set of i.i.d training samples (x i , y i ) for i = 1, . . . , n with vectors of numerical or categorical features x i ∈ X ⊂ R d and y i ∈ Y. Our aim is to design a RF predictor g(

• ; Π) = 1 M M m=1 f ( • ; Π m ) : X → Y computed from training
samples, where Y is the prediction space. Such a RF computes the average of M randomized trees predictions f ( • ; Π m ) following the principle of bagging [START_REF] Breiman | Bagging predictors[END_REF][START_REF] Oza | Online bagging and boosting[END_REF], with Π = (Π 1 , . . . , Π M ) where Π 1 , . . . , Π M are i.i.d realizations of a random variable corresponding to bootstrap and feature subsampling (see Section 4.2.1 below). Each tree is trained independently of each other, in parallel. In what follows we describe only the construction of a single tree and omit from now on the dependence on m = 1, . . . , M .

Feature binning

The split finding strategy described in Section 4.2.1 below works on binned features. While this technique is of common practice in EGB libraries [Chen and Guestrin, 2016; Ke et al., 2017; Prokhorenkova et al., 2017], we are not aware of an implementation of it for RF. The input n × d matrix X of features is transformed into another same-size matrix of "binned" features denoted X bin . To each input feature j = 1, . . . , d is associated a set B j = {1, . . . , b j } of bins, where b j b max with b max a hyperparameter corresponding to the maximum number of bins a feature can use (default is b max = 256 similarly to [START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF], so that a single byte can be used for entries of X bin ). When a feature is continuous, it is binned into b max bins using inter-quantile intervals. If it is categorical, each modality is mapped to a bin whenever b max is larger than its number of modalities, otherwise sparsest modalities end up binned together. If a feature j contains missing values, its rightmost bin in B j is used to encode them. After binning, each column satisfies X bin •,j ∈ B n j .

Random trees

Let C = d j=1 B j be the binned feature space. A random tree is a pair (T , Σ), where T is a finite ordered binary tree and Σ contains information about each node in T , such as split information. The tree is random and its source of randomness Π comes from the bootstrap and feature subsampling as explained below.

Finite ordered binary trees

A finite ordered binary tree T is represented as a finite subset of the set {0, 1} * = n 0 {0, 1} n of all finite words on {0, 1}. The set {0, 1} * is endowed with a tree structure (and called the complete binary tree): the empty word root is the root, and for any v ∈ {0, 1} * , the left (resp. right) child of v is v0 (resp. v1). We denote by intnodes(T ) = {v ∈ T : v0, v1 ∈ T } the set of its interior nodes and by leaves(T ) = {v ∈ T : v0, v1 ∈ T } the set of its leaves, both sets are disjoint and the set of all nodes is nodes(T ) = intnodes(T ) ∪ leaves(T ).

Splits and cells

The split σ v = (j v , t v ) ∈ Σ of each v ∈ intnodes(T ) is characterized by its dimension j v ∈ {1, . . . , d} and a subset of bins t v {1, . . . , b jv }. We associate to each v ∈ T a cell C v ⊆ C which is defined recursively: C root = C and for each v ∈ intnodes(T ) we define

C v0 := {x ∈ C v : x jv ∈ t v } and C v1 := C v \ C v0 .
When j v corresponds to a continuous feature, bins have a natural order and t v = {1, 2, . . . , s v } for some bin threshold s v ∈ B jv ; while for a categorical split, the whole set t v is required. By construction, (C v ) v∈leaves(T ) is a partition of C.

Bootstrap and feature subsampling

Let I = {1, . . . , n} be the training samples indices. The randomization Π of the tree uses bootstrap: it samples uniformly at random, with replacement, elements of I corresponding to in-the-bag (itb) samples. If we denote as I itb the indices of unique itb samples, we can define the indices of out-of-bag (oob) samples as

I oob = I \I itb . A standard argument shows that P[i ∈ I itb ] = 1 -(1 -1/n) n → 1 -e -1
≈ 0.632 as n → +∞, known as the 0.632 rule [START_REF] Efron | Improvements on cross-validation: the 632+ bootstrap method[END_REF]]. The randomization Π uses also feature subsampling: each time we need to find a split, we do not try all the features {1, . . . , d} but only a subset of them of size d max , chosen uniformly at random. This follows what standard RF algorithms do [START_REF] Biau | A random forest guided tour[END_REF][START_REF] Breiman | Random forests[END_REF][START_REF] Louppe | Understanding random forests: From theory to practice[END_REF], with the default

d max = √ d.

Split finding on histograms

For K-class classification, when looking for a split for some node v, we compute the node's "histogram" hist v [j, b, k] = i∈Iitb:x i ∈Cv 1x i,j = b, y i = k for each sampled feature j, each bin b and label class k seen in the node's samples (actually weighted counts to handle bootstrapping and sample weights). Of course, one has hist v = hist v0 + hist v1 , so that we don't need to compute two histograms for siblings v0 and v1, but only a single one. Then, we loop over the set of non-constant (in the node) sampled features {j : #{b : k hist v [j, b, k] 1} 2} and over the set of non-empty bins {b : k hist v [j, b, k] 1} to find a split, by comparing standard impurity criteria computed on the histogram's statistics, such as gini or entropy for classification and variance for regression.

Bin order and categorical features

The order of the bins used in the loop depends on the type of the feature. If it is continuous, we use the natural order of bins. If it is categorical and the task is binary classification (labels in {0, 1}), we use the bin order that sorts hist v [j, b, 1]/ k=0,1 hist v [j, b, k] with respect to b, namely the proportion of labels 1 in each bin. This allows to find the optimal split with complexity O(b j log b j ), see Theorem 9.6 in [START_REF] Breiman | Classification and regression trees[END_REF], the logarithm coming from the sorting operation, while there are 2 b j -1 -1 possible splits. This trick is used by EGB libraries as well, using an order of gradient/hessian statistics of the loss considered [START_REF] Chen | Xgboost: A scalable tree boosting system[END_REF][START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF]Prokhorenkova et al., 2017].

For K-class classification with K > 2, we consider two strategies: (1) one-versusrest, where we train M K trees instead of M , each tree trained with a binary oneversus-rest label, so that trees can find optimal categorical splits and (2) heuristic, where we train M trees and where split finding uses K loops over bin orders that sort hist v [j, b, k]/ k hist v [j, b, k ] (w.r.t b) for k = 0, . . . , K-1. If a feature contains missing values, we do not loop only left to right (along bin order), but right to left as well, in order to compare splits that put missing values on the left or on the right.

Split requirements

Nodes must hold at least one itb and one oob sample to apply aggregation with exponential weights, see Section 4.2.2 below. A split is discarded if it leads to children with less than n min-leaf itb or oob samples and we do not split a node with less than n min-split itb or oob samples. These hyperparameters only weakly impact WW's performances and sticking to default values (n min-leaf = 1 and n min-split = 2, following scikit-learn [START_REF] Louppe | Understanding random forests: From theory to practice[END_REF]Pedregosa et al., 2011]) is usually enough (see Appendix 4.7.4 from the supplementary material).

Related works on categorical splits

In [START_REF] Coppersmith | Partitioning nominal attributes in decision trees[END_REF], an interesting characterization of an optimal categorical split for multiclass classification is introduced, but no efficient algorithm is, to the best of our understanding, available for it. A heuristic algorithm is proposed therein, but it requires to computing, for each split, the top principal component of the covariance matrix of the conditional distribution of labels given bins, which is computationally too demanding for an RF algorithm intended for large datasets. Regularized target encoding is shown in [START_REF] Pargent | Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features[END_REF] to perform best when compared with many alternative categorical encoding methods. Catboost [Prokhorenkova et al., 2017] uses target encoding, which replaces feature modalities by label statistics, so that a natural bin order can be used for split finding. To avoid overfitting on uninformative categorical features, a debiasing technique uses random permutations of samples and computes the target statistic of each element based only on its predecessors in the permutation. However, for multiclass classification, target encoding is influenced by the arbitrarily chosen ordinal encoding of the labels. LightGBM [START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF]] uses a one-versus-rest strategy, which is also one of the approaches used in WW for categorical splits on multiclass tasks. For categorical splits, where bin order depends on labels statistics, WW does not use debiasing as in Prokhorenkova et al. [2017], since aggregation with exponential weights computed on oob samples allows to deal with overfitting.

Tree growth stopping. We do not split a node and make it a leaf if it contains less than n min-split itb or oob samples. The same applies when a node's impurity is not larger than a threshold ε (ε = 0 by default). When only leaves or non-splittable nodes remain, the growth of the tree is stopped. Trees grow in a depth-first fashion so that childs v0 and v1 have memory indexes larger than their parent v (as required by Algorithm 3 below).

Prediction function: aggregation with exponential weights

Given a tree T grown as described in Sections 4.2.1 and 4.2.1, its prediction function is an aggregation of the predictions given by all possible subtrees rooted at root, denoted {T : T ⊂ T }. While T is grown using itb samples, we use oob samples to perform aggregation with exponential weights, with a branching process prior over subtrees, that gives more importance to subtrees with a good predictive oob performance.

Node and subtree prediction

We define v T (x) ∈ leaves(T ) as the leaf of T containing x ∈ C. The prediction of a node v ∈ nodes(T ) and of a subtree T ⊂ T is given by

y v = h((y i ) i∈Iitb : x i ∈Cv ) and y T (x) = y v T (x) , ( 4.1)
where h : ∪ n 0 Y n → Y is a generic "forecaster" used in each cell and where a subtree prediction is the one of its leaf containing x.

A standard choice for regression (Y = Y = R) is the empirical mean forecaster

y v = 1 n v i∈Iitb : x i ∈Cv y i , (4.2)
where

n v = |{i ∈ I itb : x i ∈ C v }|.
For K-class classification with Y = {1, . . . , K} and Y = P(Y), the set of probability distributions over Y, a standard choice is a Bayes predictive posterior with a prior on P(Y) equal to the Dirichlet distribution Dir(α, . . . , α), namely the Jeffreys prior on the multinomial model P(Y), which leads to

y v (k) = n v (k) + α n v + αK , (4.3) for any k ∈ Y, where n v (k) = |{i ∈ I itb : x i ∈ C v , y i = k}|.
By default, WW uses α = 1/2 (the Krichevsky-Trofimov forecaster [START_REF] Tjalkens | Sequential weighting algorithms for multi-alphabet sources[END_REF]), but one can perfectly use any α > 0, so that all the coordinates of y v are positive. This is motivated by the fact that WW uses as default the log loss to assess oob performance for classification, which requires an arbitrarily chosen clipping value for zero probabilities. Different choices of α only weakly impact WW's performance, as illustrated in Appendix 4.7.4. We use oob samples to define the cumulative losses of the predictions of all

T ⊂ T L T = i∈Ioob ( y T (x i ), y i ), (4.4)
where : Y × Y → R + is a loss function. For regression problems, a default choice is the quadratic loss ( y, y) = ( y -y) 2 while for multiclass classification, a default is the log-loss ( y, y) = -log y(y), where y(y) ∈ (0, 1] when using (4.3), but other loss choices are of course possible.

Prediction function

Let x ∈ C. The prediction function f of a tree T in WW is given by

f (x) = T ⊂T π(T )e -ηL T y T (x) T ⊂T π(T )e -ηL T with π(T ) = 2 -T , (4.5)
where the sum is over all subtrees T of T rooted at root, where η > 0 is temperature parameter and T is the number of nodes in T minus its number of leaves that are also leaves of T . Note that π is the distribution of the branching process with branching probability 1/2 at each node of T , with exactly two children when it branches. A default choice is η = 1 for the log-loss (see in particular Corollary 4.3 in Section 4.3 below), but it can also be tuned through hyperoptimization, although we do not observe strong performance gains, see Appendix 4.7.4 from the supplementary material. The prediction function (4.5) is an aggregation of the predictions y T (x) of all subtrees T , weighted by their performance on oob samples and with prior π(T ) = 2 -T . This aggregation procedure can be understood as a non-greedy way to prune trees: the weights depend not only on the quality of one single split but also on the performance of each subsequent split. Computing f from Equation (4.5) is computationally and memory-wise infeasible for a large T , since it involves a sum over all T ⊂ T and requires one weight for each T . Indeed, the number of subtrees of a minimal tree that separates n points is exponential in the number of nodes, and hence exponential in n. However, it turns out that one can compute exactly and very efficiently f thanks to the prior choice π together with an adaptation of context tree weighting [START_REF] Catoni | Statistical Learning Theory and Stochastic Optimization: Ecole d'Eté[END_REF][START_REF] Helmbold | Predicting nearly as well as the best pruning of a decision tree[END_REF][START_REF] Willems | The context-tree weighting method: Extensions[END_REF][START_REF] Willems | The context-tree weighting method: Basic properties[END_REF]. Theorem 4.1. The prediction function (4.5) can be written as f (x) = f root (x), where f root (x) satisfies the recursion

f v (x) = 1 2 w v w den v y v + 1 - 1 2 w v w den v f va (x) (4.6)
for v, va ∈ path(x) (a ∈ {0, 1}) the path in T going from root to v T (x), where w v := exp(-ηL v ) with L v := i∈Ioob:x i ∈Cv ( y v , y i ) and where w den v are weights satisfying the recursion

w den v = w v if v ∈ leaves(T ), 1 2 w v + 1 2 w den v0 w den v1 otherwise. ( 4.7)
The proof of Theorem 4.1 is given in Appendix 4.6.1 of the supplementary material, a consequence of this Theorem being a very efficient computation of f (x) is described in Algorithms 3 and 4 below. Algorithm 3 computes the weights w den v using the fact that trees in WW are grown in a depth-first fashion, so that we can loop once, leading to a O(|nodes(T )|) complexity in time and in memory usage, over nodes from a data structure that respects the parenthood order. Direct computations can lead to numerical over-or under-flows (many products of exponentially small or large numbers are involved), so Algorithm 3 works recursively over the logarithms of the weights (line 6 uses a log-sum-exp function that can be made overflow-proof). Algorithm 3 is applied once T is fully grown, so that WW is ready Algorithm 3 Computation of log(w den v ) for all v ∈ nodes(T ).

1: Inputs: T , η > 0 and losses L v for all v ∈ nodes(T ). Nodes from nodes(T ) are stored in a data structure nodes that respects parenthood order: for any

v = nodes[i v ] ∈ intnodes(T ) and children va = nodes[i va ] for a ∈ {0, 1}, we have i va > i v . 2: for v ∈ reversed(nodes) do 3: if v is a leaf
Put v ← parent(v) 6: Put α ← 1 2 exp(-ηL v -log(w den v )) 7: Put f (x) ← α y v + (1 -α) f (x) 8: end while 9: return The prediction f (x)
which is the complexity required to find the leaf v T (x) containing x ∈ C: Algorithm 4 only increases by a factor 2 the prediction complexity of a standard RF (in order to go down to v T (x) and up again to root along path(x)). More details about the construction of Algorithms 3 and 4 can be found in Appendix 4.6.1 of the supplementary material.

Theoretical guarantees

This section proposes some theoretical guarantees on the subtrees aggregation used in WW, see (4.5). We say that a loss function is η-exp-concave for some η > 0 whenever z → exp(-η (z, y)) is concave for any y ∈ Y. We consider a fully-grown tree T computed using itb samples and the set of oob samples (x i , y i ) i∈Ioob on which L T is computed using (4.4), and we denote n oob := |I oob |. Theorem 4.2 (Oracle inequality). Assume that the loss function is η-exp-concave. Then, the prediction function f given by (4.5) satisfies the oracle inequality

1 n oob i∈Ioob ( f (x i ), y i ) inf T ⊂T 1 n oob i∈Ioob ( y T (x i ), y i ) + log 2 η T n oob + 1 ,
where the infimum is over any subtree T ⊂ T and where we recall that T is the number of nodes in T minus its number of leaves that are also leaves of T . Theorem 4.2 proves that for a general loss function, the prediction function of WW is able to perform nearly as well as the best oracle subtree T ⊂ T on oob samples, with an O( T /n oob ) rate which is optimal for model-selection oracle inequalities [Tsybakov, 2003] ( T = O(log N T ) with a number of "experts" N T = |{T : T ⊂ T }| for a well-balanced T ). Let us stress again that, while finding an oracle argmin T ⊂T i∈Ioob ( y T (x i ), y i ) is computationally infeasible, since it requires trying out all possible subtrees, WW's prediction function (4.5) comes at a cost comparable to that of a standard Random Forest, as explained in Section 4.2.2 above.

The proof of Theorem 4.2 is given in Section 4.6.2 of the supplementary material, and relies on techniques from PAC-Baysesian theory [START_REF] Catoni | PAC-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning[END_REF][START_REF] Mcallester | Some pac-bayesian theorems[END_REF][START_REF] Mcallester | Pac-bayesian model averaging[END_REF]. Compared with [START_REF] Mourtada | AMF: Aggregated Mondrian forests for online learning[END_REF] about online learning, our proof differs significantly: we do not use results specialized to online learning such as [START_REF] Vovk | A game of prediction with expert advice[END_REF] nor online-to-batch conversion Cesa-Bianchi et al. [2004]. Note that Theorem 4.2 does not address the generalization error, since it would require to study the generalization error of the random forest itself (and of the fully grown tree T ), which is a topic way beyond the scope of this paper, and still a very difficult open problem: recent results [START_REF] Arlot | Analysis of purely random forests bias[END_REF][START_REF] Genuer | Variance reduction in purely random forests[END_REF][START_REF] Mourtada | Minimax optimal rates for mondrian trees and forests[END_REF]Scornet, 2016a,b;[START_REF] Scornet | Consistency of random forests[END_REF] only study stylized versions of RF (called purely random forests).

Consequences of Theorem 4.2 are Corollary 4.3 for the log-loss (classification) and Corollary 4.4 for the least-squares loss (regression). Corollary 4.3 (Classification). Consider K-class classification (Y = {1, . . . , K}) and consider the prediction function f given by (4.5), where node predictions are given by (4.3) with α = 1/2 (WW's default), where is the log-loss and where η = 1. Then, we have

1 n oob i∈Ioob ( f (x i ), y i ) inf T ⊂T 1 n oob i∈Ioob (g T (x i ), y i ) + K + 4 log 2 -1 4 T + 1 n oob ,
where g T is any constant function on the leaves of T . Corollary 4.4 (Regression). Consider regression with Y = [-B, B] for some B > 0 and the prediction function f given by (4.5), where node predictions are given by (4.2), where is the least-squares loss and where η = 1/(8B 2 ). Then, we have

1 n oob i∈Ioob ( f (x i ), y i ) inf T ⊂T 1 n oob i∈Ioob (g T (x i ), y i ) + 8(log 2)B 2 T n oob ,
where g T is any function constant on the leaves of T . The proofs of Corollaries 4.3 and 4.4 are given in Section 4.6.2 of the supplementary material. These corollaries motivate the default hyperparameter values of η, in particular η = 1 for classification.

Experiments

Our implementation of WildWood is available at the GitHub repository https: //github.com/pyensemble/wildwood. It is open-sourced under the BSD 3-Clause license on GitHub. It is a Python package that follows scikit-learn's API conventions, that is JIT-compiled to machine code using numba [START_REF] Lam | Numba: A llvm-based python jit compiler[END_REF]. Trees in the forest are grown in parallel using joblib [Joblib Development Team, 2020] and CPU threads, GPU training will be supported in future updates.

We compare WildWood (denoted WWn for n trees) with several strong baselines including RFn: scikit-learn's implementation of Random Forest [START_REF] Louppe | Understanding random forests: From theory to practice[END_REF]Pedregosa et al., 2011] using n trees; HGB: an histogram-based implementation of extreme gradient boosting (inspired by LightGBM) from scikit-learn; and several state-of-the-art and widely adopted extreme gradient boosting libraries including XGB: XGBoost [Chen and Guestrin, 2016]; LGBM: LightGBM [START_REF] Ke | LightGBM: A highly efficient gradient boosting decision tree[END_REF] and CB: CatBoost [START_REF] Dorogush | Catboost: gradient boosting with categorical features support[END_REF]Prokhorenkova et al., 2017]. We used a 32-cores server with two Intel Xeon Gold CPUs, two Tesla V100 GPUs and 384GB RAM for the experiments involving hyperoptimization (Table 4.1) and used a 12-cores Intel i7 MacBook Pro with 32GB RAM and no GPU to obtain training times achievable by a "standard user" (Table 4.2). All experiments can be reproduced using Python scripts on the repository.

Description of the experiments

We use publicly available and open-source datasets from the UCI repository [START_REF] Dua | UCI machine learning repository[END_REF], including small datasets (hundreds of rows) and large datasets (millions of rows), their main characteristics are given in Table 4.5 together with URLs in Table 4.6, see the supplementary material. Each dataset is randomly split into a training set (70%) and a test set (30%). We specify which features are categorical to algorithms that natively support it (HGB, LGBM, CB and WWn) and simply integer-encode them, while we use one-hot encoding for other algorithms (RFn, XGB).

For each algorithm and dataset, hyperoptimization is performed as follows: from the training set, we use 4/5 for training and 1/5 for validation and do 50 steps of sequential optimization using the Tree Parzen Estimator implemented in the hyperopt library [START_REF] Bergstra | Hyperopt: a python library for model selection and hyperparameter optimization[END_REF]. More details about hyperoptimization are provided in Appendix 4.7.2 of the supplementary material. Then, we refit on the whole training set with the best hyperparameters and report scores on the test set. This is performed 5 times in order to report standard deviations.

We use the area under the ROC curve (AUC), for K-class datasets with K > 2 we average the AUC of each class versus the rest. This leads to the test AUC scores displayed in Table 4.1 (the same scores with standard deviations are available in Table 4.3 of the supplementary material). We report also in Table 4.2 (see Table 4.4 from supplementary material for standard deviations) the test AUC scores obtained with default hyperparameters of all algorithms on the 5 largest considered datasets together with their training times (timings can vary by several orders of magnitude with varying hyperparameters for EGB libraries, as observed by the timing differences between Figure 4.3 and Table 4.2).

Discussion of experiments results

We observe in Table 4.1 that EGB algorithms, when hyperoptimized, lead to the best performances over the considered datasets compared with RF algorithms, and we observe that WW always improves the performance of RF, at the exception of few datasets for which the performance is identical. When using default hyperparameters for all algorithms, we observe in Table 4.2 that the test AUC scores can decrease significantly for EGB libraries while RF algorithms seem more stable, and that there is no clear best performing algorithm in this case.

The results on both tables show that WW is competitive with respect to all baselines both in terms of performance and computational times: it manages to always reach at least comparable performance with the best algorithms despite only using 10 trees as a default. In this respect, WW maintains high scores at a lower computational cost.

Conclusion

We introduced WildWood, a new Random Forest algorithm for batch supervised learning. Tree predictions in WildWood are aggregation with exponential weights of the predictions of all subtrees, with weights computed on bootstrap out-of-thebag samples. This leads to improved predictions in each individual tree, at a small computational cost, since WildWood's prediction complexity is similar to that of a standard Random Forest. Moreover, thanks to the histogram strategy, Wild-Wood's implementation is competitive with strong baselines including popular extreme boosting libraries, both in terms of performance and training times. Note also that WildWood has few hyperparameters to tune and that the performances obtained with default hyperparameters are usually good enough in our experiments. WildWood's implementation is still evolving and many improvements coming with future updates are planned, including the computation of feature importance, GPU training, distributed training (we only support single-machine training for now), among other enhancements that will further improve performances and accelerate computations. Room for improvement in WildWood comes from the fact that the overall forest prediction is a simple arithmetic mean of each tree prediction, while we could perform also exponentially weighted aggregation between trees. Future works include a WildWood-based implementation of isolation-forest [START_REF] Liu | Isolation forest[END_REF], using the same subtrees aggregation mechanism with the log loss for density estimation, to propose a new algorithm for outliers detection. The expression in Equation (4.5) involves sums over all subtrees T of the fully grown tree T (involving an exponential in the number of leaves of T ). However, it can be computed efficiently because of the specific choice of the prior π. More precisely, we will use the following lemma [START_REF] Helmbold | Predicting nearly as well as the best pruning of a decision tree[END_REF], Lemma 1] several times to efficiently compute sums of products. Let us recall that nodes(T ) stands for the set of nodes of T . Lemma 4.5. Let g : nodes(T ) → R be an arbitrary function and define G :

nodes(T ) → R as G(v) = T ⊂Tv 2 -T v ∈leaves(T ) g(v ), (4.8)
where the sum over T ⊂ T v means the sum over all subtrees T of T rooted at v. Then, G(v) can be computed recursively as follows:

G(v) = g(v) if v ∈ leaves(T ) 1 2 g(v) + 1 2 G(v0)G(v1) otherwise,
for each node v ∈ nodes(T ).

For the sake of completeness, we include a proof of this statement.

Proof. First, let us notice that the case v ∈ leaves(T ) is straightforward since there is only one pruning T of T v which satisfies T = 0 (recall that T is the number of internal nodes and leaves in T minus the number of leaves in T that are also leaves of T v ). For the second case, we can expand G(v) by taking into account the pruning which only leaves v as a leaf, the rest of the prunings can be expressed through pairs of prunings T 0 and T 1 of T v0 and T v1 respectively. Moreover, it can be shown that such a pruning T satisfies T = 1 + T 0 + T 1 , thus we get the following expansion

G(v) = 1 2 g(v) + T 0 ⊂T v0 T 1 ⊂T v1 2 -(1+ T 0 + T 1 ) v ∈T 0 g(v0v ) v ∈T 1 g(v1v ) = 1 2 g(v) + 1 2 T 0 ⊂T v0 2 -T 0 v ∈T 0 g(v0v ) • T 1 ⊂T v1 2 -T 1 v ∈T 1 g(v1v ) = 1 2 g(v) + 1 2 G(v0)G(v1).
This concludes the proof of Lemma 4.5.

Let us introduce w T = π(T ) exp(-ηL T ) for any T ⊂ T , so that Equation (4.5) writes

f (x) = T ⊂T w T y T (x) T ⊂T w T , ( 4.9)
where the sums hold over all the subtrees T of T rooted at root (the root of the full tree T ). We will show how to efficiently compute and update the numerator and denominator in Equation (4.9). Note that w T may be written as

w T = π(T ) exp(-ηL T ) = 2 -T exp -η i∈Ioob ( y v T (x i ) , y i ) = 2 -T exp -η v∈leaves(T ) i∈Ioob : x i ∈Cv ( y v T (x i ) , y i ) (4.10) = 2 -T exp -η v∈leaves(T ) i∈Ioob : x i ∈Cv ( y v , y i ) (4.11) = 2 -T exp -η v∈leaves(T ) L v = 2 -T v∈leaves(T )
w v , (4.12)

where we recall that

L v = i∈Ioob : x i ∈Cv ( y v , y i ) and w v = exp(-ηL v ).
Equality (4.10) comes from the fact that the set of cells {C v : v ∈ leaves(T )} is a partition of C by construction, and that the stopping criterion used to build T ensures that each leaf node in leaves(T ) contains at least one sample from I oob (see Section 4.2.1). Equality (4.11) comes from the fact that the prediction of a node is constant and equal to y v for any x ∈ C v .

Denominator of Equation (4.9). For each node v ∈ nodes(T ), denote

w den v = T ⊂Tv 2 -T v ∈leaves(T ) w v , (4.13)
where once again the sum over T ⊂ T v means the sum over all subtrees T of T rooted at v. We have that (4.12) entails

w den root = T ⊂Troot 2 -T v∈leaves(T ) w v = T ⊂Troot w T = T ⊂T w T . (4.14)
So, we can compute recursively w den root very efficiently, using a recursion on the weights w den v using Lemma 4.5 with g(v) = w v . This leads to the recursion stated in Theorem 4.1, see Equation (4.7). Now, we can exploit the fact that decision trees are built in a depth-first fashion in WildWood: all the nodes v ∈ T are stored in a "flat" array, and by construction both the child nodes v0 and v1 have indexes that are larger than the one of v. So, we can simply loop over the array of nodes in reverse order, and compute w den v = w v if v ∈ leaves(T ) and w den v = 1 2 w v + 1 2 w den v0 w den v1 otherwise: we are guaranteed to have computed w den v0 and w den v1 before computing w den v . This algorithm is described in Algorithm 3. Since these computations involve a large number of products with exponentiated numbers, it typically leads to strong over-and Secondly, we go up in the tree along path(x) and use again (4. ). Moreover, we observe that the recursions used in (4.7) and (4. 19) only need to save both w v and w den v for any v ∈ nodes(T ). Finally, we have using (4.14) and (4.17) that

f (x) = T ⊂T w T y T (x) T ⊂T w T = w num root (x) w den root =: f root (x),
and we want to compute f root (x) recursively from f v (x) where v ∈ path(x). First, whenever v = v T (x) we have

f v (x) = w num v (x) w den v = w v y v w v = y v ,
while for v = v T (x) and v ∈ path(x), we write

f v (x) = w num v (x) w den v = 1 2 w v y v + 1 2 w den v(1-a) w num va (x) w den v (4.20) = 1 2 w v w den v y v + 1 2 w den v(1-a) w den va w den v w num va (x) w den va (4.21) = 1 2 w v w den v y v + 1 - 1 2 w v w den v f va (x), (4.22)
where (4.20) comes from (4.19) while (4.22) comes from (4.7). This proves the recursion stated in Equation (4.6) from Theorem 4.1, and to Algorithm 4. This concludes the proof of Theorem 4.1.

Proofs of the results from Section 4.3

The proof of Theorem 4.2 is partly inspired from the proof of Theorem 2 in Dalalyan and Tsybakov [2008], that we generalize to exp-concave losses, while only leastsquares regression is considered therein. Let E be a measurable space and P, Q be probability measures on it. The Kullback-Leibler divergence between P and Q is defined by

KL(P, Q) = E log dP dQ dP
whenever P is absolutely continuous with respect to Q and equal to +∞ otherwise. Also, if h : E → R is a measurable function such that E hdP is well-defined on R ∪ {-∞, +∞}, we introduce

P h := h E hdP • P,
the probability measure on E with density h/ hdP with respect to P . A classical result is the Donsker-Varadhan variational formula [START_REF] Donsker | Asymptotic evaluation of certain markov process expectations for large time-iii[END_REF], which is at the core of the proofs of many PAC-Bayesian theorems [START_REF] Catoni | PAC-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning[END_REF][START_REF] Mcallester | Pac-bayesian model averaging[END_REF] where the supremum is over all probability measures on E, and where the supremum is achieved for P = Q exp(h) whenever the term on the left-hand side is finite.

Proof of Theorem 4.2. Recall that the oob loss of a subtree T ⊂ T is given by

L T = i∈Ioob ( y T (x i ), y i )
and let us introduce

p T = π(T ) exp(-ηL T ) T π(T ) exp(-ηL T ) (4.24)
for any subtree T ⊂ T . First, we use the fact that is a η-exp-concave loss function, hence η-mixable, see Section 3.3 from Cesa-Bianchi and Lugosi [2006], which entails, since p T is a probability measure over the set of all subtrees T ⊂ T , that

T p T y T (x i ), y i - 1 η log T p T exp(-η ( y T (x i ), y i )) ,
where the sums over T are over all subtrees T ⊂ T . Now, summing this inequality over i ∈ I oob and using the convexity of the log-sum-exp function leads to

i∈Ioob T p T y T (x i ), y i - 1 η i∈Ioob log T p T exp(-η ( y T (x i ), y i )) - n oob η log T p T exp - η n oob i∈Ioob ( y T (x i ), y i ) = - n oob η log T p T exp - η n oob L T .
By plugging the definition of p T into the previous expression, and by introducing ρ(T ) := ηL T /n oob , we obtain

S n oob := 1 n oob i∈Ioob ( f (x i ), y i ) - 1 η log T π(T ) exp -(n oob + 1)ρ(T ) + 1 η log T π(T ) exp -n oob ρ(T ) .
The Hölder inequality implies that for any probability measure P over the set of subtrees of T . This leads to

1 n oob i∈Ioob ( f (x i ), y i ) 1 n oob T P (T )L T + 1 η(n oob + 1) KL(P, π)
for any P , since KL(P, π exp(h) ) 0. So for the particular choice P = δ T (the Dirac mass at T ) for any subtree T ⊂ T , we have

1 n oob i∈Ioob ( f (x i ), y i ) 1 n oob L T + 1 η(n oob + 1) log π(T ) -1 1 n oob L T + log 2 η T n oob + 1 ,
which concludes the proof of Theorem 4.2.

The proof of Corollary 4.3 requires the next Lemma.

y(k) = n v (k) + 1/2 n v + K/2
where n v = K k=1 n v (k) and the log loss (y , y) = -log y (y). Then, the following inequality holds

i ( y, y i ) -inf p i (p, y i ) K -1 2 .
Proof. We know that the optimal p is given by p k = n v (k)/n v . Indeed, it is the solution to the following constrained convex optimization problem

min p - K k=1 n v (k) log p k subject to K k=1 p k = 1,
where we consider non-negativity to be already enforced by the objective and imposing p k 1 is redundant with the constraint K k=1 p k = 1. We can write the Lagrangian function as

L(p, λ) = - K k=1 n v (k) log p k + λ K k=1 p k -1
and one can check the KKT conditions when taking p k = n v (k)/n v and λ = n v . Since we are dealing with a convex problem with linear constraints, this is a sufficient 4.1 of all algorithms after hyperoptimization on the considered datasets. Standard-deviations displayed between parentheses are computed from 5 trainings with different random seeds. We observe that WW has better (or identical in some cases) performances than RF on all datasets and that it is close to that of EGB libraries (bold is for best EGB performance, underline for best RFn or WWn performance). • alpha: 0 with probability 0.5, and log-uniform distribution [e -16 , e 2 ] with probability 0.5;

• lambda: 0 with probability 0.5, and log-uniform distribution [e -16 , e 2 ] with probability 0.5;

• gamma: 0 with probability 0.5, and log-uniform distribution [e -16 , e 2 ] with probability 0.5;

LightGBM

• learning_rate: log-uniform distribution [e -7 , 1];

• num_leaves: discrete log-uniform distribution [1, e 7 ];

• feature_fraction: uniform [0.5, 1];

• bagging_fraction: uniform [0.5, 1];

• min_data_in_leaf: discrete log-uniform distribution [1, e 6 ];

• min_sum_hessian_in_leaf: log-uniform distribution [e -16 , e 5 ];

• lambda_l1: 0 with probability 0.5, and log-uniform distribution [e -16 , e 2 ] with probability 0.5;

• lambda_l2: 0 with probability 0.5, and log-uniform distribution [e -16 , e 2 ] with probability 0.5;

CatBoost

• learning_rate: log-uniform distribution [e -7 , 1];

• random_strength: discrete uniform distribution over {1, 20};

• one_hot_max_size: discrete uniform distribution over {0, 25};

• l2_leaf_reg: log-uniform distribution [1, 10];

• bagging_temperature: uniform [0, 1].

HistGradientBoosting

• learning_rate: log-uniform distribution [e -4 , 1];

• max_leaf_nodes: discrete log-uniform distribution [1, e 7 ];

• min_samples_leaf: discrete log-uniform distribution [1, e 6 ];

• l2_regularization: 0 with probability 0.5, and log-uniform distribution [e -16 , e 2 ] with probability 0.5;

RandomForest

• max_features: None with probability 0.5, and sqrt with probability 0.5;

• min_samples_leaf: uniform over {1, 5, 10} and we set min_samples_split = 2 × min_samples_leaf;

WildWood

• multiclass: multinomial with probability 0.5, and ovr with probability 0.5;

• min_samples_leaf: uniform over {1, 5, 10} and we set min_samples_split = 2 × min_samples_leaf;

• step: log-uniform distribution [e -3 , e 6 ];

• dirichlet: log-uniform distribution [e -7 , e 2 ];

• cat_split_strategy: binary with probability 0.5, and all with probability 0.5;

• max_features: None with probability 0.5, and sqrt with probability 0.5.

Datasets

The main characteristics of the datasets used in the paper are summarized in Table 4.5. We provide in Table 4.6 the URL of all the datasets used, most of them are from the UCI machine learning repository [START_REF] Dua | UCI machine learning repository[END_REF].

Sensitivity of hyperparameters of Wildwood

In

Supplementary details about assets used (versions and licenses)

The versions and licenses of the libraries used in our experiments are:

• catboost (0.25.1), Apache License 2.0

• hyperopt (0.2.5), license: https://github.com/hyperopt/hyperopt/blob/ master/LICENSE.txt

• joblib (0.17), BSD-3-Clause License where the space of parameters Θ ⊂ R d is called comparison class, and n ∈ N denotes the number of total rounds. Online logistic regression falls into the supervised online learning setting, where a learning algorithm A takes as input at time t all the previous observations S t-1 = ((x 1 , y 1 ), . . . , (x t-1 , y t-1 )) as well as the features x t of time t, and predicts y t = A(S t-1 , x t ). An algorithm A is proper if the predictor returned by the learning algorithm x → A(S t , x) belongs to the space of functions {x → θ x, θ ∈ Θ} for all time steps t 1; otherwise, the algorithm is improper.

• In the proper setting, the predictor A(S t-1 )(•) cannot use x t . Follow-The-Leader-type methods apply, and the problem can also be seen as instance of Online Convex Optimization (OCO).

• In the improper setting, the predictor is allowed to use the knowledge of x t : indeed, the predictor writes x t → A(S t-1 , x t )(x t ).

We can further separate algorithms into two families: Bayesian and non-Bayesian methods. In this chapter, we introduce a new improper non-Bayesian algorithm, to investigate if it is possible to achieve an optimal regret while remaining computationally efficient. We also propose a survey of recent advances on the topic of online logistic regression. Excellent introductory manuals on the general subject of online learning include Hazan [2019]; Orabona [2019] and Cesa-Bianchi and Lugosi [2006]. We formally formulate the online logistic regression problem and introduce some notations useful in the following. We consider the online problem over a series of rounds. At each round t, an example feature x t ∈ X is showed; a label y t ∈ Y will be later observed. We denote the features space X ⊂ R d , Y = {-1, +1} for binary labels and the prediction space Y = R. In the point of view of online density estimation, an algorithm A plays θ ∈ R d , such that the probability of a label for an example x is

p(y | x, θ) := 1 1 + exp(-y • θ x) = σ(y • θ x)
with σ the sigmoid function σ(u) := e u /(1 + e u ) for u ∈ R. The loss at time t for the logistic model θ is the log-loss

(θ x t , y t ) = -log p(y t | x t , θ) = -log σ(y t • θ x t ).
Equivalently, the algorithm A produces, for an example x, the prediction

y = log p(+1 | x, θ) p(-1 | x, θ) = θ x,
and suffers the logistic loss : Y × Y → R defined as ( y, y) = log (1 + exp (yy))) = -log σ( yy). The goal is to obtain guarantees on the regret for any individual sequence, including in the worst-case scenario.

Assumption 5.1 (Bounded features). The features are bounded i.e. x t 2 R with some constant R > 0, for all t. About the order of BR. The results that we present in this section are valid for arbitrary parameters B, R > 0 andd, n 1. In order to make these bounds more concrete, we now discuss some natural scaling for the norm BR, following Remark 2 of Mourtada and Gaïffas [2019]. Remark 5.1 (Parameter scaling). In the case n d, consider a random variable X ∈ R d and denote Σ = E[XX ]. Assume that Σ is well-conditioned, c := Σ op Σ -1 op = O(1); this means that X is approximately isotropic, and can be ensured in practice by rescaling covariates. Assume also the bounded leverage condition Σ -1/2 X ρ √ d for some ρ 1, and denote ψ :

= θ Σ = E[ θ, X 2 ] 1/2 as the signal strength. Then, θ X Σ -1/2 op θ Σ Σ 1/2 op Σ -1/2 X c 1/2 ψρ √ d, namely we have BR c 1/2 ψρ √ d = O( √ d).

Properties of the logistic function

The algorithms with better regret guarantees for online logistic regression often come from some more advanced studies on the logistic function. Denote f t (θ) := (θ x t , y t ) = -log σ(y t • θ x t ) for θ ∈ R d , the logistic loss function taking the parameter θ, induced by (x t , y t ). The function f t is infinitely differentiable, its first and second derivatives write

∇f t (θ) = -y t σ(-y t x t θ)x t , ∇ 2 f t (θ) = σ (-y t x t θ)x t x t .
In the following, we make some observations on the logistic function and point out the relationships between these properties and some known algorithms for online logistic regression. Denote θ t the point around which we make the approximation.

• Under Assumption 5.1, f t is R-Lipschitz, since ∇f t (θ) R for any (x t , y t ) and any θ. Under the same condition, ∇f t is R 2 -Lipschitz, i.e. f t is R 2 -smooth.

• The logistic loss function is convex, ∇ 2 f t (θ) 0 for any θ and any (x t , y t ). Hence, for any fixed θ t , its first order approximation around θ t defined as

ft : θ → f t (θ t ) + ∇f t (θ t ) (θ -θ t ) (5.1)
is a lower-bound of f t for any θ. This is the key property in the theoretical guarantees for Online Gradient Descent [Zinkevich, 2003].

• The logistic loss is exp(-BR)-exp-concave on the ball B(R d , B) under Assumptions 5.1 and 5.2. The exp-concavity can be understood as a property stating that the Hessian is large in the direction of the gradient. Indeed, αexp-concavity of a function f t is equivalent to ∇ 2 f t (θ) α∇f t (θ)∇f t (θ) for any θ; for the logistic function, this comes from σ (u) = σ(u)(1 -σ(u)) = exp(-u)σ 2 (u), hence σ (u) exp(-BR)σ 2 (u) whenever u BR. Online Newton Step [Hazan et al., 2007] relies on the following quadratic approximation of the logistic loss

ft : θ → f t (θ t ) + ∇f t (θ t ) (θ -θ t ) + β 2 (θ -θ t ) ∇f t (θ t )∇f t (θ t ) (θ -θ t ) (5.2)
with β exp(-BR) the exp-concavity parameter. Such ft is showed [Hazan et al., 2007] to be a lower-bound of f t for any θ and any θ t ∈ B(R d , B). Also relying on the exp-concavity are the Bayesian exponential weighted algorithms as exposed in Hazan et al. [2007]; [START_REF] Kakade | Online bounds for bayesian algorithms[END_REF].

• While the self-concordance [START_REF] Nesterov | Introductory lectures on convex optimization: A basic course[END_REF] is a property on the control of the third derivative by the second derivative, namely |f | 2(f ) 3/2 , the logistic function is not self-concordant. [START_REF] Bach | Self-concordant analysis for logistic regression[END_REF] introduced "generalized R-self-concordance" defined by |f | Rf , with some R > 0. For the logistic function, its generalized self-concordance comes from a more precise study on the derivatives of the sigmoid function, σ (u) = (1 -2σ(u))σ (u), hence |σ (u)| σ (u) for any u ∈ R. A consequence of the generalized R-selfconcordance (under Assumption 5.1) of the logistic function states that exp

(-R θ -θ t )∇ 2 f t (θ t ) ∇ 2 f t (θ) exp(R θ -θ t )∇ 2 f t (θ t ) ( 5
: θ → f t (θ t ) + ∇f t (θ t ) (θ -θ t ) + exp(-R θ -θ t ) + R θ -θ t -1 R 2 θ -θ t 2 (θ -θ t ) ∇ 2 f t (θ t )(θ -θ t ) (5.4)
is showed [Bach, 2010, Proposition 1] to be a lower-bound of f t (θ) for any θ and any θ t , again under Assumption 5.1. In the batch learning setting, Sample Minmax Predictor [Mourtada and Gaïffas, 2019] in the case of ridge-penalized logistic regression also relies on the property (5.3) in their analysis for the excess risk.

• • Also, the logistic function ( y, y) = -log σ( yy) is 1-mixable, in the sense that, for any probability distribution π over Y = R, there exists a prediction y π such that, for any y ∈ {-1, +1}, E ŷ∼π [exp(-η ( y, y)] exp(-η ( y π , y)) (5.6) with η = 1. To see this for the logistic function, we construct

y π = σ + (E ŷ∼π [σ( y)])
where σ + : (0, 1) → R is the inverse of the sigmoid function, σ + (p) = log p 1-p , hence σ + (σ(u)) = u for any u ∈ R. Such y π satisfies (5.6) with equality. Bayesian methods Foster et al. [2018] and Jézéquel et al. [2021] rely on the 1-mixability of the logistic function.

Proper algorithms

We propose a literature review of online logistic regression methods. While some of presented methods also apply to other problems, here we only present them as solutions to the online logistic regression, and we highlight the necessary conditions in each case. Within proper algorithms, there are Follow-The-Leader methods, gradient-based methods and Bayesian methods.

Follow-The-Leader methods

Let us start with probably the most intuitive idea: gradient-free methods. Follow-The-Leader (FTL) consists in choosing, at time t, the parameter θ that would have been the best to use on the previous t -1 rounds in hindsight, i.e. to choose

θ t = argmin θ∈R d L t-1 (θ) with L t-1 (θ) := t-1 s=1 s (θ).
However, the regret of this strategy can be linear in the number of rounds, considering for example the case where d = 1, x t = 1 and y t alternate between -1 and +1.

Follow-The-Regularized-Leader (FTRL) is introduced in order to stabilize the FTL solution, where the function to minimize is given with an additional regularization term on θ, i.e. the algorithm shall choose at time t,

θ t = argmin θ∈R d {L t-1 (θ) + pen(θ)} ,
where the penalization term pen(θ) could be in 1 , 2 or other forms. with a constant λ > 0. They showed that O( √ BR + log(n)) regret is granted with λ = 2/BR. However, it was not clear whether this result is generalizable to larger dimensions and more general features.

Another variant of FTL, Follow-The-Approximate-Leader (FTAL), consists in choosing

θ t = argmin θ∈R d Lt-1 (θ) with Lt-1 (θ) := t-1 s=1 ˜ s (θ),
where each ˜ s is an approximation function of the true loss function s . Indeed, several gradient methods (OGD, ONS) could be seen as FTAL methods [Hazan et al., 2007;Zinkevich, 2003], see details in the following paragraph. Interestingly, sub-linear or logarithmic regret upper-bounds are granted with these methods.

Gradient-based methods

Online gradient descent (OGD) for online logistic regression was first reported in Zinkevich [2003]. It starts with an arbitrary θ 1 ∈ Θ, then successively updates the predictions by

θ t = Proj Θ (θ t-1 -η t ∇ t-1 (θ t-1 ))
with η t > 0 the step-size parameter at time t, and Proj Θ the Euclidean projector operator onto the compact convex feasible set Θ. The regret analysis of OGD is based on the convexity of the logistic function, and on the fact that a convex function is lower-bounded by its tangent hyperplans, see Equation (5.1). This results in a regret upper-bound O(BR √ n) with the choice of step-sizes η t = B/(R √ t), as well as a regret lower-bound Ω(BR √ n) in the worst case [Hazan, 2019] which is sublinear in the number of rounds n. OGD can also be seen2 as choosing at time t,

θ t = argmin θ∈Θ ∇ t-1 (θ t-1 ) (θ -θ t-1 ) + 1 2η t θ -θ t-1 2 .
OGD results in O(nd) computational complexity in time for n total rounds, plus the complexity costs in the projection steps. Then, Online Newton Step (ONS) was proposed and analyzed in Hazan et al. [2007]. It starts with an arbitrary θ 1 ∈ Θ, then uses the predictions

θ t = Proj Θ,A t-1 A -1 t-1 b t-1 with A t := t s=1 ∇ s ∇ s , b t := t s=1 ∇ s ∇ s θ s - 1 β ∇ s , where ∇ t := ∇ t (θ t )
with β = 1 2 min{ 1 4BR , exp(-BR)} as in (5.2), and Proj Θ,A t-1 the projector operator onto the compact convex feasible set Θ under the norm induced by A t-1 . The regret analysis of ONS is based on the fact that the logistic loss function is R-Lipschitz and exp(-BR)-exp-concave on the bounded feasible set, implying the quadratic lower-bound (5.2) of the logistic function. The regret of ONS admits an upper-bound [Hazan et al., 2007] in O(exp(BR)d log(n)), which is logarithmic in the number of rounds n, but with an exponential factor in BR due to the expconcavity coefficient. ONS can be seen as running FTAL [Hazan et al., 2007], the approximation function being the quadratic lower-bound function (5.2) i.e. choosing at time t

θ t = argmin θ∈Θ Lt-1 (θ) with Lt-1 (θ) := t-1 s=1 ˜ s (θ), where ˜ t (θ) := t (θ t ) + ∇ t (θ -θ t ) + β 2 (θ -θ t ) ∇ t ∇ t (θ -θ t ).
At each step the computational cost is O(d 2 ) as we need to iteratively update A -1 t , thanks to the matrix inversion lemma; the total computational complexity is O(nd 2 ) in time for n total rounds, plus the complexity costs in the projection steps; and O(d 2 ) space in memory is needed.

However, under the proper setting and by constructing adversary examples, Hazan et al. [2014] showed that in the regime n = O(exp(B)), for one-dimensional inputs the regret is at least Ω(B 2/3 n 1/3 ) in the worst case [Hazan et

Improper algorithms

Within improper algorithms, there are Bayesian and non-Bayesian methods.

Bayesian improper methods

Foster et al.

[2018] used a uniform prior over Θ in Bayesian mixture with exponential weights Vovk's Aggregation Algorithm [START_REF] Vovk | Aggregating strategies[END_REF]. In the case of online binary logistic regression and without smoothing, the algorithm of Foster et al. [2018] initializes p 0 as the uniform distribution over Θ, successively predicts

y t = σ + (E θ∼p t-1 [σ(θ x t )])
where σ + : (0, 1) → R is the inverse of the sigmoid function, σ + (p) = log p 1-p , and updates the distribution p t over Θ by

p t (θ) ∝ exp - t s=1 (θ x s , y s ) .
With this procedure, the regret has an upper-bound in O(d log(BRn/d)). This work can be generalized into m-class softmax regression, with a regret in O(md log(n/md)). However, this algorithm is computationally expensive, as it relies on Monte Carlo methods and the log-concavity of the probability distributions p t : its computational complexity is in O(B 6 n 12 (Bn + d) 12 ).

Recent work of [START_REF] Shamir | Logistic regression regret: What's the catch[END_REF] investigated the achievable minimax regret rate of online logistic regression from the scope of information theory. Using the redundancy capacity Theorem, [START_REF] Shamir | Logistic regression regret: What's the catch[END_REF] Finally, a recent work Jézéquel et al. [2021] focus on the multi-class logistic regression, proposing Gaussian Aggregating Forecaster (GAF) which relies on the quadratic approximation (5.5) and on the Bayesian mixture with Gaussian distributions.

Non-Bayesian improper algorithms

As a non-Bayesian improper algorithm, AIOLI [Jézéquel et al., 2020] is based on a new quadratic lower bound (5.5) of the logistic function, with adaptive curvature which is better than exp-concavity. It also relies on a Vovk-Azoury-Warmuthinspired [START_REF] Azoury | Relative loss bounds for on-line density estimation with the exponential family of distributions[END_REF][START_REF] Vovk | Competitive on-line statistics[END_REF] and SMP-inspired [Mourtada and Gaïffas, 2019] regularization, which is a regularization by samples. AIOLI computes the following estimator at time t,

θ t = argmin θ∈R d Lt-1 (θ) + (θ x t , +1) + (θ x t , -1) + λ θ 2 with Lt-1 (θ) := t-1 s=1 ˜ s (θ)
, where the surrogate loss function is defined by

˜ t (θ) := t (θ t ) + ∇ t (θ t ) (θ -θ t ) + exp(y t θ t x t ) 2(1 + BR) (θ -θ t ) ∇ t (θ t )∇ t ( θ t ) (θ -θ t )
as in (5.5). The regret of AIOLI is upper-bounded by O(dBR log(BRn)) with a computational cost in O(nd 2 + nBR log(n)). Recent work FOLKLORE [Agarwal et al., 2021] generalizes the quadratic lower bound (5.5) to the multi-class case and proposes an improper algorithm with similar regret and computational cost compared to AIOLI. However, the regret upper-bound of AIOLI might be seen as suboptimal for the following reasons: 1) from Remark 5.1, BR scales as O( √ d), then regret in O(d √ d) of AIOLI is not optimal; 2) consider the online-to-batch conversion on AIOLI, the algorithm's expected bound of excess risk is in O(dBR log(BRn)); apart from the factor log n, unavoidable in the online setting, this bound is of a factor BR worse than the expected excess risk of SMP [Mourtada and Gaïffas, 2019].

Sample Minmax Predictor (SMP) [Mourtada and Gaïffas, 2019] is an algorithm for the batch setting. In particular, for binary logistic regression with (X i , Y i ) i=1,...,n i.i.d. data and λ > 0 the penalization parameter, the ridge-penalized SMP computes the estimator θ z λ,n = argmin

θ∈R d 1 n + 1 n i=1 ( θ, Z i ) + ( θ, z ) + λ θ 2
where we denote Z i = -Y i X i , and z = (x, y) a virtual sample. Then SMP predicts the conditional probability, for any (x, y) ∈ R d × {-1, +1}, by The expected excess risk of ridge-penalized SMP for logistic regression is in O((d + B 2 R 2 )/n). But it was not clear whether this procedure could be applied in the online setting and how.

An open problem.

As far as we know, there is no algorithm with an optimal regret guarantee for the problem of online logistic regression. Bayesian algorithms [Foster et al., 2018;Jézéquel et al., 2021] enjoy better regret upper-bounds but suffer from computational complexity; non-Bayesian algorithms [START_REF] Bibliography | Efficient methods for online multiclass logistic regression[END_REF]Hazan et al., 2007;Jézéquel et al., 2020], less computationally demanding, fail to provide similar regret upper-bounds. We believe that there is a room between them, and that an online version of SMP [Mourtada and Gaïffas, 2019] or SMPinspired methods, which would be some non-Bayesian improper algorithms, could achieve this goal. In this direction, we investigate some non-Bayesian improper algorithms with realistic computational complexity, and their regret upper-bounds, even though we do not improve the state-of-the-art of this problem.

We summarize the regret upper-bounds and the computational costs of these algorithms in Table 5.1 below.

Table 5.1 -Regret upper-bounds and computational costs in O(•) of several algorithms for online binary logistic regression: OGD [Zinkevich, 2003], ONS [Hazan et al., 2007], Foster [Foster et al., 2018], GAF [Jézéquel et al., 2021], AIOLI [Jézéquel et al., 2020

OSMP: One-Step Minmax Predictor

In this section, we explore a SMP-inspired [Mourtada and Gaïffas, 2019] predictor for online logistic regression, and propose a study on the corresponding regret.

Algorithm

Under the setting of online binary logistic regression, at time step t, we have seen (x 1 , y 1 ), ..., (x t-1 , y t-1 ), predicted y 1 , ..., y t-1 ; now we are provided with x t , and we need to propose our prediction y t . Consider the "λ-ridge penalized regret" at time t with some λ > 0, ( y s , y s ), L t-1 (θ) := t-1 s=1 (θ x s , y s ), where L t-1 stands for actual sum of losses suffered by successively proposed predictions ( y s ) until time t -1; L t-1 is the function taking parameter θ, corresponding to the sum of losses until time t -1 suffered by a fixed parameter θ.

OSMP can be seen as a minimax strategy and a very pessimistic one: it always considers the worst possible scenario, in terms of the label y t and the adversary parameter θ ∈ ∆, measured by the λ-ridge penalized regret at time t; from this, OSMP aims to find the best possible (one-step) predictor of time t against the worst scenario.

In Equation (5.8), we may search θ exactly in the same comparison class ∆ = Θ; using a larger search space ∆ ⊇ Θ is also possible, meaning that our minimax strategy is even more pessimistic. Remark 5.3. OSMP defined in Equation (5.8) is equivalent to predict probabilities p(+1 | x t ) = p t and p(-1 | x t ) = 1 -p t with p t = argmin p∈(0,1) sup yt∈{-1,1} sup θ∈∆ L t-1 -log p -L t-1 (θ) + (θ x t , y t ) + λ θ 2

(5.9) using p t = σ( y t ).

Notations

Let us introduce some useful notations. Denote L λ,t (θ) := L t (θ) + λ θ 2 , L λ,t := inf θ∈∆ L λ,t (θ), (5.10) and L y λ,t (θ) := (θ x t , y) + L λ,t-1 (θ) for y ∈ {-1, +1}, and their infimum L y λ,t := inf θ∈∆ (θ x t , y) + L λ,t-1 (θ) for y ∈ {-1, +1}.

We also introduce the notations We remark that θ t as well as θ (x,y) t exist and are unique, as the minimizers of strongly convex functions over the convex set ∆.

Alternative expression and explicit algorithm

Relying on the specificities of the log-loss, we deduce the following alternative expression of OSMP, which allows the explicit algorithm computing OSMP. Lemma 5.4. OSMP as defined in Equation (5.8) is equivalent to -y) .

y t = -L +1 λ,t + L -
Recall that σ(-y) = 1 -σ( y) for the sigmoid function, hence sup θ∈∆ {σ(θ x t ) exp (-L λ,t-1 (θ))} + sup θ∈∆ {σ(-θ x t ) exp (-L λ,t-1 (θ))} and hence the expression of y t as in the statement of the Lemma. Now we are ready to summarize the computation of OSMP in Algorithm 5. Remark 5.5. OSMP predicts probabilities p(+1 | x t ) = p t and p(-1 | x t ) = 1 -p t with p t = exp(-L +1 λ,t ) exp(-L +1 λ,t ) + exp(-L -1 λ,t )

y t = argmin
(5.12) for y ∈ {-1, +1}, by remarking that p t = σ( y t ).

Algorithm 5 OSMP description.

1: Inputs: Parameters λ > 0, n, d 1, constants B, R > 0, search space ∆ 2: Initialize: Function L λ,0 (θ) = λ θ 2 3: for t = 1, . . . , n do 4:

Receive x t 5:

Compute L y λ,t ← inf θ∈∆ (θ x t , y) + L λ,t-1 (θ) for y = -1 and y = +1 6:

Predict y t ← -L +1 λ,t + L -1 λ,t 7:

Receive y t 8:

Update function L λ,t (θ) = L λ,t-1 (θ) + (θ x, y t ) 9: end for Equation (5.12) is analogous to Equation (47) (which comes from its Theorem 2) of Mourtada and Gaïffas [2019] for the Sample Minmax Predictor under the batch setting. In place of the penalization exp -λ θ ) in each term of Equation (5.12), as the penalization for the loss until time t -1, precisely due to the online setting. Remark 5.6. Recall that the FTRL consists in using θ t-1 , x t as the prediction at time t, and that the oracle would use θ t , x t as the solution for the penalized regret minimization problem until time t. Depending on the possible values for y t ∈ {-1, +1}, the oracle prediction for y t , as the solution of the penalized regret minimization problem until time t, is either θ t , x t or θ -yt t , x t . Interestingly, y t is an average of these two potential oracle solutions, weighted by their "performance" measured by α. Note that this formulation cannot be used to compute y t for prediction, as it requires the knowledge of y t (the environment only reveals true y t after the prediction).

Analysis of regret and instant regret

We move to the analysis of the regret of the successive predictions ( y t ) t produced by OSMP (Algorithm 5). Recall that for a sequence (x t , y t ) t and some predictions ( y t ) t , the regret until round n against θ writes where we can simply upper-bound the last term λ θ 2 by λB 2 . Then it remains to upper-bound the sum. We introduce instant regret as the component term in the sum of this expression. Definition 5.7 (Instant regret). With some abuse of language, we call "instant regret" at time t the quantity defined by r t := ( y t , y t ) -L λ,t + L λ,t-1 , (5.13)

where L λ,t is defined in Equation (5.10).

The following Lemma is analogous to Theorem 2 of Mourtada and Gaïffas [2019], in particular its Equation (15) on the upper-bound of the excess risk. Lemma 5.8. The instant regret defined in (5.13) also writes r t = log exp(-L +1 λ,t + L λ,t-1 ) + exp(-L -1 λ,t + L λ,t-1 ) .

(5.14)

Proof. We plug the expression (5.11) of y t into the definition of r t to obtain r t = log 1 + exp -y t (-L +1 λ,t + L -1 λ,t ) -L λ,t + L λ,t-1 = log 1 + exp L yt λ,t -L -yt λ,t -L λ,t + L λ,t-1 , then, we notice that L yt λ,t = L λ,t hence L yt λ,t = L λ,t , and we obtain the statement in Lemma.

Next we need to upper-bound the sum of the instant regrets to obtain an upperbound of the regret on the OSMP.

An instant regret upper-bound via generalized self-concordance

From now on, we use the search space ∆ = R d as this is technically practical. Denote x H := Hx, x 1/2 for any x ∈ R d and H ∈ R d×d symmetric positive matrix. The following Proposition provides an upper-bound of the instant regrets of OSMP. Proposition 5.9. Under Assumption 5.1, we run OSMP (Algorithm 5) with λ R 2 and ∆ = R d . Consider its instant regret r t defined in Equation (5.13). that we plug back into Equation (5.17), and we remark that σ is an even function, to conclude on the statement of Proposition.

The following Lemma is a generalization of Lemma 4 of Mourtada and Gaïffas [2019] by allowing f to be a differentiable function (instead of a linear one); its proof is based the similar arguments. Since F is Σ-strongly convex, by substituting x and x , for every x and x , ∇F (x ) -∇F (x), x -x x -x 2 Σ .

Letting x = x and x = x * , we have

∇F (x), x -x * x -x * 2 Σ .
On the other hand, the Cauchy-Schwarz inequality implies that

-∇f (x), x -x * ∇f (x) Σ -1 x -x * Σ ,
plugging in, we have Proposition 5.9 is as far as goes our analysis of OSMP, for the reasons described below. The classical approach [Cesa- Bianchi and Lugosi, 2006;Hazan et al., 2007] of online learning literature to upper-bound the instant regret is to use the particular structure of the quadratic form associated to the Hessian matrix of the loss. However, with the upper-bound (5.15), the Hessian ∇ 2 L λ,t (θ t ) depends on θ t . Let us explain how it works when there is not such a dependence. In that case, we typically bring the upper-bound on the instant regret into an expression of the form x A -1 x = 1 -det(B) det(A) .

x t 2 A -1 t = x t ,
In our case, from Proposition 5.9, we have an upper-bound of the instant regret that writes (up to the constant factor e) σ ( θ t , x t ) x t 2 (∇ 2 L λ,t (θt)) -1 , where ∇ 2 L λ,t (θ t ) = t s=1 σ ( θ t , x s )x s x s + 2λI d , which is indeed very similar to the expression in Equation (5.21). Unfortunately, directly applying Equation (5. 22) would not bring us to a telescopic sum, as the Hessian ∇ 2 L λ,t (θ t ) does depend on θ t in each term of the sum i.e. each term σ ( θ t , x s )x s x s depends on θ t . We can only write

σ ( θ t , x t ) x t 2 (∇ 2 L λ,t (θt)) -1 = 1 - det ∇ 2 L λ,t-1 (θ t ) det ∇ 2 L λ,t (θ t ) log det ∇ 2 L λ,t (θ t ) det ∇ 2 L λ,t-1 (θ t ) ,
but the sum of RHS terms is not telescopable.

In the following, we might want to control each term σ ( θ t , x s ) for indices s < t, and in particular we might want to find a lower-bound of σ ( θ t , x s ) in function of σ ( θ t , x t ) that eventually leads a term in a telescopic sum. We give below a list of directions that we tried together with the specific difficulties in each. In this way, we obtain for OSMP a regret upper-bound that is logarithmic in the number of total rounds n; but exactly because of the use of the uniform lower-bound (5.23), the factor exp(BR) in the regret is unavoidable. That is the reason why we might need to use some bounds that are more precise than the uniform one (5.23).

• From the stability lemma (Lemma 5.10), we know that θ t is "not very far" from θ -yt t , and that θ t is "not very far" from θ t-1 . For example, using the fact that the function L λ,t is 2λ-strongly convex together with the stability lemma, we know that the distance between θ t and θ t-1 is upper-bounded by a constant, i.e. θ t -θ t-1 R 2λ . On the other hand, since log(σ ) = σ /σ = 1 -2σ < 1, we have for every u ∈ R, v ∈ [0, 1/2], log σ (u + v) -log σ (u) v hence σ (u + v) e v σ (u) e 1/2 σ (u). Applying this with u = θ t , x s and v = θ t-1 -θ t , x s yields (up to change (u, v) to (-u, -v) to guarantee v 0, as σ is an even function)

σ ( θ t , x s ) e -1/2 σ ( θ t-1 , x s ) (5.24)

for any indices t > 1 and s. We plug this into the Hessian ∇ 2 L λ,t (θ t ) (Equation (5.16)), then

∇ 2 L λ,

Some special cases in dimension 1

In dimension d = 1, we rewrite Equation (5.15) of Proposition 5.9 as r t e • x 2 t σ (θ t x t ) t s=1 x 2 s σ (θ t x s ) + 2λ .

(5.25) For x t taking values in {-R, 0, R}. Let us start with a simplified situation where x t ∈ {-1, 1}. Denote z t = -y t x t , then z t ∈ {-1, +1}. Denote P t := card{s t : z s = 1} the number of z s = 1, and N t := card{s t : z s = -1} the number of z s = -1. Then, L λ,t (θ) = -P t log σ(-θ) -N t log σ(θ) + λθ 2 .

(5.26)

• For λ > 0 (and we are in particular interested in the case λ 1), although we do not have a closed form for θ t or L t , we can make the following observation: since σ is an even function, z s ∈ {-1, +1}, x 2 s σ (θ t z s ) takes the same value for all s t. Therefore, it can be deduced from Equation (5.25) that • This upper-bound in form of the one of (5.27) remains true if we allow x t to take value 0: in Equation (5.25), we can simply ignore all terms in s < t with x s = 0 as they don't affect the value of the RHS of Equation (5.25). Now consider the case where x t only takes values in {-R, 0, R} for some R > 0.

We remark that for all s t,

x 2 s σ (θ t z s ) = 0, x s = 0, ξ t , otherwise, with ξ t = R 2 σ (Rθ t ). We can still deduce an upper-bound in form of (5.27) by ignoring terms with x s = 0, hence Regret n e • log n + B 2 R 2 + O(1) with λ = R 2 .

AOSMP: Approximated One-Step Minmax Predictor

In this section, we study a variant of the minmax approach presented in the previous Section: instead of doing minmax on the exact cumulative loss function, we propose to do minmax on an approximation of the cumulative loss function. We rely on the following quadratic approximation of the logistic function as in Jézéquel et al. [2020].

Lemma 5.12 (Lemma 5 of Jézéquel et al.

[2020] restated). Given x t R for all time t. For the logistic loss function induced by (x t , y t ) the features and the label of time t, t (θ) = -log σ(y t θ x t ), consider the quadratic surrogate of t (θ), developed around any θt ∈ R d , defined as ˜ t (θ) := t ( θt ) + g t (θθt ) + 1 2 η t x t (θθt )

2

(5.28) with g t := ∇ t ( θt ) = -y t σ( θt , x t )x t , η t := σ t 1 + BR := σ ( θt , x t ) 1 + BR Then, for any θ ∈ B(R d , B), ˜ t (θ) t (θ). Interested readers can refer to [Jézéquel et al., 2020] for the proof of this Lemma, which relies on an exact study of the logistic function. The quadratic lowerbound (5.28) is better than the one from exp-concavity, as the curvature parameter η t depends adaptively on the scalar product between the features x t and the point θt from which we do the development.

Algorithm

Now we present the approximated version of OSMP. We directly write the Approximated One-Step Minmax Predictor (AOSMP) with the search space ∆ = R d . Definition 5.13 (AOSMP). Given λ > 0 the ridge penalization parameter, the AOSMP computes the following estimator (5.29) with L t-1 := t-1 s=1 ( y s , y s ) the actual sum of losses suffered by successively proposed predictions ( y s ) until time t -1, and Lt-1 the function taking parameter θ, corresponding to the sum of the approximated losses until time t -1 suffered by a fixed parameter θ, i.e. Lt-1 (θ) := t-1 s=1 ˜ s (θ) where ˜ s are successive quadratic approximation functions defined in Equation (5.28) with the choice of θs specified below (Equation (5.30)).

y t =
AOSMP (5.29) is very much similar to the "exact" one defined in (5.8): both are minmax, taking the best against the worst possible scenario in terms of real label y t and the adversary parameter θ. The difference is that in (5.29), in the evaluation of the adversary parameter θ, we replace the actual cumulative loss function until the previous time L t-1 by the cumulative surrogate quadratic form Lt-1 . Replacing the true loss function by its quadratic surrogate is a common practice in the literature of online logistic regression [Hazan et al., 2007;Jézéquel et al., 2020], as the quadratic functions are easier to compute, and at the same time the quadratic forms usually allow telescopic sum in the computations in the regret analysis, as we will see later in this section.

Notations and the choice of θt

We introduce the notations Lλ,t (θ) := Lt (θ) + λ θ

Alternative expression and explicit algorithm

Similarly to OSMP, we have an explicit expression for AOSMP. We skip the proof of this Lemma as it is very much similar to the one of Lemma 5.4, relying on the properties of the log-loss. Lemma 5. Update function Lλ (θ) = Lλ,t-1 (θ) + ˜ t (θ) 10: end for

Pseudo instant regret and regret upper-bound

Now we move to the regret analysis of the AOSMP. As before, we introduce pseudo instant regret as the component term in the sum over time steps. Definition 5. 16 (Pseudo instant regret). We call "pseudo instant regret" at time t the quantity defined by r t := ( y t , y t ) -L λ,t + L λ,t-1 .

(5.33)

Figure 1 -

 1 Figure 1 -Architecture ZiMM Encoder-Decoder de-bout-en-bout.

(

  y s , y s ) -inf θ∈Θ t s=1 (θ x s , y s ) + λ θ 2 .

1. 4

 4 Online logistic regression: towards an efficient algorithm with better regret guarantee . . . . . . . . . . . . . 1.4.1 Online learning: general principles . . . . . . . . . . . . . 1.4.2 Online logistic regression, related works . . . . . . . . . . 1.4.3 Contribution: study of some non-Bayesian improper algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  al., 2020; He et al., 2020; Krizhevsky et al., 2012], Natural Languages Processing [Brown et al., 2020; Devlin et al., 2018]), the question of leveraging patient-centered healthcare records available in large observational databases using such techniques naturally become of great interest.

Figure 1 . 1 -

 11 Figure1.1 -Illustration of the sequence of claims observed for a patient. Through claims, we observe three types of events: drug purchases (blue), medical procedures (red) and diagnosis (yellow) before the medical act (prostate surgery in the example) that happens at time T i for patient i. All events are timestamped, and the time delta is a day long. Several events can occur the same day and some days have no event: events are typically very irregularly sampled. All these events, observed before T i , are used to learn the embedding vector x i ∈ R d of patient i. After T i , we only keep the events corresponding to the blurry relapses considered (drug purchases among a set of drugs for urinary problems). These blurry relapses are used to build the label vector y i ∈ N B of patient i.

Figure 1

 1 Figure 1.2 -ZiMM Encoder-Decoder end-to-end architecture.

Figure 2

 2 Figure 2.1 -Architecture of the ZiMM Encoder-Decoder end-to-end architecture. A detailed graphical representation of Steps 2-4 is shown in Figure 2.3 below.

Figure 2

 2 Figure2.2 -Illustration of the sequence of claims observed for a patient. This corresponds to the output of Step 1 from Figure2.1. Through claims, we observe three types of events: drug purchases (blue), medical procedures (red) and diagnosis (yellow) before the medical act (prostate surgery in the example) that happens at time T i for patient i. All events are timestamped, and the time delta is a day long. Several events can occur the same day and some days have no event: events are typically very irregularly sampled. All these events, observed before T i , are used to learn the embedding vector x i ∈ R d of patient i. After T i , we only keep the events corresponding to the blurry relapses considered (drug purchases among a set of drugs for urinary problems). These blurry relapses are used to build the label vector y i ∈ N B of patient i.

Figure 2 . 4 -

 24 Figure 2.4 -Flowchart leading to the final cohort considered in the experiments.

Figure 2 . 5 -

 25 Figure2.5 -Left-hand side. Number of patients as a function of the overall number of drugs prescriptions related to urinary problems during the follow-up 18 months period. The y-axis displays #{i : n i = b} where b = 1, . . . , B is given by the x-axis. We observe a sharp decrease from the mode b = 1 followed by a second weak mode around b = 15. The number of patients with n i = 0 is equal to 84 328 and is not displayed for readability of the plot. Right-hand side. Number of patients having at least one drug prescription (related to urinary problems) in each time bucket: the y-axis displays #{i, y i,b ≥ 1} as a function of b given by the x-axis. We observe a sharp decrease from the mode b = 1 and a plateau between b = 7 and b = 18, since patients often continue to purchase the drug after the surgery.

Figure 2

 2 Figure2.6 -Left-hand side. Distribution of the observation period of each patient before T i . The strong variability of the observation period displayed here is related to the variability of the medical practice itself: some patients are treated for urological problems with drugs for a long time before the surgery, while other patients have surgery sooner. Right-hand side. Distribution of the durations of all hospitalizations before T i . Most hospitalizations are one-day short, and the distribution is heavy-tailed, since these hospitalizations can be related to a large set of possible health problems.

Figure 2 . 7 -

 27 Figure2.7 -Left-hand side. Distribution of the number of events per patient before T i . The bulk of the distribution is between 5 and 200 unique events, with an average of 117 events per patient. Right-hand side. Distribution of the number of days with at least one medical code in the history of patients before T i . We observe that most patients have more than 10 days in their history with medical codes. Both figures show that most patients have a significant amount of medical code variability and time variability before T i .

Figure 2 . 8 -

 28 Figure2.8 -Validation mean-AP (y-axis) of some models evaluated during random search, where each point corresponds to a single model. The set of models is the same on all four displays and the best one (ZiMM ED default) is indicated by a red point. Each display aligns all the models that share a specific hyper-parameter, being, respectively from left to right, the number of heads used in the self-attention layer, the number of hidden units used in the recurrent layer of the ZiMM Encoder, the number of stacked recurrent layers in ZiMM Encoder and finally the number of hidden units used in the ZiMM Decoder.

  2.3 about the embeddings of medical codes).We use UMAP[START_REF] Mcinnes | UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction[END_REF] in order to reduce the dimension from 64 to 2. The resulted projections are shown in Figure2.9. The UMAP algorithm requires four hyper-parameters: the number of neighbors to consider when approximating the local metric (n-neighbors), the desired separation between close points in the embedding space (min-dist), the number of training epochs (n-epochs) and finally the projection dimension (d). We fixed d = 2 and n-epochs = 500. ICD10 disease codes are mapped in PheWAS phenotypes[START_REF] Diogo | Phenome-wide association studies across large population cohorts support drug target validation[END_REF] and drugs codes are mapped to the first level of the ATC classification (main anatomical group consisting of a single letter), leading to the color scheme used in Figure2.9.

Figure 2 . 9 -

 29 Figure 2.9 -UMAP projections of the embeddings of medical codes. Left-hand side. Each point is the projection of the embedding vector of an ICD code, colored with its corresponding PheWAS phenotypes. Note that many points share the same color since many ICD10 codes have the same PheWAS. The legend provides only the most common diagnoses. Right-hand side. UMAP projections of the drugs embeddings, colored by the first level of the ATC classification (one letter).

"

  性相近也,习相远也。 By nature, men are nearly alike; by practice, they get to be wide apart. " Confucius (translated by James Legge) Contents 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 72 3.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.3 Unsupervised training improves supervised performance 74 3.3.1 Inequalities for unsupervised training with multiple classes 75 3.3.2 Guarantees on the average supervised loss . . . . . . . . . 76 3.4 Convergence of gradient descent for contrastive unsupervised learning . . . . . . . . . . . . . . . . . . . . . . . . 77 3.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.7 Appendix: technical proofs . . . . . . . . . . . . . . . . . 81 3.7.1 Proofs for Section 3.3 . . . . . . . . . . . . . . . . . . . . 81 3.7.2 Proofs for Section 3.4 . . . . . . . . . . . . . . . . . . . . 84

  al. [2020a]; He et al. [2020]), while such embeddings are contrasted for dissimilar pairs (called the negatives).

  -way classification task is a subset T ⊆ C of cardinality |T | = k + 1, which induces the conditional distribution D T (c) = ρ(c | c ∈ T ) for c ∈ C and we define D T (x, c) = D T (c)D c (x).

. 10 )

 10 where we recall that L µ sup (f, T ) is given by (3.5). The next Proposition is a generalization to arbitrary values of k and N of Lemma 4.3 from Saunshi et al. [2019], where it is assumed k = 1 and N = 1. Proposition 3.3. Consider the unsupervised loss L N un (f ) from Equation (3.6) with N negative samples. Assume that ρ is uniform over C and that 2 k + 1 N C . Then, any encoder function f

Figure 3 . 1 -Figure 3 . 2 -

 3132 Figure 3.1 -Illustration of Lemma 3.1: we observe that Inequality (3.7) is satisfied on these examples, even without the log N C term, and that both losses behave similarly (5 runs are displayed together with their average).

Finally, Figure 3

 3 .3 displays the minimum and maximum Euclidean norms of the outputs of the encoder along training. On these examples, we observe that one can indeed assume these norms to be lower and upper bounded by constants, as stated in Assumption 3.2.

Figure 3 . 3 -

 33 Figure 3.3 -Minimum and maximum Euclidean norms of the outputs of the encoder along contrastive unsupervised training. We observe that Assumption 3.2 is satisfied on these examples (5 runs are displayed together with their average), the dashed line shows that the minimum norms are away from 0 even in the early iterations.

  Figure4.2 -Mean test AUC and standard-deviations (y-axis) using 10 train/test splits for WW and scikit-learn's implementations of RF[START_REF] Louppe | Understanding random forests: From theory to practice[END_REF] and Extra Trees[Geurts et al., 2006], using default hyperparameters, on several datasets. Thanks to subtrees aggregation, WW improves these baselines, even with few trees (x-axis is the number of trees).

Figure 4

 4 Figure 4.3 -Test AUC (top) and training time (bottom) of WW compared with very popular EGB libraries (after hyperoptimization of all algorithms, see Section 2.3 for details). WW's performance, which uses only 10 trees in this display, is only slightly below such strong baselines, but is faster (training times are on a logarithmic scale) on the considered datasets.

  then

9 :Algorithm 4

 94 return The set of log-weights {log(w den v ) : v ∈ nodes(T )} to produce predictions using Algorithm 4 below. Note that hyperoptimization of η or α, if required, does not need to grow T again, but only to update w den v for all v ∈ nodes(T ) with Algorithm 3, making hyperoptimization of these parameters particularly efficient. The recursion used in Algorithm 4 has a complexity O(|path(x)|) Computation of f (x) for any x ∈ C. 1: Inputs: Tree T , losses L v and log-weights log(w den v ) computed by Algorithm 3 2: Find v T (x) ∈ leaves(T ) (the leaf containing x) and put v ← v T (x) 3: Put f (x) ← y v (the node v forecaster, such as (4.2) for regression or (4.3) for classification) 4: while v = root do 5:

  18): whenever v ∈ intnodes(T ) and va ∈ path(x) for a ∈ {0, 1}, we have w num v(1-a) (x) = w den v(1-a) since v(1 -a) / ∈ path(x). This recursion has a complexity O(|path(x)|) where |path(x)| is the number of nodes in path(x), and is typically orders of magnitude smaller than |nodes(T )| (in a well-balanced binary tree, one has the relation |path(x)| = O(log 2 (|nodes(T )|))

T

  π(T ) exp -n oob ρ(T ) T π(T ) exp -(n oob + 1)ρ(T ) noob/(noob+1) , thus S n oob -1 η(n oob + 1) log T π(T ) exp -(n oob + 1)ρ(T ) .Using (4.23) with h(T ) = -(n oob + 1)ρ(T ) and Q = π, we have log T π(T ) exp -(n oob + 1)ρ(T ) = -T P (T )(n oob + 1)ρ(T ) -KL(P, π) + KL(P, π exp(h) )

Lemma 4. 6 .

 6 Consider classification with Y = {1, . . . , K} and a node v ∈ nodes(T ). Denote n v (k) the number of samples of class k in node v. We consider the Krichevsky-Trofimov estimator

Assumption 5. 2 (

 2 Comparison class). We use the 2 ball of radius B centered at the origin as the comparison class, i.e. Θ = B(R d , B) := {θ ∈ R d : θ 2 B} with some constant B > 0. Such Θ is compact and convex, it is sometimes called the feasible set.

  For example, in a COLT Open Problem 2012, in d = 1 dimension and x t ∈ {-R, 0, +R} setting, McMahan and Streeter [2012] used FTRL with a beta regularizer pen(θ) = c(θ, λ, λ) with c(θ, N, P ) := P log(1 + exp(-θ)) + N log(1 + exp(θ))

  θ t := argmin θ∈∆ L λ,t (θ), θ (x,y) t := argmin θ∈∆ (θ x, y) + L λ,t-1 (θ) for y ∈ {-1, +1}.

  x t ) exp(-L λ,t-1 (θ)) σ( y) , log σ(-θ x t ) exp(-L λ,t-1 (θ)) 1 -σ( y)theny t = argmin ŷ∈R max sup θ∈∆ σ(θ x t ) exp (-L λ,t-1 (θ))σ( y) ,sup θ∈∆ σ(-θ x t ) exp (-L λ,t-1 (θ)) 1 -σ( y). , for a, b > 0 as to solve the minimax problem of two parametrized linear functions, we have σ( y t ) = sup θ∈∆ σ(θ x t ) exp (-L λ,t-1 (θ))

(

  xt,y) t 2 of their Equation (47), in our case we have exp -L λ,t-1 (θ (xt,y) t

(

  y t , y t ) -n t=1 (θ x t , y t ). Note that n t=1 (θ x t , y t ) = L λ,n (θ) -λ θ 2 .Since we are under Assumption 5.2, and we assume Θ ⊆ ∆, for any comparison parameter θ ∈ Θ,Regret n (θ) n t=1 ( y t , y t ) -inf θ ∈∆ L λ,t (θ ) + inf θ ∈∆ L λ,t-1 (θ ) + λ θ 2 ,

  Lemma 5.10. (Stability) Let Ω be a nonempty open convex subset of R d , andF : Ω → R a differentiable function. Assume that F is Σ-strongly convex on Ω,where Σ is a d × d symmetric positive matrix, in the sense that, for every x, x ∈ Ω,F (x ) F (x) + ∇F (x), x -x + Assume G : x → F (x)+f (x) with f a differentiable function, such that G is strongly convex and G reaches its minimum at some x ∈ Ω. Denote x * ∈ Ω the minimum of F . Then x -x * Σ ∇f (x) Σ -1and -∇f (x), x -x * ∇f (x) 2 Σ -1 . Proof. Since x * and x are respectively minimizers of F and G, we have 0 = ∇F (x * ) and 0 = ∇F (x) + ∇f (x), the latter implies ∇F (x), x -x * = -∇f (x), x -x * .

x -x * 2 Σ

 2 ∇f (x) Σ -1 x -x * Σ hence x -x * Σ ∇f (x) Σ -1 and -∇f (x), x -x * ∇f (x) 2 Σ -1 .

  n e • log n + B 2 + O(1) with λ = 1.

  al. [2020, Lemma 5], à savoir ˜

t (θ) := t ( θt ) + g t (θθt ) + 1 2 η t x t (θθt ) 2 , avec g t := ∇ t ( θt ), η t := σ ( θt , x t )/(1 + BR) et nous choisissons θt = argmin θ∈R d Lyt λ,t (θ) avec Ly λ,t (θ) := (θ x t , y) + Lλ,t-1 (θ).

  pour la régression logistique en ligne : pour OSMP, nous avons seulement trouvé une limite supérieure pour le regret instantané r t e • σ ( θ t , x t ) • x t

	Chapter 1
	Introduction

Contents 1.1 Deep Learning and applications in healthcare . . . . . . 1.1.1 Large observational databases in healthcare: EHRs, medical claims data . . . . . . . . . . . . . . . . . . . . . . . . 1.1.2 Contribution: ZiMM, a deep learning model for long term and blurry relapses with non-clinical claims data . . . . . 1.2 Contrastive learning, unsupervised learning and supervised learning . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Contribution: theoretical guarantees on the contrastive unsupervised representation learning for classification and on its convergence . . . . . . . . . . . . . . . . . . . . . . 1.3 From Random Forest to WildWood . . . . . . . . . . . . 1.3.1 Random Forest: general principles . . . . . . . . . . . . . 1.3.2 Aggregation algorithms . . . . . . . . . . . . . . . . . . . 1.3.3 Contribution: WildWood, a new random forest algorithm

  and epidemiological studies [Aguade et al., 2020; Feldman et al., 2021] have been conducted on SNDS thanks to individual data availability.

Table 1

 1

	Model	mean-AP AUC-ROC AUC-PR
	LRl2-SF	0.19	0.64	0.50
	GBDT-SF	0.24	0.67	0.56
	MLP-SF	0.18	0.64	0.49
	LRl2-DF	0.21	0.65	0.53
	GBDT-DF	0.25	0.68	0.57
	MLP-DF	0.19	0.65	0.50
	Word2vec-ISS 0.20	0.65	0.53
	LSTM-ISS	0.21	0.67	0.54
	Patient2Vec	-	0.68	0.55
	ZiMM ED	0.306	0.701	0.619

.1 -Predictive performances (on test data) of benchmark models and ZiMM ED architecture. ZiMM ED appears to perform the best among all models both for multi-output and binary prediction.

  this case F is called comparison class. The objective of most of the studies in online learning[Cesa-Bianchi and Lugosi, 2006; Hazan, 2019; Orabona, 2019] is to obtain guarantees on regret that is valid for any individual sequence (y 1 , . . . , y n ). Therefore, we are usually interested in upper-bounds on the worst case regret of an algorithm A, i.e. upper-bounds on

  ], FOLKLORE[START_REF] Bibliography | Efficient methods for online multiclass logistic regression[END_REF]. AOSMP stands for Approximated One-Step Minmax Predictor (Algorithm 2), C GD stands for the computational cost of the gradient descent for finding L y

	Algorithm	Type	Regret upper-bound Computational cost
	OGD	Proper	BR	√	n	nd
	ONS	Proper	de BR log(n)	nd 2
	Foster	Improper, Bayesian	d log(BRn)	B

t . Algorithms with * also handle the multi-class case. 6 n 12 (Bn + d) 12 GAF* Improper, Bayesian dBR log(n) + dB 2 nd 2 + n 4 AIOLI Improper, non-Bayesian dBR log(BRn) nd 2 + nBR log(n) FOLKLORE* Improper, non-Bayesian dBR log(n)

  learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/ paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf. 8 M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2020. 18 O. Catoni. Statistical Learning Theory and Stochastic Optimization: Ecole d'Eté de Probabilités de Saint-Flour XXXI -2001, volume 1851 of Lecture Notes in Mathematics. Springer-Verlag Berlin Heidelberg, 2004. ISBN 9783540445074. URL https://books.google.fr/books?id=3Ih8CwAAQBAJ. 23 N. Cesa-Bianchi and G. Lugosi. Prediction, learning, and games. Cambridge university press, 2006. 22, 25, 26

	Chapter 2
	ZiMM: a deep learning model
	for long term and blurry
	relapses with non-clinical claims
	data

N. Cesa-Bianchi, A. Conconi, and C. Gentile. On the generalization ability of on-line learning algorithms. IEEE Transactions on Information Theory, 50(9): 2050-2057, 2004. 27, 28 T. Chen and C. Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785-794, 2016. 23 T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597-1607. PMLR, 2020. 8, 18 K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014. 14 C.-Y. Chuang, J. Robinson, Y.-C. Lin, A. Torralba, and S. Jegelka. Debiased contrastive learning. In NeurIPS, 2020. 19 M. Cuggia, D. Polton, G. Wainrib, and S. Combes. Health data hub -mission de préfiguration. Technical Report, Oct. 2018. URL https://www.vie-publique. fr/rapport/37719-health-data-hub-mission-de-prefiguration. 12 A. S. Dalalyan and A. B. Tsybakov. Aggregation by exponential weighting and sharp oracle inequalities. In International Conference on Computational Learning Theory, pages 97-111. Springer, 2007. 23 J. Devlin, M.-W. Chang, K. Lee, and K. N. Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. North American Association for Computational Linguistics (NAACL), 2018. URL https: //arxiv.org/abs/1810.04805. 8 FDA. Real-world data: Assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products -guidance for industry. Technical Report, Sept. 2021. URL https:// www.fda.gov/regulatory-information/search-fda-guidance-documents/ real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-r 8 1 " 吾生也有涯,而知也无涯。 There is a limit to our life, but to knowledge there is no limit. " Zhuang Zhou (translated by James Legge) Contents 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.1.1 Related works . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.1.2 Aim of this chapter and contributions . . . . . . . . . . . 43 2.2 Proposed architecture . . . . . . . . . . . . . . . . . . . . . 44 2.2.1 ZiMM: Zero-inflated Multinomial Mixture model . . . . . 44 2.2.2 Preprocessing of the data . . . . . . . . . . . . . . . . . . 46 2.2.3 The ZiMM Encoder . . . . . . . . . . . . . . . . . . . . . 47 2.2.4 The ZiMM Decoder . . . . . . . . . . . . . . . . . . . . . 49 2.2.5 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.3 Application: prediction of post-surgical relapse of urinary problems . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.3.1 SNIIRAM: A non-clinical claims dataset . . . . . . . . . . 50 2.3.2 Benign prostatic hyperplasia (BPH) . . . . . . . . . . . . 51 2.3.3 Surgery identification and labels construction . . . . . . . 52 2.3.4 Cohort construction and exploration . . . . . . . . . . . . 52 2.3.5 Implementation and evaluation metrics . . . . . . . . . . 56 2.3.6 Baselines . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.3.7 Ablation study: model performances and variations . . . 58 2.3.8 Visualization of the embeddings produced for diagnoses and drugs codes . . . . . . . . . . . . . . . . . . . . . . . 61 2.4 Conclusion and future works . . . . . . . . . . . . . . . . . 62

  2018;[START_REF] Shickel | Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis[END_REF]. A notable attempt is stacked denoising autoencoders[START_REF] Miotto | Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records[END_REF]. Denoising autoencoders are also used in [Beaulieu-Jones and Greene, 2016] to develop patient representation from various binary clinical descriptors on synthetic data. Sums of word-level skip-gram embedded vectors of clinical codes are used inChoi et al. [2016d] to create full-record representations. Word-level semantic embeddings for diagnosis and intervention codes are constructed in[START_REF] Pham | Predicting healthcare trajectories from medical records: A deep learning approach[END_REF], using pooling and concatenation to aggregate embeddings into a vector representing a single admission, while Nguyen et al.[2017] uses word-level embedding as preprocessing for a CNN architecture.Only a few studies were made based on claims data due to its complexity and rare availability. Choi et al. learned distributed representations of medical codes (e.g. diagnoses, medications, procedures) from electronic health records (EHRs) and claims data using Skip-gram and applied them to predict future clinical codes and risk groups[START_REF] Choi | Multi-layer representation learning for medical concepts[END_REF]. Cui2vec is a recent study in learning clinical concept embeddings[START_REF] Beam | Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data[END_REF], which applied word2vec[Mikolov et al., 2013] and Glove[START_REF] Pennington | Glove: Global vectors for word representation[END_REF] on multiple medical resources such as structured claims data, biomedical journal articles and unstructured clinical notes.Xiao et al. [2018] proposed learning a distributional representation of clinical concepts considering temporal dependencies along the longitudinal sequence of a patient's records based on claims data.Other efforts have aimed at encoding temporal aspects of EHR data for predictive tasks. InChoi et al. [2016a], time-stamped events are used as inputs to a particular type of RNNs to predict future disease diagnosis, while a graph-based attention model is used in[START_REF] Choi | Graph Convolutional Transformer: Learning the Graphical Structure of Electronic Health Records[END_REF] to learn concept representations. Another interesting recent effort is Rajkomar et al.[2018], which maps raw EHR data to the FHIR format (Fast Healthcare Interoperability Resources

codes embeddings Embeddings aggregation inside a time window via multi-head self-attention mechanism Concatenation with embeddings of horizon and hospital stay length

	2	Medical
			Drugs	Diagnoses	Procedures
	3
			t 0 i	t 1 i	T i
	4
		i t 0 t 1 i t 2 i	T i	...

Figure 2.3 -A graphical illustration of representation learning of patient pathway in the ZiMM Encoder: (2) embedding tensors for each type of medical code;

(3) aggregation of embedding vectors through multi-head self-attention inside a time window, where each gray block corresponds to one day (t i 0 , t i 1 , . . .), and T i is the day of the medical act of interest; (4) concatenation of aggregated embedding vectors with embeddings for time horizon and hospital length stay.

  to be trained and the second layer outputs a d E -dimensional vector given by For each patient, this leads to a sequence of fixed-sized embedding vectors that encode both medical and time information at each (non-empty) timestamp t T i , as displayed in Step (4) of Figures 2.1 and 2.3. Now, this sequence of vectors is used as the input of a stack of layers, including recurrent layers (LSTM[Hochreiter and Schmidhuber, 1997], bi-directional LSTM[Schuster and Paliwal, 1997], GRU[START_REF] Cho | Learning phrase representations using rnn encoder-decoder for statistical machine translation[END_REF]) or convolutional layers, in a sequence-to-one network architecture, since we want to output a single vector x i ∈ R d that encodes the full pathway of a patient before T i . The results obtained with different types of layers are shown in Section 2.3.7 below.

	Health pathway encoder. At each timestamp, the following embedding vectors
	are concatenated:
	• Medical code embedding computed by Equations (2.1) and (2.2);
	• Time-horizon embedding (see above);
	• Duration of the event embedding (see above).
	E C µ C where µ C = softmax(b tanh(BW )),	(2.2)

with W = concatenate(w 1 , . . . , w k ) ∈ R K×|C| and b and B are trainable parameters. This self-attention mechanism is trained and performed separately for each type of code, as displayed in Step (3) of Figure

2

.3. Note that this aggregation approach is permutation-invariant for codes that occur within the same day. We use dropout, drop-connect

[START_REF] Wan | Regularization of neural networks using dropconnect[END_REF]

, Frobenius-norm weight penalization from

[START_REF] Lin | A structured self-attentive sentence embedding[END_REF]

and weight-decay to regularize this self-attention mechanism.

  Absence of clinical information and "forced" normalization makes this data highly complex and heterogeneous. A consequence is that it requires a lot of domain expertise about the way healthcare is reimbursed in France in order to prepare it as training data for machine learning algorithms. In this paper, we exploit medication, procedure and diagnosis codes only. Diagnoses are primarily coded with ICD-10, while procedures are coded with CCAM, a French medical classification of clinical procedures. Medications codes use the French pharmaceutical categories CIP13[START_REF] Bezin | The national healthcare system claims databases in France, SNIIRAM and EGB: Powerful tools for pharmacoepidemiology[END_REF] that we map to the international ATC (Anatomical Therapeutic Chemical Classification) classification system.

SNIIRAM (French national health insurance information system) contains health reimbursements claims of almost all French citizens since 2015 (more than 65 million) and has been used for health research on many topics, to cite but a few Atramont et al.

[2018]

;

[START_REF] Fonteneau | The use of administrative health databases in infectious disease epidemiology and public health[END_REF]

;

[START_REF] Scailteux | French administrative health care database (SNDS): The value of its enrichment[END_REF]

. Since, in France, most health-cares are at least partially reimbursed by the administration, this database contains records corresponding to very various information going from hospital stays to drug purchases in city pharmacies, all coded with different systems

[Tuppin et al., 2017]

.

  al., 2018; Fonteneau et al., 2017; Morel et al., 2019; Scailteux et al., 2019]. Though most EHR studies are dealing with clinical EHR's Coorevits et al. [2013], it is clearly crucial to exploit such claims-only EHR, and we hope that our work is a first step towards broader use, in healthcare and machine learning research community, of non-clinical claims datasets.

Table 2 .

 2 1 -Number of medical codes observed and remaining when applying the filters used in the cohort construction. Bold values correspond to the vocabulary sizes of the medical codes used when training the ZiMM ED architecture.

		Medications Medical procedures Diagnoses	Total
	#Unique	12 785	9 578	5 885	28 248
	#Unique before T i	10 664	7 460	4 563	22 687
	#Unique seen 50 times before T i	6 713	1 155	1 403	9 271

Table 2 .

 2 2 -Predictive performances (on test data) of benchmark models and ZiMM ED architecture. ZIMM ED appears to perform the best among all models both for multioutput and binary prediction.

	Model	mean-AP AUC-ROC AUC-PR
	LRl2-SF	0.19	0.64	0.50
	GBDT-SF	0.24	0.67	0.56
	MLP-SF	0.18	0.64	0.49
	LRl2-DF	0.21	0.65	0.53
	GBDT-DF	0.25	0.68	0.57
	MLP-DF	0.19	0.65	0.50
	Word2vec-ISS	0.20	0.65	0.53
	LSTM-ISS	0.21	0.67	0.54
	Patient2Vec	-	0.68	0.55
	ZiMM ED (best model) 0.306	0.701	0.619

  Table 2.3 below.

					Hyper-parameter	Value
	Data	preprocessing	Maximum #days observed in patients' sequence before T i Maximum #medical events within the same day Vocabulary size for medications coded with ATC Vocabulary size for diagnoses coded with ICD-10 Vocabulary size for medical procedures coded with CCAM	50 24 1036 1391 1146
	Embedding of	medical codes	and time	Embedding dimension of medical codes L2 penalization rate for medical codes embedding Gaussian dropout rate Embedding dimension for time horizon and hospitalization duration L2 penalization rate for time embedding Batch normalization epsilon	64 0.005 0.3 4 0.01 1e-06
	Embeddings	aggregation	Aggregation mode Number of heads Weight drop-connect rate Dropout rate L2 penalization rate	self-attention 3 0.3 0.2 0.01
					Recurrent layer type	LSTM
	ZiMM	Encoder	#hidden units per layer #layers Dropout rate	64 1 0.3
					Recurrent dropout rate	0.2
		ZiMM Decoder		Recurrent layer type #hidden units per layer #RNN layers used in parallel for each value ni = b #common RNN layers Gaussian dropout rate Recurrent dropout rate	GRU 32 1 1 0.3 0.2
		Training		Optimizer type Learning rate Batch size	Nadam 0.001 256
					Table 2.3 -ZiMM-ED default parameters.

Table 2 .

 2 4 -Performances of some variations around the ZiMM ED default architecture, for which all hyperparameters are given in Table2.3 above. PP* stands for "preprocessing".

		Architecture modification	# params val set ×10 5 mean-AP mean-AP AUC-ROC AUC-PR test set
		ZiMM ED default	3.5	0.292	0.304	0.704	0.619
	PP*	CIP-13 drugs encoding 100-days sequence	7.1 3.5	0.279 0.291	0.286 0.300	0.690 0.701	0.604 0.616
	Embedding	Common embedding space 13.9 Without ∆t embedding 3.5 Positional encoding 3.5 E dim = 128 7.4	0.292 0.293 0.292 0.291	0.298 0.300 0.302 0.300	0.698 0.702 0.702 0.701	0.613 0.617 0.620 0.615
	Aggregation	SA N heads = 1 SA N heads = 5 SA N heads = 8 mean	3.2 3.7 4.1 3.1	0.293 0.290 0.292 0.287	0.303 0.300 0.300 0.297	0.701 0.704 0.701 0.696	0.619 0.620 0.616 0.612
		Nunits = 128	4.6	0.293	0.301	0.701	0.618
	Encoder	N layers = 2 bi-LSTM GRU Conv1D	3.8 4.2 3.3 3.2	0.290 0.290 0.294 0.231	0.306 0.300 0.300 0.234	0.701 0.700 0.702 0.649	0.619 0.617 0.614 0.536
		Multi-head transformer	6.2	0.279	0.287	0.694	0.607
	Decoder	FC layer LSTM basic RNN Nunits = 18	3.3 3.5 3.4 3.4	0.290 0.291 0.292 0.292	0.300 0.301 0.301 0.298	0.691 0.703 0.701 0.702	0.617 0.619 0.619 0.619
		N layers = 2	3.5	0.288	0.297	0.699	0.615

  al., 2020; Hénaff et al., 2019]. Contrastive methods show also strong success in natural language processing [Devlin et al., 2018; Logeswaran and Lee, 2018; Mikolov et al., 2013; van den Oord et al., 2018], video classification [Sun et al., 2019], reinforcement learning [Srinivas et al., 2020] and time-series [Franceschi et al., 2019].

  ;[START_REF] Probst | Hyperparameters and tuning strategies for random forest[END_REF] and applicability to a wide range of domains, recent examples include bioinformatics[START_REF] Qi | Random forest for bioinformatics[END_REF], genomic data [Chen and Ishwaran, 2012], predictive medicine [Alghatani et al., 2021; Subasi et al., 2017], intrusion detection [Chen et al., 2021], astronomy [Hoffman et al., 2021], car safety [Tavakoli et al., 2021], differential privacy [Patil and Singh, 2014], COVID-19 [She et al., 2020] among many others. A non-exhaustive list of developments about RF methodology include extremely randomized forests [Geurts et al., 2006], decision forests [Criminisi et al., 2012], prediction intervals [Calviño, 2020; Roy and Larocque, 2020; Zhang et al., 2020], ranking [Zhou and Qiu, 2018], nonparametric smoothing [Verdinelli and Wasserman, 2021], variable importance Kazemitabar et al. [2017]; Louppe et al. [2013]; Loyal et al. [2021], combination with boosting [Ghosal and Hooker, 2021], generalized RF [Athey et al., 2018], robust forest Li and Martin [2017], online learning [Lakshminarayanan et al., 2014; Mourtada et al., 2021] and results aiming at a better theoretical understanding of RF [Arlot and Genuer, 2014; Biau, 2012; Biau et al., 2008; Genuer, 2012; Mentch and Zhou, 2020; Mourtada et al., 2020; Scornet, 2016a,b; Scornet et al., 2015; Zhou and Mentch, 2021].

Table 4 .

 4 1 -Test AUC of all algorithms after hyperoptimization on the considered datasets. Standard-deviations are reported in Table4.3 from the supplementary material. We observe that WW has better (or identical in some cases) performances than RF on all datasets and that it is close to that of EGB libraries (bold is for best EGB performance, underline for best RFn or WWn performance).

		XGB	LGBM	CB	HGB	RF10 RF100 WW10 WW100
	adult	0.930	0.931	0.927	0.930	0.916 0.919 0.918 0.919
	bank	0.933	0.935	0.925	0.930	0.917 0.929 0.924 0.931
	breastcancer	0.991	0.993	0.987	0.994 0.974 0.978 0.992 0.992
	car	0.999	1.000 1.000 1.000 0.996 0.996 0.997 0.998
	covtype	0.999 0.999	0.998	0.999 0.996 0.998 0.996 0.998
	default-cb	0.780	0.783	0.780	0.779	0.748 0.774 0.773 0.778
	higgs	0.853	0.857	0.847	0.853	0.812 0.834 0.818 0.835
	internet	0.934	0.910	0.938	0.911	0.841 0.911 0.923 0.928
	kddcup	1.000 1.000 1.000 1.000 0.997 0.998 1.000 1.000
	kick	0.777	0.770	0.777	0.771	0.736 0.752 0.756 0.763
	letter	1.000 1.000 1.000 1.000 0.997 0.999 0.996 0.999
	satimage	0.991 0.991 0.991	0.987	0.980 0.989 0.983 0.991
	sensorless	1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000
	spambase	0.990 0.990	0.987	0.986	0.980 0.986 0.983 0.987

Table 4 .

 4 2 -Training times (seconds) of all algorithms with their default hyperparameters (no hyperoptimization) on the 5 largest considered datasets and test AUC corresponding to these training times. Test AUC scores are worse than that of Table 4.1, since no hyperoptimization is used. WW, which uses only 10 trees here (default number of trees), is almost always the fastest algorithm, for performances comparable to that of all baselines (bold is for best EGB training time or performance, underline for best RF or WW training time or performance). Standard deviations are reported in Table 4.4 of the supplementary material.

			Training time (seconds)				Test AUC
		XGB LGBM	CB	HGB	RF	WW	XGB	LGBM	CB	HGB	RF	WW
	covtype	10	3	120	14	21	3	0.986	0.978 0.989 0.960 0.998 0.979
	higgs	36	30	653	85	1389 179	0.823	0.812 0.840 0.812 0.838 0.813
	internet	9	4	188	8	0.4	0.3 0.918 0.828	0.910	0.500 0.862 0.889
	kddcup	175	41	2193	31	208	12	1.000 0.638	0.988	0.740 0.998 1.000
	kick	7	0.4	50	0.7	31	5	0.768	0.757 0.781 0.773 0.747 0.751

  and that we use here as well in the proof of Theorem 4.2.

					Its
	states that
	log	E	exp(h)dQ + KL(P, Q) -hdP = KL(P, Q exp(h) )	(4.23)
	holds for any probability measures P and Q on E and any measurable function
	h : E → R. This entails in particular that
		log		exp(h)dQ = sup	hdP -KL(P, Q) ,
			E	P

Table 4 .

 4 3 -The same test AUC scores as in Table

Table 4 .

 4 4 -The same training time table as in Table 4.2, as average over 5 runs, with standard deviation computed from 5 runs reported between parenthesis, for default parameters for each model. Top: training time in seconds; bottom: test AUC. We observe that WW is almost always the fastest algorithm, for performances comparable to ones of all baselines (bold is for best EGB training time or performance, underline for best RF or WW training time or performance).

					Training time (seconds)
			XGB	LGBM	CB	HGB	RF	WW
	covtype	10 (0.6)	3 (0.1)	120 (9.3)	14 (7.7)	21 (0.9)	3 (0.1)
	higgs		36 (0.6)	30 (1.4)	653 (8.7)	85 (0.2)	1389 (11.1) 179 (4.5)
	internet	9 (0.7)	4 (0.1)	188 (2.4)	8 (0.3)	0.4 (0.0)	0.3 (0.0)
	kddcup	175 (5.1)	41 (2.6)	2193 (13.2) 31 (0.2)	208 (3.8)	12 (0.8)
	kick		7 (0.2)	0.4 (0.0)	50 (0.7)	0.7 (0.1)	31 (0.1)	5 (0.0)
					Test AUC
			XGB	LGBM	CB	HGB	RF	WW
	covtype	0.986 (2e-04)	0.978 (2e-03)	0.989 (9e-05)	0.960 (1e-02)	0.998 (6e-05)	0.979 (5e-04)
	higgs	0.823 (3e-04)	0.812 (2e-04)	0.840 (8e-05)	0.812 (2e-04)	0.838 (9e-05)	0.813 (1e-04)
	internet	0.918 (2e-05)	0.828 (0e+00)	0.910 (8e-03)	0.500 (0e+00)	0.862 (3e-03)	0.889 (7e-03)
	kddcup	1.000 (3e-07)	0.638 (3e-02)	0.988 (7e-03)	0.740 (6e-02)	0.998 (2e-03)	1.000 (3e-06)
	kick	0.768 (4e-04)	0.757 (0e+00)	0.781 (3e-04)	0.773 (2e-03)	0.747 (2e-03)	0.751 (2e-03)

  Table 4.7 we illustrate the effects of hyperparameters on WW's performance on a few datasets, measured by the test AUC. We can observe in this table that it is only weakly affected by different combinations of hyperparameters.Table4.5 -Main characteristics of the datasets used in experiments, including number of samples, number of features, number of categorical features, number of classes and the Gini index of the class distribution on the whole datasets (rescaled between 0 and 1), in order to quantify class unbalancing. Table4.7 -Areas under the ROC curves (AUC) obtained on test samples with WildWood (using 100 trees in the forest) on the adult, bank and car datasets with combinations of several hyper parameters. We observe that WildWood's performance does not vary significantly with respect to these hyperparameters.

	Dataset n min-leaf	adult α	bank # Samples # Features # Categorical features # Classes Gini car η AUC n min-leaf α η AUC n min-leaf α η	AUC
	adult bank	1 1 1	0.1 0.1 0.1	48,842 0.1 0.913 45,211 1.0 0.916 10.0 0.918	14 16	1 1 1	0.1 0.1 0.1	0.1 1.0 10.0	8 0.919 10 0.926 0.929	2 2	0.1 0.1 0.1	0.1 0.36 1.0 0.21 10.0	0.992 0.995 0.995
	breastcancer 1 0.5 car 1 0.5 1 0.5	0.1 1.0 10.0	569 0.913 1,728 0.917 0.919	30 6	1 1 1	0.5 0.5 0.5	0.1 1.0 10.0	0 0.920 6 0.927 0.929	2 4	0.5 0.5 0.5	0.1 0.47 1.0 0.46 10.0	0.992 0.994 0.995
	covtype 1 1	2.5 2.5	581,012 0.1 0.913 1.0 0.917	54	1 1	2.5 2.5	0.1 1.0	0 0.921 0.927	7	2.5 2.5	0.1 0.62 1.0	0.990 0.992
	default_cb 1 2.5	30,000 10.0 0.919	23	1	2.5	10.0	3 0.929	2	2.5	0.34 10.0	0.993
	higgs	5 5	0.1 0.1	11,000,000 0.1 0.913 1.0 0.916	28	5 5	0.1 0.1	0.1 1.0	0 0.919 0.926	2	0.1 0.1	0.1 0.50 1.0	0.990 0.992
	internet 5 kddcup99 0.1 5 0.5 5 0.5	10,108 10.0 0.918 4,898,431 0.1 0.913 1.0 0.916	70 41	5 5 5	0.1 0.5 0.5	10.0 0.1 1.0	70 0.928 7 0.920 0.926	46 23	0.1 0.5 0.5	10.0 0.88 0.1 0.58 1.0	0.992 0.990 0.992
	kick letter	5 5 5	0.5 2.5 2.5	72,983 10.0 0.918 20,000 0.1 0.913 1.0 0.917	32 16	5 5 5	0.5 2.5 2.5	10.0 0.1 1.0	18 0.928 0 0.921 0.927	2 26	0.5 2.5 2.5	10.0 0.22 0.1 0.96 1.0	0.992 0.987 0.991
	satimage 5 10	2.5 0.1	10.0 0.1	5,104 0.918 0.913	36	5 10	2.5 0.1	10.0 0.1	0 0.928 0.920	6	2.5 0.1	10.0 0.81 0.1	0.991 0.983
	sensorless 10 0.1	58,509 1.0 0.916	48	10	0.1	1.0	0 0.926	11	0.1	0.91 1.0	0.987
	spambase 10 0.1 10 0.5	10.0 0.1	4,601 0.917 0.913	57	10 10	0.1 0.5	10.0 0.1	0 0.927 0.920	2	0.1 0.5	10.0 0.48 0.1	0.987 0.983
	10	0.5	1.0	0.916		10	0.5	1.0	0.926		0.5	1.0	0.987
	10	0.5	10.0	0.917		10	0.5	10.0	0.927		0.5	10.0	0.987
	10	2.5	0.1	0.913		10	2.5	0.1	0.920		2.5	0.1	0.981
	10	2.5	1.0	0.916		10	2.5	1.0	0.926		2.5	1.0	0.985
	10	2.5	10.0	0.918		10	2.5	10.0	0.928		2.5	10.0	0.986

  . Zhou and G. Qiu. Random forest for label ranking. Expert Systems with Applications, 112:99-109, 2018. ISSN 0957-4174. doi: https://doi.org/10.1016/j. eswa.2018.06.036. URL https://www.sciencedirect.com/science/article/ pii/S095741741830383X. 94We consider online logistic regression: an online learning problem with binary labels and logistic loss. Under this setting, at each time step t = 1, 2, . . ., an agent successively makes predictions y t , based on all the previous observations ((x 1 , y 1 ), . . . , (x t-1 , y t-1 )) and the features x t of time t. Once the true binary label y t ∈ {-1, +1} is unveiled, the agent suffers a logistic loss ( y t , y t ) := log (1 + exp (y t y t ))) . quality of the successive predictions ( y t ) t is assessed by the regret, defined as the excess loss induced by ( y t ) t versus an algorithm that uses a logistic model with the best possible fixed parameter θ ∈ Θ in hindsight,

	5.1 Introduction
	The Regret n :=	n t=1	( y t , y t ) -inf θ∈Θ	n t=1	(θ x t , y t )
	• lightgbm (3.2.1), MIT License
	• matplotlib (3.3.1), license: https://github.com/matplotlib/matplotlib/
	blob/master/LICENSE/LICENSE
	• numba (0.52), BSD 2-Clause "Simplified" License
	• numpy (1.19.2), BSD-3-Clause License
	• pandas (1.2.4), BSD-3-Clause License
	• python (3.7.9), Python Software Fundation Licence version
	• scikit-learn (0.24.2), BSD 3-Clause License
	• scipy (1.5.4), BSD 3-Clause License

• seaborn (0.11), BSD-3-Clause License • xgboost (1.4.1), Apache License 2.0 All the datasets used are publicly accessible and have no copyright restrictions.

Y

  Consider an individual sequence (x t , y t ) t and the successive predictions ( y t ) t produced by an algorithm A, the regret against a comparator θ ∈ R d for n total rounds writes

			n		n
	Regret n (θ) :=	( y t , y t ) -			(θ x t , y t ),
			t=1	t=1
	and the regret against the comparison class Θ ⊆ R d for total rounds n writes
		n			n
	Regret n :=	t=1	( y t , y t ) -inf θ∈Θ	t=1	(θ x

t , y t ).

  (θ) in expression of f t (θ t ). Especially, the approximation function ft defined by ft

.3) for any θ and any θ t . The property (5.3) controls the variation of the Hessian around θ t , hence it can deduce the local strong convexity around θ t . Moreover, such bounds (5.3) on second derivatives can be applied on the function f t to deduce bounds of f t

  Recent work of Jézéquel et al. [2020] introduced a new quadratic approximation of the logistic function

ft : θ → f t (θ t ) + ∇f t (θ t ) (θ -θ t ) + exp(y t θ t x t ) 2(1 + BR) (θ -θ t ) ∇f t (θ t )∇f t (θ t ) (θ -θ t ), (5.5)

and showed that under Assumption 5.1, ft is a lower-bound of f t on the ball B(R d , B), for any θ t . The proof of this lower-bound relies on a study of the logistic function which is more sophisticated than the exp-concavity: instead of a fixed coefficient β being uniform and exponential in BR in (5.2), the secondorder coefficient in (5.5) adaptively depends on y t θ t x t . Recent works

[START_REF] Bibliography | Efficient methods for online multiclass logistic regression[END_REF] Jézéquel et al., 2021

] also rely on the lower-bound (5.5) while generalizing it to the multi-class case.

  [START_REF] Kakade | Online bounds for bayesian algorithms[END_REF] was a first result suggesting that a regret of O(d log(n/d)) is achievable for online logistic regression, by analyzing the following Bayesian Model Averaging with Gaussian prior p 0 = N (θ; 0, ν 2 I d ). It predicts at time t, For this procedure, with variational techniques, Kakade and Ng [2005] showed a regret upper-bound in O(d log(Bn/d)), plus an additional penalty term that depends on the prior, as well as on the squared 2 -norm of the best comparator θ [Kakade and Ng, 2005, Theorem 2.2]. It is not clear how to efficiently compute such a predictor.

	y t =	θ • p t-1 (dθ) x t
	Θ
	and updates the distribution p t by
		t
	p t (θ) ∝ exp -	(θ x s , y s ) p 0 (θ).
		s=1
	lary 7]; and for any larger dimension d 2, Ω( √	al., 2014, Corol-Bn) regret in the worst case [Hazan
	et al., 2014, Corollary 8]. Therefore, any regret bound of form O(B log n) is impos-
	sible for proper algorithms.
	Bayesian proper method

  Note that these works are results on the characterization of the minimax regret, and do not provide an explicit algorithm.

				proposed regret lower-and upper-bounds
	under Bayesian setting. Among other bounds, in particular, for "smaller d", with
	θ 2 B,	d 2	log	n d 2 Regret n	d 2	log	B 2 n d	.

In a similar direction, another recent work Jacquet et al. [2021] computed the precise minimax regret in O d 2 log(2n/π) in the case of categorical feature values.

  ], FOLKLORE [Agarwal et al., 2021]. AOSMP stands for Approximated One-Step Minmax Predictor (Algorithm 6 in this work), C GD stands for the computational cost of the gradient descent for finding L y t . Algorithms with * also handle the multi-class case. Bayesian dBR log(n) + B 2 R 2 nd 2 + nC GD

	Algorithm	Type	Regret upper-bound Computational cost
	OGD	Proper	BR	√	n	nd
	ONS	Proper	de BR log(n)	nd 2
	Foster	Improper, Bayesian	d log(BRn)	B 6 n 12 (Bn + d) 12
	GAF*	Improper, Bayesian	dBR log(n) + dB 2	nd 2 + n 4
	AIOLI	Improper, non-Bayesian	dBR log(BRn)	nd 2 + nBR log(n)
	FOLKLORE* Improper, non-Bayesian	dBR log(n)	nd 2 + nBR log(n)
	AOSMP	Improper, non-

  As for the prediction at time t, the One-Step Minmax Predictor (OSMP) aims to minimize this quantity with respect to the worst case over all possible values of label y t ∈ {-1, 1} and over all possible "adversary parameter" θ. Let us study this predictor under Assumptions 5.1 and 5.2. Definition 5.2 (OSMP). Given λ > 0 the ridge penalization parameter. Denote ∆ ⊆ R d the search space for the parameter θ, such that Θ ⊆ ∆ and such that ∆ is convex. OSMP computes the estimator y t = argmin + ( y, y t ) -L t-1 (θ) + (θ x t , y t ) + λ θ 2

	ŷ∈R	sup yt∈{-1,1}	sup θ∈∆	L t-1 (5.8)
	with notations
			t-1
		L t-1 :=
			s=1
			t	t
			( y s , y s ) -	(θ x s , y s ) + λ θ 2 .	(5.7)
			s=1	s=1

  Proof. Since L t-1 does not depend on y t nor on θ, we can drop it from argmin; also, we can swap sup yt and sup θ , and rewrite the Equation (5.8) as

				1 λ,t .	(5.11)
	y t = argmin	sup	sup	( y, y t ) -(θ x t , y t ) -L λ,t-1 (θ)
	ŷ∈R	θ∈∆	yt∈{-1,1}
	where we notice that
	sup yt∈{-1,1}			yt∈{-1,1}	log	σ(y t θ x t ) σ(y t y)
				= max log	σ(θ x t ) σ( y)	, log	σ(-θ x t ) σ(

( y, y t ) -(θ x t , y t ) = sup

  Denote θ -yt . We remark for OSMPy t θ -yt t , x t y t y t = L -yt λ,t -L λ,t y t θ t , x t ,and we can understand y t from Equation (5.11) as a linear combination of θ -yt t , x t and θ t , x t . Namely, there is 0 α 1 such thaty t y t = α • θ t , x t + (1 -α) • θ -yt t ,x t with α = L -yt λ,t -L λ,t -y t θ -yt t , x t y t θ t , x t -y t θ -yt t , x t .

	t	= θ	(xt,-yt) t

  The instant regret is upper-bounded, namelyr t e • σ ( θ t , x t ) • x t 2 (∇ 2 L λ,t (θt)) -1 , (5.15)where ∇ 2 L λ,t (θ t ) stands for the Hessian of L λ,t evaluated on θ t ,∇ 2 L λ,t (θ t ) = (θ x t , y t ) + L t-1 (θ) + λ θ 2 ,denote z t = -y t x t , and denote-θ t ∇ 2 L λ,t (θt) e 1/2 x t (∇ 2 L λ,t (θt)) -1 .Hence, putting together Equations (5.18) and (5.19), we obtainσ( θ -yt t , z t ) -σ( θ t , z t ) e • σ ( θ t , z t ) • x t 2 (∇ 2 L λ,t (θt)) -1 = e • σ ( θ t , z t ) • (∇ 2 L λ,t (θ t )) -1 x t , x t= e • Tr (∇ 2 L λ,t (θ t )) -1 σ ( θ t , z t )x t x t

	and
	θ -yt t
		t
		σ ( θ t , x s )x s x s + 2λI d .	(5.16)
		s=1
	Proof. Recall that
	θ t = argmin	L λ,t (θ) = argmin
	θ∈R d	θ∈R d
	θ -yt

t := argmin θ∈R d (θ x t , -y t ) + L t-1 (θ) + λ θ 2 = argmin θ∈R d L t (θ) -θ, z t + λ θ 2 ,

  A -1 t x t , where A t := and ends up with a telescopic sum on RHS. We recall Lemma 11.11 of Cesa-Bianchi and Lugosi [2006] below. Lemma 5.11 (Lemma 11.11 of Cesa-Bianchi and Lugosi [2006]). Let B be an arbitrary d × d full-rank matrix, and x ∈ R d an arbitrary vector. Let A = B + xx . Then

				t
				x s x s + 2λI d .	(5.21)
			s=1
	Then, we can use Cesa-Bianchi and Lugosi [2006, Lemma 11.11] which leads to
	x t , A -1 t x t = 1 -	det A t-1 det A t	log	det A t det A t-1	(5.22)

  • A first idea is to use the uniform lower-bound, i.e. for θ t From this, we simply use σ ( θ t , x t ) 1/4 and apply Lemma 11.11 of Cesa-Bianchi and Lugosi [2006], with A t := exp(-BR) t s=1 x s x s + 2λI d , we obtainσ ( θ t , x t ) x t 2 (∇ 2 L λ,t (θt)) -1Putting this into Equation (5.15), we can upper-bound the sum of instant regrets by a telescopic sum which results

			1 4	x t	2 A -1 t
		=	exp(BR) 4	1 -	det A t-1 det A t	exp(BR) 4	log	A t A t-1	.
	n t=1	r t	exp(BR + 1) 4	log	det A n det A 0
	where we can use
	log det (A n /2λ) d log 1 +	e -BR nR 2 2dλ	.
	Finally, with the choice λ = R 2 , we have
	Regret n	e BR+1 4	d log 1 +		e -BR 2d	n + B 2 R 2 .
									B and x s	R,
	σ ( θ t , x s ) exp(-BR),	(5.23)

then

∇ 2 L λ,t (θ t ) exp(-BR) t s=1 x s x s + 2λI d .

  t (θ t ) σ ( θ t , x t )x t x t + e -1/2 , x s )x s x s + 2λI d σ ( θ t , x t )x t x t + e -1/2 ∇ 2 L λ,t-1 (θ t-1 ).Hence, applying Lemma 11.11 of Cesa-Bianchi and Lugosi[2006], we obtain

	t-1
	σ ( θ t-1
	s=1

σ ( θ t , x t ) x t 2 (∇ 2 L λ,t (θt)) -1 1 -e -d/2 det ∇ 2 L λ,t-1 (θ t-1 ) det ∇ 2 L λ,t (θ t ) log det ∇ 2 L λ,t (θ t ) det ∇ 2 L λ,t-1 (θ t-1 ) + d 2 .

  argmin L t-1 + ( y, y t ) -Lt-1 (θ) + (θ x t , y t ) + λ θ 2

		sup	sup
	ŷ∈R	yt∈{-1,+1}	θ∈R d

  Lyλ,t (θ) with Ly λ,t (θ) := (θ x t , y) + Lλ,t-1 (θ). is the point around which we do the quadratic development for ˜ t in Equation (5.28). Making such choice is possible after the true label y t is revealed.

	2 , L λ,t (θ) := (θ x t , y) + Lλ,t-1 (θ) λ,t := inf θ∈R d Lt (θ) + λ θ 2 and for y ∈ {-1, +1}, the functions Ly and their infimum Ly λ,t := inf θ(xt,y) θ∈R d We remark that θ(xt,y) t exists and is unique as the minimizer of a strongly convex function Ly t . In particular, we choose θt := θ(xt,yt) t (5.30) θ∈R Denote for y ∈ {-1, +1}, such that θt

d (θ x t , y) + Lλ,t-1 (θ)

for y ∈ {-1, +1}. t := argmin

  Compute θt ← argmin θ∈R d (θ x t , y t ) + Lλ,t-1 (θ) , and function ˜ t (θ) as specified in Equation (5.28)

		p t =	exp(-L+1 λ,t ) exp(-L+1 λ,t ) + exp(-L-1 λ,t )	(5.32)
		Let us summarize AOSMP in Algorithm 6.
	4: 5:	Receive x t Compute Ly
			λ,t
	7:	Receive y t
	8:

14. AOSMP defined in Equation (5.29) is equivalent to y t = -L+1 λ,t + L-1 λ,t (5.31) Remark 5.15. AOSMP predicts the probabilities p(+1 | x t ) = p t and p(-1 | x t ) = 1 -p t with Algorithm 6 AOSMP description: computation of y t . 1: Inputs: Parameters λ > 0, n, d 1, constants B, R > 0 2: Initialize: Function Lλ,0 (θ) = λ θ 2 3: for t = 1, . . . , n do λ,t ← inf θ∈R d (θ x t , y) + Lλ,t-1 (θ) for y = -1 and y = +1 6: Predict y t ← -L+1 λ,t + L-1 9:

(∇ 2 L λ,t (θt)) -1 ; pour AOSMP, nous avons prouvé une limite supérieure de regret dans un ordre similaire avec l'algorithme de pointe[Jézéquel et al., 2020] pour la régression logistique binaire en ligne au meilleur de notre connaissance. La question d'un algorithme efficace avec une meilleure limite supérieure de regret pour la régression logistique en ligne reste ouverte.

We considered that a TURP surgery corresponds to the CCAM codes JGFA005, JGFA009, JGFA015, JDPE002 or JGNE003.

Θ e -ηL θ π(dθ )

This chapter is based on the joint work with A.Kabeshova, S. Gaiffas, E. Bacry, B. Lukacs.

We considered that a TURP surgery corresponds to the CCAM codes JGFA005, JGFA009, JGFA015, JDPE002 or JGNE003.

Following clinicians recommendations, we do not consider TURP surgeries that occur after the first surgery block, and consider only drugs prescriptions as blurry relapses. Consequently, we remove patients with repeated surgeries that are not part of the same block from the cohort construction, as explained in Section 2.3.4.

https://github.com/stephanegaiffas/zimm.git

This chapter is based on the joint work with I.Merad, S. Gaiffas, E. Bacry.

i = ∇ z (z i ).

This chapter is based on the joint work with S. Gaiffas, I. Merad.

Table4.6 -The URLs of all the datasets used in the paper, giving direct download links and supplementary details.

This chapter is based on the joint work with I. Merad, J. Mourtada, S. Gaiffas.

An excellent post on FTRL-version and OMD-version of OGD https://www.timvanerven.nl/ blog/ftrl-vs-omd/.

Remerciements Acknowledgments

This research benefited from the National Health Insurance Fund (CNAM) partnership with Data Science initiative at École polytechnique. We thank Maryan Morel, Dinh Phong Nguyen, Youcef Sebiat, Dian Sun, researchers and engineers working on this project. Yiyang Yu was supported by grants from Région Ile-de-France. Anastasiia Kabeshova was supported by CNAM-Polytechnique research partnership. This research is also supported by the Agence Nationale de la Recherche as part of the "Investissements d'avenir" program (reference ANR-19-P3IA-0001; PRAIRIE 3IA Institute). Finally, we wish to thank Pr. Eric Vicaut, for his help in the understanding and the usage of the database.

débugging. Merci à Ibrahim pour ton enthousiasme, pour nos passionnantes discussions. Merci à Prof. Lukacs pour vos

Dataset URL adult https://archive.ics.uci.edu/ml/datasets/Adult bank https://archive.ics.uci.edu/ml/datasets/bank+marketing breastcancer https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(diagnostic) car https://archive.ics.uci.edu/ml/datasets/car+evaluation covtype https://archive.ics.uci.edu/ml/datasets/covertype default_cb https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients higgs https://archive.ics.uci.edu/ml/datasets/HIGGS internet https://kdd.ics.uci.edu/databases/internet_usage/internet_usage.html kddcup99 https://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html kick https://www.openml.org/d/41162 letter https://archive.ics.uci.edu/ml/datasets/letter+recognition satimage https://archive.ics.uci.edu/ml/datasets/Statlog+(Landsat+Satellite) sensorless https://archive.ics.uci.edu/ml/datasets/dataset+for+sensorless+drive+diagnosis spambase https://archive.ics.uci.edu/ml/datasets/spambase

Chapter 4

WildWood: a new Random Forest Algorithm 1 " 莫听穿林打叶声,何妨吟啸且徐行。竹杖芒鞋轻胜 马,谁怕?一蓑烟雨任平生。 Listen not to the rain beating against the trees, Why don't you slowly walk and chant at ease? Better than saddled horse I like sandals and cane, O I would fain? Spend a straw-cloaked life in mist and rain. "

Abstract

We introduce WildWood (WW), a new ensemble algorithm for supervised learning of Random Forest (RF) type. While standard RF algorithms use bootstrap out-of-bag samples to compute out-of-bag scores, WW uses these samples to produce improved predictions given by an aggregation of all possible subtrees of each fully grown tree in the forest. This is achieved by aggregation with exponential weights computed over out-of-bag samples, that are computed exactly and very efficiently thanks to an algorithm called context tree weighting. This improvement, combined with a histogram strategy to accelerate split finding, makes WW fast and competitive compared with other well-established ensemble methods, such as standard RF and extreme gradient boosting algorithms.

Introduction

We introduce WildWood (WW), a new ensemble method of Random Forest (RF) type [START_REF] Breiman | Random forests[END_REF]. The main contributions of this work and the main advantages of WW are as follows.

Firstly, we use out-of-bag samples (trees in a RF use different bootstrapped samples) very differently than what is done in standard RF [START_REF] Biau | A random forest guided tour[END_REF][START_REF] Louppe | Understanding random forests: From theory to practice[END_REF]. Indeed, WW uses these samples to compute an aggregation of the predictions of all possible subtrees of each tree in the forest, using aggregation with exponential weights [START_REF] Catoni | Statistical Learning Theory and Stochastic Optimization: Ecole d'Eté[END_REF]. This leads to much improved predictions: while only leaves contribute to the predictions of a tree in standard RF, the full tree structure contributes to predictions in WW. An illustration of this effect is given in Figure 4.1 on a toy binary classification example, where we can observe that subtrees aggregation leads to improved and regularized decision functions for each individual tree and for the forest. We further illustrate in Figure 4.2 that each tree becomes a stronger learner, and that excellent performance can be achieved even when WW uses few trees. A remarkable aspect of WW is that this improvement comes only at a small computational cost, thanks to a technique called "context tree weighting", used in lossless compression or online learning to aggregate all subtrees of a given tree [START_REF] Catoni | Statistical Learning Theory and Stochastic Optimization: Ecole d'Eté[END_REF][START_REF] Helmbold | Predicting nearly as well as the best pruning of a decision tree[END_REF] Numerator of Equation (4.9). The numerator of Equation (4.9) almost follows the exact same argument as the denominator, but since it depends on the input vector x ∈ C of features for which we want to produce a prediction, it is performed at inference time. Recall that path(x) is the sequence of nodes that leads to the leaf v T (x) containing x ∈ C and define, for any v ∈ nodes(T ), w v (x) = w v y v (x) if v ∈ path(x), and w v (x) = w v otherwise. We have

Note that (4.15) comes from (4.12) while (4.16) comes from the definition of w v (x) (note that a single term from the product over v ∈ leaves(T )

We are now in position to use again Lemma 4.5 with g(v) = w v (x). Defining

we can conclude that

and that the following recurrence holds:

This recurrence allows to compute w num v (x) from w v (x), but note that a direct use of this formula would lead to a complexity O(|nodes(T )|) to produce a prediction for a single input x ∈ C. It turns out can we can do much better than that.

Indeed, whenever v / ∈ path(x), we have by definition that w v (x) = w v and that w v (x) = w v for any descendant v of v, which entails by induction that w num v (x) = w den v for any v / ∈ path(x). Therefore, we only need to explain how to compute w num v (x) for v ∈ path(x). This is achieved recursively, thanks to (4.18), starting at the leaf v T (x) and going up in the tree to root:

Now, using the concavity of the logarithm gives

This concludes the proof of Lemma 4.6.

Proof of Corollary 4.3. The log-loss is trivially 1-exp-concave, so that we can choose η = 1. Following Theorem 4.2, it remains to bound the regret of the tree forecaster T with respect to the optimal labeling of its leaves. For classification and the log loss, we use Lemma 4.6 to obtain

for any subtree T and any v ∈ leaves(T ). Now, summing over v ∈ leaves(T ), of cardinality ( T + 1)/2, leads to

which concludes the proof of Corollary 4.3.

Proof of Corollary 4.4. The square loss is 1/(8B 2 )-exp-concave on [-B, B], see Cesa-Bianchi and Lugosi [2006], so we can choose η = 1/(8B 2 ). Following Theorem 4.2, it remains to bound the regret of the tree forecaster T with respect to the optimal labeling of its leaves. For regression with the least-squares loss, since we use the empirical mean forecaster (4.2), we have

for any subtree T and any leaf v ∈ leaves(T ). The rest of the proof follows that of Corollary 4.3.

Appendix: experiments

Supplementary details on experiments

We report in Table 4.3 the same test AUC scores as in Table 4.1 of all algorithms after hyperoptimization on the considered datasets. Standard-deviations displayed between parentheses are computed from 5 trainings with different random seeds. We observe that WW has better (or identical in some cases) performances than RF on all datasets and that it is close to that of EGB libraries (bold is for best EGB performance, underline for best RFn or WWn performance). We report also in Table 4.4 the same training time and test AUC as in Table 4.2, with standard-deviations displayed between parentheses computed from 5 trainings with different random seeds, all with default hyperparameters of each algorithm. We observe that WW is almost always the fastest algorithm, for performances comparable to ones of all baselines (bold is for best EGB training time or performance, underline for best RF or WW training time or performance).

Supplementary details about hyperparameters tuning

In this Section, we provide extra information about hyperparameters optimization. For XGBoost, LightGBM and CatBoost, with all other hyperparameters fixed, we use early stopping by monitoring the log loss on the validation set, the maximum number of boosting iterations being set at 5, 000. The best number of iterations is used together with other best hyperparameters to refit over the whole training set before evaluation on the test set. For scikit-learn's Random Forest and WildWood, we report results both for 10 and 100 trees, note that the default choice is 10 for WildWood (since subtrees aggregation allows to use fewer trees than RF) while default is 100 in scikit-learn. We list the hyperparameters search space of each algorithm below.

XGBoost

as (θ x t , -y

Then from Equation (5.14),

We use the fact that L λ,t-1 (θ t ) L λ,t-1 and L λ,t-1 (θ -yt t ) L λ,t-1 , and σ(-u) = 1 -σ(u), then log(1 + u) u for any u > -1, to write

(5.17)

Next, to upper-bound the RHS, we roughly follow similar techniques as in the proof of Theorem 5 of Mourtada and Gaïffas [2019]. For the sake of completeness, we present the main steps here. Since L λ,t (θ) -θ, z t = L -yt λ,t (θ), and the function L λ,t is 2λ-strongly convex (and x t R from Assumption 5.1), it follows from Lemma 5.10 below that

where we also use the assumption λ R 2 for the last inequality. Next, since log(σ ) = σ /σ = 1 -2σ < 1, we have for every In this way, we get rid of the exponential factor in the regret upper-bound, but we have a term that is linear in the number of total rounds n: the approach with (5.24) fails to show a logarithmic regret. We might notice that the d/2 additional term comes exactly from the factor e -1/2 in Equation (5.24); to avoid the linear term, a lower-bound with a factor more precise than e -1/2 might be useful instead of Equation (5.24).

• Indeed, we may more heavily exploit the generalized R-self-concordance of the function L λ,t , hence more precisely characterize the distance between θ t and θ t-1 . As consequence, L λ,t is not only 2λ-strongly convex but also "locally" e -1/2 ∇ 2 L λ,t (θ t )-strongly convex. Using this together with Lemma 5.10, we have

On the other hand, with similar arguments as before, namely from log(σ

where we can use the Cauchy-Schwarz inequality

We plug this into the Hessian ∇ 2 L λ,t (θ t ) (Equation (5.16)), then

But it is not clear what can be done with this.

Finally, we also remark that there is a similar situation in the proof of Theorem 6 of Mourtada and Gaïffas [2019], Equations ( 92)- (94). Under the setting of i.i.d. data and while looking for upper-bounds in expectation, they used an argument of exchangeability (Z 1 , . . . , Z t-1 , Z t follow the same distribution). Unfortunately, this argument is not possible in the online setting, as we do not make any assumption on the distribution of data (x t , y t ) t , and we look for a regret upper-bound that is valid for any individual sequence. Lemma 5.17. An alternative expression for the pseudo instant regret writes

(5.34)

Proof. We plug the expression of y t (Equation (5.31)) into the definition of r t (Equation (5.33)) to obtain σ(-u) = 1 -σ(u) and log(1 + u) u on the third line. Then, using similar arguments (namely the generalized R-self-concordance of the function Lyt λ,t under Assumption 5.1, then λ R 2 , followed by a study of the function σ) as in the proof of Proposition 5.9, we obtain

(5.38)

Putting together Equations (5.35), (5.36), (5.37) and (5.38) implies

.

By summing for t from 1 to n, using L0 (θ) = 0, A 0 = 2λI d , and the telescopic sum, we have

Now we are ready to state the Theorem on the regret upper-bound of AOSMP. Theorem 5.19. Let λ, R, B > 0 and d, n 1. Let (x 1 , y 1 ), . . . , (x n , y n ) ∈ X × Y be an arbitrary sequence of observations. Under Assumptions 5.1 and 5.2, we run AOSMP (Algorithm 6) with λ R 2 . Then, its regret satisfies the following upperbound, against any θ ∈ B(R d , B),

In particular, by choosing λ = R 2 , it yields

Regret n e

Then, applying Lemma 5.18 as we assume λ R 2 to upper-bound the sum of pseudo instant regrets term, and using the notations of the same Lemma, we obtain

To upper-bound the RHS, we recall that

We obtain for AOSMP, a regret upper-bound in O(dBR log(n) + B 2 R 2 ), which is logarithmic in the number of total rounds n, with a multiplicative constant of order dBR, and an additional term B 2 R 2 . Considering Remark 5.1, B 2 R 2 d, the regret upper-bound is in O(d √ d log(n)). Finally, we remark that the upper-bound of AOSMP is of the same order as the one of AIOLI [Jézéquel et al., 2020], since we use the same quadratic approximation (Lemma 5.12).

Discussion and perspective

In this work, for online logistic regression, we investigated two new candidate algorithms inspired by SMP [Mourtada and Gaïffas, 2019].

• For OSMP, we found an upper-bound for instant regret r t e • σ ( θ t , x t ) • x t 2 (∇ 2 L λ,t (θt)) -1 . Despite our initial thoughts, we were facing some technical difficulties and were not able to obtain a regret upper-bound at the same level with best known algorithm.

• For AOSMP which uses a quadratic surrogate of the logistic function, we proved a regret upper-bound which is similar to the one of Jézéquel et al. [2020], stateof-the-art for binary online logistic regression to the best of our knowledge.

The question for better regret with an efficient algorithm for online logistic regression remains open. Also, we believe that the generalization of OSMP and AOSMP in the case of multi-class logistic regression is certainly possible.