Alain Coulais

David Cornu

Florence Durret

Florent Mertens

François Boulanger

Martina Wiedner

Murielle Chevrier

J'aimerais tout d'abord exprimer toute ma gratitude envers Benoît pour sa bienveillance et sa patience. Je garderais un souvenir chaleureux de nos discussions durant ces trois années et je ne pourrais jamais assez souligner la chance d'avoir pour encadrant quelqu'un qui, malgré ses responsabilités de chercheurs, d'enseignant et ensuite directeur de laboratoire, a toujours répondu à la question "Combien de temps tu as pour discuter ?" par "Le temps qu'il faudra".

Je souhaiterais ensuite remercier Barbara, collègue de bureau puis amie, pour son soutien et tous ces moments partagés lors de ces trois dernières années. Merci également à Woihiba pour son dynamisme et sa gentillesse, Anaëlle pour sa générosité, Françoise pour son expérience, Fabienne pour son écoute et Florian et Evan pour leur accueil et leur bienveillance à mes débuts. Enfin toute ma gratitude va aux membres de l'Observatoire qui, au travers de discussions, m'ont beaucoup apporté :

Il n'y a qu'une façon d'échouer, c'est d'abandonner avant d'avoir réussi.

The only way to fail is to give up before you succeed.

Abstract

The Epoch of Reionization (EoR) is a key period in the history of our Universe. The growth of the first structures and the overall phase transition toward an ionized intergalactic medium are complex processes that need to be better understood. However, observations of the EoR are rare and difficult to obtain. In these conditions, it is especially relevant to develop and improve ways of inferring the astrophysical parameters of our model of Universe, based on these observations. In this work, I first use refined supervised learning algorithms to perform astrophysical parameters inference from a simulated 21-cm power spectrum. I present both a neural network and ridge kernel regression, the latter being our best algorithm and achieving a reconstruction error is less than 1% from an unaltered signal. I then tackle on the problem of accurately predicting the maximum-likelihood values when considering noised signal. For noised input, the lowest theoretical reconstruction error is of the order of 1% which corresponds to 10% of the 1σ confidence level due to the SKA thermal noise (as estimated with Bayesian inference).

In the second part, after identifying the ionized bubbles in the numerical simulation HIRRAH-21 and matching halos and bubbles, we see that the dispersion in the relation between the bubbles volume and the luminosity of their halo can be taken into account using the clumping factor. Finally, I develop a new analytical model linking the bubble size distribution to the physical relation between ionized bubble volumes and their collapsed mass content M (V ). This model was designed to be adaptable to many conditions as the functional form of the physical relation is parametrizable, one can choose to include dispersion in the relation, the numerical sample variance can be considered and the underlying halo mass function formalism can be changed. This model can predict the bubble size distribution with precision given a parametrization of M (V ) but can also be used in reverse. Starting from an observable bubble size distribution, this model can infer the value of the astrophysical parameters defining the M (V ) relation. The inferred values are in good agreement with the value directly deduced from numerical results for low global ionization fraction x H II < 0.03. However, it worsen with increasing global ionization fraction due to the percolation effect. I thus develop a percolation algorithm that is able to apply a percolation effect to an already existing bubble size distribution. This algorithm helps the analytical model to increase its agreement with the numerical bubble until x H II ∼ 0.25.

Résumé

L'Aube Cosmique et l'Époque de la Reionisation sont deux périodes d'importance dans l'histoire de notre Univers, puisque c'est lors de celles-ci que les premières étoiles et les premières galaxies se forment à partir des légères fluctuations de densité dans la brume de gaz primordial. Une fois les premières étoiles formées, celles-ci commencent à rayonner, produisant des photons qui ionisent l'hydrogène avoisinant une fois absorbés. À mesure que l'hydrogène proche est ionisé, le rayonnement d'une étoile peut atteindre des zones de plus en plus éloignées et la bulle de gaz ionisé autour de l'étoile grossit progressivement. Après un certain temps, avec l'augmentation du nombre de sources et l'agrandissement des bulles d'ionisations, ces dernières finissent par se rejoindre et s'unir, phénomène appelé percolation, jusqu'à ce que l'intégralité de l'Univers soit ionisé.

Ces ères s'étant terminées il y a plus de 12 milliards d'années, il est très difficile de les appréhender bien que leur impact soit considérable. L'un des rares traceurs existant, et le plus prometteur de tous, est le signal 21-cm. Ce rayonnement, dont la longueur d'onde est logiquement λ = 21cm, est issu de la transition entre les deux états fondamentaux hyperfins de l'atome d'hydrogène neutre, caractérisés physiquement par un changement de spin de son électron. Isolé, un atome d'hydrogène dans l'état énergétique fondamental hyperfin de plus haute énergie se désexciterait après en moyenne 11 millions d'années ce qui rend cette transition inexistante en pratique sur Terre. Toutefois, à l'échelle de l'Univers dans les périodes reculées observées où le gaz primordial n'est quasiment composé que d'hydrogène et d'hélium, et donc où peu d'autres transitions existent, le signal 21-cm est observable. Il s'observe comme une fluctuation par rapport à une intensité de base issue des photons du fond diffus cosmologique, émis au moment de la recombinaison. Plus précisément, le spectre de corps noir du fond diffus cosmologique émet principalement à plus petites longueurs d'onde mais sa contrepartie dans les longueurs d'onde radios est suffisante pour créer une intensité de référence et entraîner des interactions avec l'hydrogène neutre. Si ces photons traversent un nuage d'hydrogène "froid", dont la plupart des atomes sont dans l'état de plus basse énergie, une partie sera absorbée et on observera une diminution du signal. Au contraire si une région de l'univers possède un grand nombre d'atomes d'hydrogène aux états fondamentaux de plus haute énergie, il est vraisemblable que ceux-ci émettent des photons 21 cm venant s'ajouter à ceux de base faisant augmenter l'intensité par rapport au niveau initial.

Lorsqu'un photon voyage à travers l'Univers durant une période particulièrement longue, il subit l'expansion de celui-ci et sa longueur d'onde intrinsèque augmente. Dès lors un photon émis à une longueur d'onde de 21cm à l'Époque de la Reionisation, subira cette expansion et nous atteindra avec une longueur d'onde entre λ = 1.5 m et 3.5 m. Lorsque leur longueur d'onde s'est suffisamment éloignée de 21cm, les photons n'interagissent plus avec les atomes d'hydrogène et nous parviennent avec une intensité directement liée à l'état de l'Univers au moment où ils ont été émis. En observant les longueurs d'ondes de 21cm à quelques mètres on peut donc avoir accès aux photons du signal 21cm émis depuis l'Aube Cosmique jusqu'à nos jours et qui ont chacun conservés les informations sur l'état de l'Univers lors de leur interaction. Théoriquement, l'observation du signal 21cm peut donc donner accès l'organisation spatiale de l'Univers et son évolution, c'est-à-dire à une tomographie de celui-ci, sur ces 13 derniers milliards d'années.

De par son potentiel détaillé précédemment, il est aisé de comprendre l'intérêt de l'astrophysique pour le signal 21cm dont la cause physique, la transition entre les états fondamentaux de l'hydrogène, a même été dépeinte sur les plaques emportées par les sondes Pioneer et Voyager. Cependant, d'un point de vue pratique, son observation s'avère extrêmement ardue à l'Époque de la Réionisation et est un défi actuellement entrepris par de nombreuses collaborations internationales à travers le monde (MWA, LOFAR, EDGES ...) qui doivent développer de nombreuses méthodologies pour l'extraire d'un signal particulièrement corrompu par d'autres sources. En effet ce signal est en l'état si faible, que le bruit thermique du télescope, issu de la conversion du signal électromagnétique en information électrique, peut suffire à le corrompre entièrement si des mesures ne sont pas prises pour réduire son impact.

En plus de ce bruit, d'autres sources plus proches de nous dans l'Univers, comme les galaxies à noyaux actifs ou le rayonnement synchrotron issu des supernovae de notre galaxie, émettent des signaux dans les mêmes longueurs d'onde que le signal observé. Ces signaux d'avant-plan sont 100.000 fois plus intenses que le signal à observer et le masquent totalement pour une observation brute. Pour pallier à ce problème, les interféromètres actuels mènent des campagnes de cartographie du ciel (LOFAR pour l'hémisphère nord et MWA pour l'hémisphère sud) afin de déterminer très précisément le profil des signaux provenant de chaque avant-plan et donc pouvoir soustraire leur contribution au signal total pour ne garder que le signal 21cm issu de l'Époque de la Réionisation. Au vu du très grand nombre de sources d'avant-plan existantes, il reste difficile de soustraire chaque contribution et le signal résiduel est encore un mélange entre le signal cible et la contribution des sources encore non déterminées. Toutefois, un nouveau télescope en construction, le Square Kilometer Array (SKA) doté d'une sensibilité en moyenne 10 fois supérieure aux télescopes actuels pourrait réussir, sous réserve d'avoir été correctement calibré, à observer le signal total malgré toutes ces contraintes observationnelles. Ce télescope issu d'une collaboration mondiale et basé en Australie et en Afrique du Sud devrait être opérationnel dans quelques années et améliorer significativement notre compréhension de l'Univers en fournissant un flot élevé d'observations de très grande qualité.

Si aucun télescope opérationnel à ce jour n'a encore la sensibilité pour observer assez intensément le signal sur chaque ligne de visée et pouvoir ainsi cartographier l'Univers, il est tout de même possible de déterminer des contraintes sur certaines quantités statistiques, qui décrivent le signal tout en étant plus accessibles car calculées sur un ensemble d'observations plutôt que sur des observations individuelles de chaque ligne de visée. L'accent est actuellement porté sur la contrainte du spectre de puissance, qui est mathématiquement la transformée de Fourier de la fonction d'auto-corrélation du signal et rend compte de l'intensité des fluctuations gaussiennes dans ce dernier. De nombreuses limites supérieures sur l'amplitude du spectre de puissance ont déjà été publiées mais n'ont pas encore atteint son intensité estimée par la plupart des théories, ne permettant de réfuter que les plus extrêmes d'entre elles.

Puisque l'observation du signal 21cm est encore très complexe et qu'il est pourtant nécessaire de mieux le comprendre pour mieux l'isoler dans le signal total, de nombreuses modélisations numériques de l'Univers à l'Époque de la Réionisation ont été développées afin de répondre à ces problématiques. Comme il est absolument impossible de modéliser notre Univers entier à la résolution de chaque atome, ces simulations se basent sur des approximations plus ou moins importantes pour obtenir leurs résultats. Tout d'abord, toutes les simulations ne considèrent qu'une région de taille finie de l'Univers, typiquement une centaine de millions de parsec comobiles, et limitent leur résolution en masse à quelques milliards de masses solaires, soit la taille d'un halo galactiqus dès lors considéré comme une unique "particule" possédant des propriétés rendant compte de la physique interne s'y déroulant.

Certaines simulations, dites semi-numériques, par souci de rapidité de calcul infèrent les conséquences du rayonnement sans faire de transfert radiatif. C'est le cas par exemple de la simulation 21cmFAST (Mesinger et al., 2011) qui se base sur le champ de densité pour prévoir la position des halos, calculer la masse d'étoile formée et leur efficacité d'ionisation, et ainsi déduire la taille de la zone ionisée autour dudit halo. L'avantage majeur de telles simulations est que leur temps de calcul rapide permet de les lancer à de multiples reprises avec des valeurs différentes des paramètres astrophysiques et d'explorer une vaste gamme de possibilité. On pourra ainsi théoriquement isoler la valeur des paramètres qui produisent un résultat ressemblant le plus aux observations et mieux comprendre les effets astrophysiques sous-jacents quantifiés par ces paramètres. Les capacités prédictives des simulations semi-numériques sont toutefois limitées de par l'importance de leurs approximations et il n'est possible que de déduire les grandes tendances v dans l'évolution du signal 21cm et des autres grandeurs. D'autres simulations, dites numériques, plus coûteuses en terme de temps de calcul mais aussi plus détaillées dans leur prise en compte des effets physiques permettent de rendre compte plus précisément de l'évolution des quantités observables, sous réserve que notre compréhension actuelle de l'Univers ne soit pas biaisée. Le code LICORICE (Semelin et al., 2007) développé au sein de notre équipe de recherche fait partie de cette catégorie de simulations et, malgré son optimisation avancée, nécessiterait environ 10 ans de calcul sur un seul coeur. Dans LICORICE, le rayonnement est non seulement pris en compte par la propagation de photons (propagé par paquets et absorbé suivant un champ de densité calculé sur une grille adaptative) mais ceux-ci peuvent également être de plusieurs types : ionisants (UV), Lyman-α et rayons X afin d'être plus proche de la réalité.

Durant la première partie de ma thèse, mon travail s'est porté sur le développement d'une nouvelle méthode de détermination de la valeur des paramètres astrophysiques des modèles basée sur des techniques d'intelligence artificielle comme les réseaux de neurones. Jusqu'à présent la méthode de détermination de la valeur de paramètres astrophysiques à partir d'une observation se basait sur la technique dite Markov Chain Monte Carlo (MCMC). Pour cette technique, une valeur aléatoire du paramètre était fournie à une simulation, souvent semi-numérique, qui produisait une observable pouvant être comparée à la vraie observation. Grâce à cette comparaison il était possible de déduire si la valeur du paramètre était en réalité plus faible ou plus élevée que celle estimée et d'à nouveau lancer une simulation avec une valeur légèrement modifiée. Ce processus devait être répété itérativement pendant des dizaines voir des centaines de milliers de fois avant d'atteindre les distributions des valeurs des paramètres correspondants à l'observation.

Lorsqu'on le sort de son contexte, on remarque que le travail à effectuer ici est de déterminer une caractéristique associée à un objet observé, une tâche pour laquelle les méthodes d'apprentissages supervisée comme les réseaux de neurones sont particulièrement efficaces. Pour être performantes, les méthodes d'apprentissage supervisé nécessitent un nombre conséquent d'exemples où l'observable et les paramètres qui lui sont associés sont connus, formant ce qu'on appelle l'échantillon d'apprentissage. Face aux limitations des simulations numériques en terme de temps de calcul évoquées précédemment, nous avons choisi d'utiliser la simulation seminumérique 21cmFAST afin de créer un échantillon d'apprentissage de 2400 cas. Dans le cadre du développement de cette méthodologie, nous avons fait le choix d'utiliser seulement comme observable le spectre de puissance du signal, puisque celui-ci est plus facilement accessible par les observations, en supposant que l'information contenue dans celui-ci est suffisante pour contraindre les paramètres avec précision. Les trois paramètres astrophysiques ont été choisi parmi les paramètres d'entrée de la simulation et caractérisent l'efficacité d'ionisation des sources, la distance moyenne parcourue par un photon, et la masse minimale des proto-galaxies capables de former efficacement des étoiles. Un autre échantillon, dit de test, de 512 cas pour des valeurs intermédiaires des paramètres a également été produit afin de tester les capacités de prédiction de la méthode sur des cas auxquels elle n'a pas été confrontée lors de son apprentissage.

Après une phase importante de calibration du réseau de neurones, où ont été testés l'architecture, les algorithmes et paramètres d'apprentissages et les fonction de coût, l'erreur de reconstruction de la valeur du paramètre par le réseau est de l'ordre du pourcent, améliorant d'un facteur 10 la qualité de prédiction par rapport au modèle de référence de l'époque. Ce premier résultat plus qu'encourageant m'a conduit à considérer d'autres méthodes d'apprentissage supervisé moins populaires que les réseaux dans l'optique de voir si celles-ci produisaient d'encore meilleurs résultats. En effet, la dimensionnalité plutôt faible de notre observable (le spectre de puissance étant échantillonné pour 12 nombre d'ondes à 10 temps différents) semble tout aussi facilement appréhendable par des méthodes de régressions que par les réseaux de neurones davantage développés pour traiter des images entières composées de plusieurs milliers de pixels et donc plusieurs milliers de dimensions. Après avoir testé plusieurs algorithmes, mon choix s'est porté vi sur la régression regularisée à noyau, combinaisons de plusieurs régression plus connues, méthode dont l'erreur de prédiction s'est révélée inférieure au pourcent et améliorant d'un facteur 5 les prédictions de notre meilleur réseau testé.

Comme expliqué précédemment, l'une des limites majeures dans l'observation du signal 21cm est la présence de bruit dans l'observation. Par l'intermédiaire d'un code, développé pour l'occasion, et émulant le bruit thermique attendu de SKA nous avons donc créé des échantillons de signaux bruités afin d'entraîner nos méthodes à reconstruire la valeur des paramètres astrophysiques malgré une corruption partielle du signal. Comme attendu, et malgré l'application de diverses techniques d'atténuation de l'importance du bruit issues de la radioastronomie, les capacités de prédiction de toutes les méthodes se sont significativement dégradées (d'un ordre de grandeur environ) comparé au cas non-bruité. Le fait que les méthodes ont maintenant la double tâche de débruiter le signal puis de prédire la valeur des paramètres explique cette dégradation. Lorsque le signal n'est pas bruité, celui-ci correspond forcément à une valeur définie des paramètres astrophysiques. En revanche un signal bruité peut avoir pour origine plusieurs couples "signal non bruité + réalisation du bruit" différents, certains (e.g. un signal proche avec un bruit moyen) plus probables que d'autres (e.g. un signal lointain avec une réalisation exceptionnellement intense du bruit). Les méthodes ne pouvant prédire qu'une seule valeur pour les paramètres astrophysiques, il est donc logique de leur faire prédire la valeur correspondant au signal le plus probable et donc de ne composer l'échantillon d'apprentissages que de signaux bruités associés à ces valeurs les plus probables (et non pas n'importe quelle valeur ayant une probabilité non nulle comme précédemment). Une fois cette correction réalisée, l'erreur de prédiction des méthodes a diminué pour atteindre un minimum de 1% d'erreur, logiquement plus grande que dans le cas non-bruité mais quatre fois inférieure à l'amplitude de l'incertitude due au bruit thermique estimé par inférence Bayesienne lors d'une observation de SKA et donc quasiment négligeable par rapport à cette dernière.

Dans un deuxième temps, j'ai entamé une phase d'analyse des causes et conséquences de la dispersion dans les relations physiques liant masse des halos, luminosité et volumes des bulles (mis en exergue par une simulation de particulièrement haute résolution du code LICORICE baptisée HIRRAH-21).Après avoir implémenté une méthode de calcul de la fraction d'échappement des halos, une anti-correlation entre la valeur de cette fraction d'échappement et la luminosité des halos à masse de matière noire constante est apparue. La différence en luminosité et luminosité échappée n'a toutefois pas suffit à diminuer la dispersion dans la relation liant volume des bulles et luminosité des halos. Cette dispersion s'est avérée être due à une différence dans l'environnement local des bulles caractérisée par le clumping factor. En se limitant à un certain intervalle de valeurs de ce clumping factor, la relation liant volume des bulles et luminosité des halos présente une dispersion beaucoup plus faible. Le clumping factor peut donc permettre à des simulations semi-numériques de prendre en compte à moindre coup cette dispersion. Enfin, j'ai développé un modèle théorique paramétrable associant la distribution des tailles des bulles d'ionisation de l'Époque de la Réionisation à la relation astrophysique liant la masse des halos au volume des bulles ionisées les contenant. Ce travail répond à deux objectifs principaux. Tout d'abord, la distribution des tailles de bulles d'ionisation étant une observable accessible à travers la tomographie, il est possible avec un tel modèle de contraindre les lois physiques liant la taille d'une bulle d'ionisation et la masse des halos qu'elle contient à partir d'observations. Ce modèle étant analytique il répond, tout comme le travail précédent, à la problématique de rapidité de traitement de plus en plus cruciale dans les observations actuelles. En complément, dans le cadre de travaux théoriques et de simulations pour lesquelles la relation physique sous-jacente serait supposée, notre modèle, librement accessible à la communauté, permet de produire rapidement une distribution de tailles de bulles cohérente avec la relation vii proposée.

Depuis maintenant plusieurs décennies, la cosmologie théorique a établi un modèle de distribution des masses de halos dans l'Univers, baptisé Halo Mass Function (HMF). Ce modèle a par la suite été étendu en Conditional Mass Function (CMF), des HMF à des densités moyennes différentes des valeurs moyennes de notre Univers. Ces CMF sont utilisées pour décrire des zones particulièrement surdenses, comme par exemple les régions dans lesquels se forment les halos et les étoiles responsables de l'ionisation. En se basant sur ces CMF et en liant à travers une relation physique totalement paramétrable les tailles des bulles d'ionisation aux masses des halos qu'elles contiennent, il a été possible de développer un modèle qui, pour une cosmologie spécifiée, prédit la distribution des tailles de bulles. Ce modèle a été spécifiquement développé pour être adaptable à beaucoup de théories différentes et il est possible d'en modifier la cosmologie, le formalisme de CMF considéré et la forme de la relation physique entre les tailles de bulles et les masses de halos ainsi que de prendre en compte, ou non, les effets des résolutions limitées des simulations.

Afin d'évaluer les capacités prédictives du modèle, nous l'avons comparé aux résultats d'une simulation à haute résolution produite en utilisant le code LICORICE. Cette simulation fournit indépendemment, après traitement, à la fois la distribution de taille des bulles et la relation physique entre leur taille et la masse des halos qu'elles contiennent. Ainsi, on commencera par déterminer la valeur des paramètres physiques pour laquelle la distribution des bulles d'ionisation du modèle théorique est la plus proche de celle de la simulation. On pourra ensuite vérifier si la relation physique théorique correspondant à ces paramètres est également proche de celle de la simulation, comme attendu d'un modèle prédictif. Deux formes différentes ont été choisies pour la relation physique liant volume des bulles ionisées et masse des halos qu'elles contiennent, l'une étant une proportionnalité basique et l'autre une fonction Softplus plus complexe visant à reproduire la forme observée de la relation physique de la simulation. Dans les deux cas la relation théorique inférée par le modèle est plausible mais reste relativement éloignée de la relation de référence de la simulation en ayant tendance à surestimer la masse des halos nécessaire à la formation d'une bulle de taille donnée. Les capacités prédictives du modèle en l'état sont donc limitées à une prédiction de l'ordre de grandeur pertinent plus qu'à une quantification.

Toutefois, l'une des particularité des simulations numériques issues du code LICORICE est la dispersion dans les relations physiques liant les différentes grandeurs. Cette dispersion, à priori attendue dans notre Univers également, s'explique par les différences des environnements locaux entre différentes bulles ou encore par l'histoire d'ionisation de ces même environnement : deux halos de mêmes masse pouvant correspondre à deux bulles de taille différentes si l'un de ces halos s'est formé avant l'autre et a davantage eu le temps d'ioniser son environnement. En ajoutant au modèle numérique la possibilité de prendre en compte une dispersion dans la relation physique par l'intermédiaire d'une marginalisation se basant sur une paramétrisation de ladite dispersion, il a été possible d'améliorer de manière conséquente les résultats. Avec cette dispersion, la distribution de taille de bulle théorique est encore plus à même de reproduire celle de simulation et la relation physique inférée est, dans le cas de la forme fonctionnelle la plus complexe, quasiment identique à la relation trouvée dans la simulation, impliquant cette fois une capacité prédictive importante.

Une autre limite du modèle est son incapacité grandissante à prédire la distribution de taille des bulles à mesure que celles-ci percolent en des formes complexes puis en une bulle unique. Pour pallier à ce problème, un code modélisant le processus de percolation a récemment été ajouté au modèle théorique et montre des signes encourageants à faible fraction d'ionisation globale, modifiant la distribution de taille de bulle en une forme très proche de celle de la simulation et permettant une prédiction des paramètres de la relation physique jusqu'à une fraction d'ionisation moyenne d'environ 25%. Les ajouts successifs de dispersion et de percolation au modèle ont également été inclus de manière à être désactivables dans le cas de théories ne les prenant pas en compte.

Preamble

For clarity purpose, I want to underline what was the contribution of other members of my team to the following study and what was my own work. Although I used the LICORICE code, the specific numerical simulation HIRRAH-21 was run by my supervisor Benoît Semelin. The same goes for the semi-numerical simulations of 21cmFAST that formed the data set for the supervised learning algorithms that were run by Evan Eames during his own studies (Eames et al., 2019). Additionally, the SKA thermal noise emulator used to noise the data set signals was developed by Benoît Semelin and the halo finder was first developed by Florian Bolgar during his own studies.

For my part, I develop all the other numerical and theoretical aspects of this work, under the guidance of my supervisor. More precisely (but not exhaustively) in the parameter inference from simulation, I have implemented and tuned the neural networks and ridge kernel regressions. I have studied the impact of noising the signal on the prediction and developed the theoretical method and the numerical implementation to better handle such noise in the maximum-likelihood prediction. For the dispersion analysis, I have helped in the postprocessing of HIRRAH-21 and adapted the halo finder algorithm to recompute some physical quantities. I have also implemented the algorithm that computes the escape fraction of halos. For the last part of this work centered on the ionized bubble size distribution, I have implemented the postprocessing algorithm that identifies the bubbles and match bubbles and halos. I have built the theoretical model that links the bubble size distribution to the physical relation between bubble volume and their collapsed mass content. I have then implemented this model and studied its performance before implementing the way of using it backward to match a reference distribution in order to predict the physical relation. I have also worked on the percolation routine that can be added to the model and studied its effects.

Contents

The Recombination

During the first hundred thousands years after the Big Bang, matter is too dense and hot to effectively form neutral atoms. The Universe is composed of a hot ionized plasma of protons and electrons that interact with photons before the latter can travel any significant distances. As time passes, the matter starts to cool down and the first neutral helium and hydrogen atoms begin to appear, through a process called recombination. At z ∼ 1000 (around 0.4 Myr after the Big Bang), almost all the hydrogen has undergone recombination. In these new conditions, photons are not scattered off anymore by Thomson scattering and can travel freely over long distances. Radiation coming from this time of last scattering form the Cosmic Microwave Background (CMB), that is the oldest observable electromagnetic signal.

The Dark Ages

After the recombination, the Cosmic Dark Ages begin. They last roughly half a billion year before the first stars and galaxies start to light up the universe. During the Dark Ages, dark matter particles are collapsing into halos, forming gravitational potential wells. As dark matter halos become more massive, only thermal pressure prevents the gas from also collapsing into the halos. For the gas to start collapsing, the dark matter halos mass must reach a level where gravity is able to overcome the thermal pressure. This limit mass where gravity exactly compensates the thermal pressure of the gas is called the Jeans mass and is defined as:

M J = π 5 2 c 3 s 6 (G 3 ρ) 1 2 (1.1)
where ρ is the gas density, G is the gravitational constant and c s its sound speed 1 . Throughout the Dark Ages, the mean gas density is ρ b = ρ c Ω b (1 + z) 3 where ρ c is the critical density of the universe, Ω b is the baryon density parameter and z the redshift. Starting from z t ≈ 136 where T = 2.73K (1 + z t ) = 374K [START_REF] Peebles | Principles of Physical Cosmology[END_REF], the thermal coupling to the CMB blackbody through Compton scattering becomes negligible 2 . The gas thus undergoes an adiabatic cooling and its temperature decreases as a -2 , where a is the expansion factor. When computed at z = 30, M J ∼ 3.2 × 10 4 M and decreases as a -3 2 giving an estimation of the minimum halo mass required for the gas to collapse if streaming velocities are neglected (Barkana & Loeb,

1 cs = γk b T µmp 1/2
where γ is the adiabatic index, k b is the Boltzmann constant, T is the gas temperature, µ is the mass per particle and mp the proton mass.

2 For zt 136 the temperature of the coupled gas decreases as a -1 .

CHAPTER 1. FUNDAMENTALS OF THE EPOCH OF REIONIZATION 2001). Streaming velocity is a relative velocity between the gas, that was previously coupled to radiation before the recombination, and the dark matter that was not (Tseliakhovich & Hirata, 2010). Taking this velocity into account creates spatial fluctuations of the minimum mass for halos that are able to bind gas. Overall, the first stars of the universe appeared at z ∼ 20 -30 at last ending the Dark Ages after one hundred million years (Barkana & Loeb, 2001;[START_REF] Glover | The First Galaxies: Theoretical Predictions and Observational Clues[END_REF]Schauer et al., 2021).

1.1.3 Let there be light : the Cosmic Dawn

The first stars

The very first stars to form were mostly composed of hydrogen and helium atoms produced by the primordial nucleosynthesis. As they were the first to create heavier elements through nucleosynthesis, the gas they are formed from is metal-free and is cooling less efficiently than with metals, leading to the creation of stars more massive than nowadays stars (Bromm et al., 2001;Abel, 2002). Indeed, the atomic cooling mechanism allows the gas temperature to decrease until T∼ 10 4 K. With only this cooling mechanism, thermal pressure is compensated by gravity only if the mass of the halos is larger than 10 8 M . For the first stars to form in regions of mass M < 10 8 M , another cooling mechanism has to decrease the temperature below 10 4 K. It is the molecular cooling that is induced by the existence of H 2 molecules, following the reaction:

H + e -→ H -+ γ H + H -→ H 2 + e -.

(1.2)

In this reaction, the intermediary product H -can easily reverse into H because its extra electron has a ionization potential of 0.76 eV. Furthermore, H 2 can be dissociated by soft UV wavelengths between 11.1 and 13.6eV (Field et al., 1966;Stecher & Williams, 1967). It is not a problem at the very beginning of the formation as there is no star so no background radiation in UV nor infrared. Rapidly though, the radiation emitted by other stars creates a negative feedback of H 2 formation leading to an increase of the minimum host halo mass to form stars (Hirano et al., 2015). Considering this radiation feedback, simulations find that, until the end of reionization, metal-free stars form at an almost constant rate of around 3 × 10 -5 M yr -1 Mpc -3 (Wise et al., 2012).

Believed to be more massive than present day stars (Bromm et al., 1999;Abel, 2002;Susa et al., 2014;Hirano et al., 2014), though still in debate (Stacy & Bromm, 2013, 2014;Wollenberg et al., 2020), metal-free stars create a significant amount of radiation (∼ 10 49 hydrogen ionizing photons per second) during their lifetime of about 3 to 10 Myr (Schaerer, 2002). Eventually, the first stars explode in supernovae at the end of their lives, enriching their surroundings with chemical elements on a scale of 10-100 ckpc. The formation of other stars is also locally stopped because the supernovae are expelling most of the gas from their host halos. Later on, stars will include some metal elements and be of a different type than the first stars.

The first galaxies

As time passes, dark matter halos continue to grow into heavier ones where not only stars but also larger structures can form. Stars gather into clusters, which eventually turn into protogalaxies in which the atomic line cooling is efficient. However, the future galaxy-hosting halos are often depleted of their gas because of previous explosion of supernovae within their progenitors delaying the star formation process. Once the halo mass exceeds the atomic cooling limit, stars are believed to form at a continuous specific star formation rate of the order of sSF R ≡ Ṁ M ∼ 10 -8 yr -1 (where M is the halo stellar mass) that can vary from one galaxy to another depending on the feedback from supernova explosion during the history of formation (Kimm & Cen, 2014;Kimm et al., 2017). Some simulations shows that the first galaxies contained between 10 6 and 10 7 M of metal-poor stars, when their host halos reached as dark matter mass of 10 9 M (Xu et al., 2016).

The Epoch of Reionization (EoR)

Photo-ionization

The Epoch of Reionization begins between 0.10 and 0.20 Gyr after the Big Bang (between z = 30 and 20) as radiation emitted by the first sources begins to ionize their surrounding. One of the ruling quantities of this process is the average distance that ionizing photons (E>E th = 13.6 eV ⇔ λ < λ th = 91.15 nm) can travel before being absorbed, also known as the photon mean free path. The mean free path in the neutral intergalactic medium (IGM) of a ionizing photon can be expressed as:

l = 1 σ(λ)n H (z) (1.3)
where n H (z) is the average hydrogen number density in the universe at redshift z and σ(λ) is the hydrogen cross-section for a photo-ionizing photon of wavelength λ. This cross-section can be computed through a comparison with the reference cross-section σ L = 6.3 × 10 -18 cm 2 for a photon at the ionizing threshold of wavelength λ th through σ(λ) = σ th ( λ λ th ) 3 (Verner et al., 1996;Madau, 2000). Using Planck Collaboration (2016) cosmology we can estimate the mean free path in the neutral IGM at the average universe density to be l ∼ 0.2 ckpc at z = 20 and l ∼ 1.7 ckpc at z = 10. Compared to the typical halo scales of 100 ckpc, we can see that ionizing photons cannot travel significantly through neutral regions and are typically absorbed by neutral hydrogen atom at the boundary of the already ionized regions. The ionization process thus takes the shape of an ionized mostly spherical region, often called ionization bubble, starting at CHAPTER 1. FUNDAMENTALS OF THE EPOCH OF REIONIZATION the source and growing progressively while keeping a sharp ionization front beyond which the IGM is highly neutral.

It is worth noting that the ionizing radiation possesses a negative feedback effect. Indeed once ionized, the gas inside the halo will not be able to cool down as efficiently as before. The cooling being done through atomic cooling, the decrease of neutral hydrogen content will also decrease the number of collisional excitations (Wiersma et al., 2009). Furthermore, ionizing radiations also heat up the gas above 10 4 K, effectively photo-evaporating proto-haloes of virial temperature below 10 4 K. This prevents the formation of stars that would have contributed to ionization.

Recombination

To model the reionisation process, it is necessary to consider another effect balancing the photoionisation: recombination. It occurs at temperatures T 10 4 K when ionized hydrogen (a proton) and a free electron are attracted toward each other through the Coulomb force, turning back the hydrogen into its neutral state and emitting a photon3 . This effect is quantified by the recombination time t rec per hydrogen atom:

t rec (T, z) = (α A (T )n H (z)) -1 (1.4)
where α A (T ) is the case A recombination coefficient of ionized hydrogen [START_REF] Spitzer | Physical Processes in the Interstellar Medium[END_REF]Pequignot et al., 1991;McQuinn, 2012). The case A recombination coefficient is the sum of recombination rates into all electronic levels. However direct recombination into ground level n = 1 produces a new ionizing photon. If we make the assumption that this photon immediately ionizes a surrounding hydrogen atom (a scenario named case B recombination), a second coefficient that ignores this recombination into ground level can be defined. The case B recombination coefficient is mainly used in numerical simulations to speed up computations and approximately increases the recombination time by a factor of 2. For T=10 4 K and z = 10 we obtain t rec ∼ 350 Myr, a time much longer than the time scale on which the luminosity of the galaxies increases significantly (a few 10 Myr), meaning that the recombination in the mean density IGM will not stall the ionization front efficiently.

The previous conclusion is only valid for regions at mean density. However, there exist small neutral clumps in the IGM, also known as Lyman limit systems, that have a significantly higher number density and thus a significantly higher recombination rate. These systems can slow down the ionization process by being small neutral islands inside the ionization bubbles and absorbing a fair amount of photons, effectively decreasing their mean free path (McQuinn et al., 2011;Haardt & Madau, 2012).

Ionization evolution

Local description

To accurately model the local evolution of the ionization fraction, we need to take into account one last effect: collisional ionization. During the collision between a free electron and a neutral hydrogen atom, this atom may be ionized especially if the electron momentum is high. For a volume where the physical quantities (i.e. density and temperature) can be considered constant, the evolution of the ionization fraction can be expressed as [START_REF] Spitzer | Physical Processes in the Interstellar Medium[END_REF]Cen, 1992;[START_REF] Maselli | [END_REF] :

dx H II dt = Γx H I + γ(T )x H I n e -α A (T )x H II n e (1.5)
where x H I and x H II are respectively the local neutral and ionized fraction of hydrogen, γ(T ) is the collisional ionization rate and Γ the photo-ionization rate4 .

The simple Strömgren sphere

At a larger scale, the evolution of the ionized region around a source is mainly ruled by the two competing effects of photo-ionization and recombination. Assuming a constant number of ionizing photons emitted per second Ṅγ by the source, the equilibrium is reached when recombination in the ionized region balances the photon production rate:

α B (T )n 2 H V S = Ṅγ (1.6)
where V S is the volume of the ionized region. We can thus deduce that the ionization front around this source stalls at a radius R S , called Strömgren radius [START_REF] Strömgren | Ergebnisse der exakten naturwissenschaften[END_REF], given by :

R S = 3 Ṅγ 4πα B (T )n 2 H 1 3
.

(1.7)

The Strömgren radius gives a good insight in the volume a single source can ionize. However, due to the expansion of the Universe, the average hydrogen number density (and therefore the recombination rate) decreases with time, questioning the relevance of defining this radius R S (Shapiro & Giroux, 1987). Furthermore, in practice, sources luminosity fluctuates on a much shorter time-scale than the time-scale required for R to converge to R S .

Global ionization model

To compute the spatially averaged ionization history, one has to rely on models and use parameters that encompass and average underlying physical processes. A general model for the evolution of the global ionized fraction x H II can be written (Furlanetto et al., 2006):

dx H II dt = ζ(z) df coll dt -α A C(z, x H II )x H II n e (1.8)
where f coll is the collapse fraction of the region, ζ(z) is the ionizing efficiency, C(z, x H II ) is the clumping factor that accounts for the effective impact of Lyman limit system and n e is the average electron density in the ionized region. On the right hand side of equation 1.8, the first term accounts for the photon production by the sources whereas the second term accounts for the recombination. This global model is thus consistent with the local physical description which balance photo-ionization and recombination effects. The ionizing efficiency ζ(z) encapsulates the multiple effects that affect the source ability to ionize the IGM. It can be further divided as following:

ζ(z) = A He N ion f * f esc .
(1.9)

In Equation 1.9 A He is a conversion factor turning the number of ionizing photons per baryon in stars into the number of hydrogen atoms they can ionize and is required because of the presence of helium atoms. The next two terms are N ion the mean number density of ionizing photons produced by stellar baryons and f * the star formation efficiency. Finally f esc accounts for the fraction of ionizing photons that effectively escape into the IGM without being absorbed by the overdense medium where the sources are formed. The escape fraction is one of the most challenging parameters to estimate, though many observational (Wyithe et al., 2010;Nestor et al., 2013;Mostardi et al., 2015;Smith et al., 2018) and numerical attempts have been made (Bolton & Haehnelt, 2007;Yajima et al., 2011;Wise et al., 2014;Paardekooper et al., 2015;Sun & Furlanetto, 2016;Price et al., 2016;Doussot et al., 2018) and seem to agree on f esc ∼ 1 -10%. A majority of studies also support the fact that f esc seems to decrease with increasing halo mass with an important dispersion for small halos (Trebitsch et al., 2017;Rosdahl et al., 2018) that is currently explained by a time variability due to starburst effects.

Observational probes of the EoR

Quasars (QSOs) Spectra : Gunn-Peterson effect

The Lyman-α line comes from the transition between the ground state of the hydrogen atom and its lowest excited state and corresponds to a wavelength of λ α ∼ 121.5 nm. Only neutral hydrogen can undergo such a transition and therefore absorption/emission features around λ α wavelength compared to a base signal are indicators of the existence of neutral hydrogen clouds along the line of sight. Quasars are known to emit a continuum signal at wavelengths of the same order of magnitude than λ α . However, when looking at the flux coming from high redshift (z 6) quasars, we can see a strong absorption pattern blueward of the Lyman-α (but redward of the Lyman-β) rest-frame wavelength 5 that is increasingly important with redshift (Fan et al., 2006;McGreer et al., 2015). An example of such a quasar spectrum is shown in Figure 1.3. Ideally with this absorption feature, one could access the position of neutral hydrogen clouds and reconstruct the local neutral fraction along the line of sight. The main issue that prevents this reconstruction is the very high optical depth at Lyman-α wavelength. Starting from x H I > 10 -4 the IGM is optically thick to Lyman-α photons meaning a total absorption and limiting the Figure 1.3: Spectrum of the quasar ULAS J1319+0959 at z = 6.13 (Becker et al., 2015b) illustrating the Lyman-α forest and the Gunn-Peterson effect reviewed in this section 1.3. Taken from Becker et al. (2015a). reconstruction to the very end of the EoR.

However, two effects enable more in depth observations of the EoR. First, due to the radiations from the QSO, hydrogen in their direct vicinity is believed to be more ionized compared to the typical IGM. It weakens the absorption just blueward of Lyman-alpha (that is near the QSO), giving insights on the size of the ionized region around the quasars. Moreover, the Lyman-α line has a given width and photons can be absorbed in the line wing but with ever lower probability. Even with such a low probability, for a large enough neutral column density (N H I 10 20 cm -2 ), the absorption feature will be observable in the wing and will saturate only for much higher neutral fraction values. This effect has for example allowed to place a constraint of x H II ∼ 0.6 at z = 7.08 (Greig et al., 2017). Furthermore, though lower redshift "foregrounds" have to be removed, the same effects can also be found for the Lyman-β and Lyman-γ lines with lower optical depth so higher hydrogen density needed before saturation.

The Lyman-α forest

The Lyman-α forest describes the multiple absorption features in the QSO spectra blueward of Lyman-α revealing the presence of neutral clouds in the IGM along the line of sight. These absorption lines become more abundant with redshift to finally merge into the Gunn-Peterson trough for redshift z 6. Being a "post-reionisation version" of the Gunn-Peterson trough, the Lyman-α forest cannot be used to directly constrain the EoR and is rather used to quantify the patchy remnant neutral fraction after the EoR with a lower limit of x H II > 0.94 at z = 5.9 (Mesinger, 2010;Greig & Mesinger, 2017a). An example of Lyman-α forest can be seen in the QSO spectra of figure 1.3.

The Lyman-α forest is due to neutral hydrogen clouds with low column densities (N H I ∼ 10 12-16 cm -2 ). Such clouds are particularly sensitive to photo-ionization and photo-heating from the UV background. Thus, the Lyman-α forest can be used to compute the ionization rate and temperature after the EoR and gain insights about their behaviour during the reionization. Surveys using the Lyman-α forest (York et al., 2000;Becker & Bolton, 2013) show that the ionization rate, constant from z ∼ 2 to z ∼ 5, starts to sharply increase at larger redshifts. Concerning the IGM temperature, studies of the Lyman-α forest (Schaye et al., 2000;Becker & Bolton, 2013) lead to a typical temperature of ∼ 10 4 K at z = 5.

CMB polarisation by Thompson scattering

The CMB photons, emitted from the surface of last scattering, have traveled through the Epoch of Reionization to reach the observer. During their travel, these photons have a probability of ∼ 10% to scatter off free electrons in the IGM at mean density. This scattering, called Thompson scattering, results in a polarization of the observed CMB at large angular scales. The Thompson optical depth τ associated to this effect is directly related to the column density of free electrons and thus to the average reionization history following:

τ = σ T n e dl
(1.10)

where σ T = 6.65 × 10 -25 cm 2 is the Thompson scattering cross-section and n e is the average free electron number density. For a fully ionized IGM from z = 0 to z = 6, τ ∼ 0.04. The recent measurement of τ = 0.054 ± 0.007 (Planck Collaboration et al., 2020) implies that the contribution of the EoR to the total optical depth is ∼ 0.014. It can therefore loosely be linked (assuming a simple functional form for the average ionization history) to the redshift where the Universe was half ionized, that is z re = 7.7 ± 0.7 for this measurement of τ .

Miscellaneous other paths

Apart from the previously detailed probes and the significant 21-cm signal that will be detailed in the next sections, multiple other attempts to constrain the EoR through new observational probes have been made, among them are:

• Gamma ray bursts (GRB) : the unabsorbed spectrum of these sources around λ α is a simple power law, so Gunn-Peterson effect and Lyman-α wing absorption are observable on such sources when they are observed at high redshift (Ciardi & Loeb, 2000;McQuinn et al., 2008). Currently, GRBs have mainly been observed at redshifts z < 6 but observations at higher redshifts are expected in the near future (Tanvir et al., 2019;Lidz et al., 2021).

• Fast radio bursts : are recently discovered extra-galactic radio transient signals.

During their travel between their sources and the observer, these signals are dispersed by the ionized medium. The ionized state of the universe can thus be revealed through the study of this dispersion (Fialkov & Loeb, 2016). Some observations of fast radio bursts have already been used to constrain the history of hydrogen reionization and the reionization optical depth (see section 1.4.1 for a definition) [START_REF] Heimersheim | [END_REF].

• Deuterium line : its transition is at wavelength λ = 91.6 cm but Deuterium is believed to be extremely rare in the IGM (∼ 10 5 times less abundant than hydrogen), making its observations for the EoR even more challenging than the 21-cm line and this probe is only theoretical for now (Stancil et al., 1998;Furlanetto et al., 2006).

• 3 He + transition : like the deuterium and according to theory, 3 He has only been sparsely produced during the Big Bang nucleosynthesis but its line might be detected through its anti-correlation with HI though no constrains have been established through this means for now (Furlanetto et al., 2006).

• Carbon ion CII : already observable at lower redshift, the EoR might be reached by crosscorrelating target galaxy surveys from ALMA and constraints over the global properties of galaxies over large cosmic volumes from the upcoming Fred Young Submillimeter Telescope (FYST) (Padmanabhan et al., 2021). To this day though, studies on real data have yet to be performed.

• Hydrogen Deuteride (HD) : though not yet observable for the EoR, it might be reachable through cross-correlations within future CII intensity mapping surveys. Unfortunately, such signal is out of reach for the current telescopes (Breysse et al., 2021).

• Neutral oxygen : statistics of neutral oxygen absorbers might help to trace diffuse gas far from the self-shielding halos and first ionized during the EoR process. This method might particularly be helpful for x H II < 0.9 (Doughty & Finlator, 2019). Current observations are however limited to redshift z 6.5 (Becker et al., 2019) and no constraints on the reionization process based on this method have been released.

• 21-cm forest : the physic of the 21-cm signal will be extensively detailed in section 1.4.

The main difference between this signal and the 21-cm forest is the background radiation that enables their observation. While what is commonly defined as the 21-cm signal is seen as a variation compared to the CMB background radiation, the 21-cm forest is seen as a variation of the spectrum of a point source such as a radio-loud quasar or a gammaray burst afterglow. Such a spectrum presents absorption features at the local 21-cm wavelength of the small-scale structures of the IGM. This absorption can be modulated on large scales by Wouthuysen-Field coupling and inhomogeneous heating. The continuous and fluctuating absorption level of the spectrum thus provides information about the local state of the IGM (Semelin, 2016). Though no measurement of the 21-cm forest at the EoR has been made, the current LOFAR telescope (see section 2.3.2) is believed to be able to perform it (Ciardi et al., 2013). The 21-cm signal is a signal of paramount importance for the observation of the Epoch of Reionization. It comes from the hyperfine transition between the singlet and triplet states of hydrogen in its ground electronic state. Its rest frame frequency is 1420.4057 MHz which corresponds to a wavelength of 21 cm. The excited triplet state has a vacuum lifetime of around 11 Myr, meaning that the dominant way to decay is through interaction with the environment, either due to the local radiation field or to atomic collisions. Consequently, the 21-cm signal is mainly observed as a fluctuation of the brightness of the CMB, whose photons can be absorbed or stimulate emissions. The intensity of this signal is regulated by the relative population of neutral hydrogen in the two hyperfine ground states which depends on many physical processes that can be constrained through this dependency. Its measurement can therefore unveil information about the thermal state and density of the high-redshift intergalactic medium. Above all, photons being redshifted during their propagation, the spectral distribution of the observed signal thus conveys information about the spatial distribution of hydrogen clouds and their properties along the line of sight, theoretically enabling tomography, i.e. a 3D reconstruction of our universe during the EoR.

21-cm signal

Figure 1.5: Schematic representation of the 21cm hyperfine transition between the singlet and triplet states of hydrogen in its ground electronic state. This transition corresponds to a change of spin of the electron.

Optical Depth

Physically, the optical depth characterizes the absorption of the medium along the line of sight for the considered wavelength. For a redshifted signal observed at frequency ν 0 , the optical depth of absorption along the line of sight, corrected from stimulated emission, up to redshift z is : 11) where n 0 and n 1 are the local number density of hydrogen atoms in each of the hyperfine ground states, σ 01 and σ 10 are the cross-sections for absorption and stimulated emission, φ (ν) is the normalized line profile (peaked around 1420.4057MHz for the 21-cm signal) and ν is the local rest frame frequency along the line of sight that can be related to ν 0 following:

τ ν 0 (z) = φ(ν) (n 0 σ 01 -n 1 σ 10 ) dl (1.
ν 0 = ν 1 + z 1 + v c (1.12)
with v the local proper velocity along the line of sight.

For convenience, it is a common practice to define a temperature to quantify the hydrogen population ratio between the two hyperfine ground states, called spin temperature T S (Furlanetto et al., 2006;Pritchard & Loeb, 2012) and defined as: 13) where E 10 = 5, 9 × 10 -6 eV is the energy gap between the two ground states, g 0 = 1 is the multiplicity of the singlet state and g 1 = 3 is the multiplicity of the triplet state. Considering

n 1 n 0 = g 1 g 0 exp - E 10 k b T S (1.
1.4. 21-CM SIGNAL that σ i = hνB i 4π
, where the B i are the Einstein coefficients, the optical depth of 1.13 can be rewritten:

τ ν 0 = φ(ν)n 0 σ 01 1 - B 10 g 1 B 01 g 0 e - E 10 k b T S dl = φ(ν)n 0 σ 01 1 -e - E 10 k b T S dl = φ(ν)n 0 3c 2 A 10 8πν 2 1 -e - E 10 k b T S dl (1.14)
using B 10 B 01 = 1 3 and σ 01 ≡ 3c 2 A 10 8πν 2 with A 10 = 2, 85×10 -15 the spontaneous emission coefficient of the 21-cm line. Using Equation 1.12 and assuming that v c we can relate dl to the redshifted frequency through:

dl = dl dν dν =   ν 0 H(z ) ca(1 + v c ) + ν 0 (1 + z) dv dl c(1 + v c )   -1 dν = c H(z )(1 + z )ν 0 1 + 1 H(z ) dv dl -1 dν . (1.15)
During the EoR H(z) ∼ 10 3 km.s -1 .Mpc -1 implying that the frequency width of the 21-cm line is equivalent to a length of a few kpc along the line of sight. Physical characteristics of the IGM, like number densities or proper velocities, can be considered as constant on such small scales. In addition, in an astrophysical context k b T S E 10 and n HI = n 1 + n 0 ∼ 4n 0 . The optical depth can therefore be written:

τ ν 0 = n 0 3c 2 A 10 8πν 2 1 -e - E 10 k b T S c H(z)(1 + z)ν 0 1 + 1 H(z) dv dl -1 φ(ν)dν =1 ≈ n 0 3c 2 A 10 8πν 2 E 10 k b T S c H(z)(1 + z)ν 0 1 + 1 H(z) dv dl -1 ≈ 3 32π c 3 A 10 E 10 n HI ν 2 k b T S H(z)(1 + z)ν 0 1 + 1 H(z) dv dl -1 . (1.16)
Considering that E 10 = hν and ν 0 (1 + z) = ν, one can finally express the optical depth as Furlanetto et al. (2006):

τ ν 0 ≈ 3 32π hc 3 A 10 n HI ν 2 k b T S H(z) 1 + 1 H(z) dv dl -1
.

(1.17)

Brightness Temperature Variation: an observable signal

In term of radiative transfer, the elementary quantity that represents the energy carried by radiation traveling along a given direction, per unit area, solid angle, frequency and time is the specific intensity I ν where ν is the frequency of this ray. For convenience, we usually express this specific intensity in term of brightness temperature T b which is the temperature that a blackbody with spectrum B ν must have such that I ν = B ν (T b ). In a general case the spectrum B ν is a Planck function but in our current case, the observed frequencies and temperatures are such that hν k B T allowing us to use the Rayleigh-Jeans approximate formulas:

T b (ν) ≈ I ν c 2 2k B ν 2 (1.18)
where c is the speed of light and k B is Boltzmann's constant. At cosmological scales, the cosmological redshift implies that the emergent temperature T b (ν) for a given frequency ν measured in a cloud comoving frame at redshift z will be seen at the Earth as a brightness temperature

T b (ν 0 ) = T b (ν)
1+z with ν 0 the observed frequency (expressed in equation 1.12). Thus, the brightness temperature of the CMB in a comoving frame at redshift z will be: T CMB (z) = 2.73 (1 + z)K where 2.73K is the presently observed value T CMB (0).

To get the 21-cm signal variation compared to an unaltered signal directly coming from the CMB, we need to compute the total intensity received by the observer at a frequency ν 0 . We then sum the attenuated contribution from all sources along the line of sight. z ) .

I(ν 0 ) = I CMB e -τν 0 + dI HI e -τν 0 (
( 1.19) This intensity can thus be related to a brightness temperature T B through equation 1.18. Using Rayleigh-Jeans approximation and writing all temperatures at the redshift z where the 1420MHz emission is redshifted at a frequency ν 0 , the previous equation can be written (Pritchard & Loeb, 2012):

T B = T CMB e -τν 0 + T S 1 -e -τν 0 (1.20)
where T S is the spin temperature defined in equation 1. 13. During the EoR, the intergalactic medium is, at average density, optically thin (τ ν 0 1) which allows us to simplify the expression for T B ∼ T CMB (1 -τ ν 0 ) + T S τ ν 0 . Let us define δT B as the variation of the 21-cm brightness temperature compared to an unaltered signal directly coming from the CMB. By definition T B = T CMB + δT B . Expressing δT B in the restframe of the observer 6 and using Equation 1.17 we get (Furlanetto et al., 2006;Pritchard & Loeb, 2012;Mellema et al., 2013):

δT B = T S -T CMB 1 + z τ ν 0 = 3 32π hc 3 A 10 n HI ν 2 k b H(z)(1 + z) T S -T CMB T S 1 + 1 H(z) dv dl -1
.

(1.21)

As expected, the brightness temperature variations are related to the density ratio between the singlet and triplet states of the hydrogen through the spin temperature T S . Intuitively, this equation is better understood by looking at the extreme scenario. If all hydrogen is in the singlet state, the hydrogen can only be excited and there will be a significant absorption of the 21-cm photons from the CMB. Thus, using equation 1. 13,

n 1 n 0 1 ⇔ T S 1K so T S T CMB ∼ 2.7 × (1 + z)K.
Following equation 1.21, we obtain δT B < 0 meaning that there will be a decrease of the 21-cm signal amplitude. Equation 1.21 can be further transformed (using H(z) ∼ H 0 Ω 0.5 M (1 + z) 3/2 ) and expressed in a more convenient way (Semelin et al., 2017):

δT B = 27(1-x H II )(1+δ) T S -T CMB T S 1 + 1 H(z) dv dl -1 1 + z 10 0.5 Ω b h 0.044 × 0.7 0.27 Ω m 0.5 mK.
(1.22) 6 The general form is: T B,obs (ν obs ) = T B,emis (ν emis ) 1+z

1.4. 21-CM SIGNAL

Spin Temperature T S

When determining the 21-cm signal, an important quantity of Equation 1.20 is the spin temperature T S introduced in Equation 1.13 that quantifies the relative populations of the two hyperfine levels of the hydrogen ground state. If the only governing effect on T S was the CMB radiation, the spin temperature would have quickly converged to T CMB , effectively making the 21-cm signal unobservable. Fortunately, two other effects impact the population ratio and rule the spin temperature T S fluctuations7 : the Lyman-α pumping also called the Wouthuysen-Field mechanism and particle collisions.

Influence of particle collision

In the early intergalactic medium, hydrogen atoms can collide with various species like free electrons, free protons, neutral helium, ionized helium, deuterium and logically other hydrogen atoms. Such collisions can induce a spin exchange and we thus need to define the rate (per atom) of excitation C 01 and de-excitation C 10 . Each species contribute to these rates by respectively n i κ i 01 and n i κ i 10 where n i is the number density of species i, κ i 01 its rate coefficient for hydrogen spin excitation in collisions with this species and κ i 10 its de-excitation counterpart. This coefficients are computed and fit using quantum mechanical considerations (Smith, 1966;Zygelman, 2005;Kuhlen et al., 2006;Furlanetto & Furlanetto, 2007a,b).

Figure 1.6: Rate coefficients for spin exchange for : H-H, H-e -and H-H + collisions as function of the temperature. Adapted from Smith (1966) Among the possible interacting species, low-abundance species like deuterium are believed to contribute little to the spin temperature T S as their low abundance cannot be compensated by their rate coefficients even if the latter have been shown to be large. Though more abundant, neutral helium does not contribute to T S either because of the Pauli exclusion principle which prevents it from exchanging spin when in its ground state. Let alone the ionized Helium, whose rate coefficient for spin exchange has not been computed yet 8 , the last three interacting species are free electrons, free protons (i.e. ionized hydrogen) and neutral hydrogen. Figure 1.6 shows the three species rate coefficients for spin exchange as function of the kinetic temperature based on Smith (1966) work. Before the Epoch of Reionization, the ionized fraction of the intergalactic medium was around 10 -4 , meaning that neutral hydrogen was far more abundant than free protons or electrons, to the extent that it widely compensates its lower rate coefficient for spin exchange compared to the other species. Consequently, for low ionization fraction, the collisional contribution to the spin temperature T S is dominated by hydrogen-hydrogen collisions. For higher ionization fraction x H II 10 -2 , the contribution of the two other types of collisions might be comparable to the hydrogen-hydrogen collision but in such conditions the whole collisional contribution is negligible compared to the Lyman-α pumping effect.

Influence of Lyman-α pumping

A Lyman-α photon excites the hydrogen atoms from its ground state up to its 2p state when absorbed. When the Lyman-α photon is re-emitted, the hydrogen atom can go back to either of its two hyperfine ground state when both transitions are allowed from the excited state as illustrated in Figure 1.7. This phenomenon, called Lyman-α pumping or Wouthuysen-Field mechanism (Wouthuysen, 1952;Field, 1958), allows the hydrogen atom to change its spin.

Figure 1.7: Allowed transitions between the 2p and 1s states of the hydrogen atom with transition that contribute to spin exchange (plain) and transitions that do not contribute (dash). Taken from Pritchard & Furlanetto (2006) To include this effect into T S computation, we define P 01 and P 10 which are respectively the excitation and de-excitation rates through this mechanism. This rate encompasses many physical processes related to the Lyman-α emission and many studies have been performed to better understand them (Meiksin, 2000;Pritchard & Furlanetto, 2006;Chuzhoy & Shapiro, 2006;Hirata, 2006).

Spin Temperature computation

Considering CMB radiation, Lyman-α pumping and collisions, and assuming that all of these mechanisms are effective on time scales shorter than the growth of density fluctuation or local variation of the Lyman-α flux so that the hyperfine level populations are in equilibrium, the spin temperature T S is determined by: n 1 (A 10 + I CMB B 10 + P 10 + C 10 ) = n 0 (I CMB B 01 + P 01 + C 01 ) .

(1.23)

In the same way as for T S in Equation 1.13, it is common practice to relate the gas kinetic temperature T K (Field, 1958;Furlanetto et al., 2006;[START_REF] Furlanetto | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF] and define the effective color temperature of the UV radiation field T c as:

C 01 C 10 = g 1 g 0 exp E 10 T K k b ∼ 3 1 - E 10 T K k b and P 01 P 10 ≡ 3 1 - E 10 T c k b . (1.24)
Finally, using Equations 1. 13, 1.18, 1.23, 1.24 and the fact that A 10 = 2hν 3 c 2 B 10 , we can express the spin temperature as:

T -1 S = T -1 CMB + x α T -1 c + x c T -1 K 1 + x α + x c (1.25)
where x α and x c are coupling coefficients for UV scattering and collisions respectively and are defined as :

x α = P 10 E 10 k b A 10 T CMB and x c = C 10 E 10 k b A 10 T CMB .
(1.26) Hz -1 sr -1 ) and the X-ray heating rate (in units of eV s -1 baryons -1 ) at the minimum point. grey curves indicates cases with τ > 0.09, and a non-excluded f X =0 case in black; these cases are all excluded from the colour bar range. Taken from Cohen et al. (2018).

Temperatures evolution

Using Equations 1.21 and 1.25 we can theoretically evaluate the 21-cm signal and determine the various periods when this signal is dominated by one of the underlying quantities rather than the others (Pritchard & Furlanetto, 2007;[START_REF] Weltman | [END_REF].

1. Collision dominated, gas-radiation coupling regime : after the recombination and until z ∼ 200 the IGM is dense enough so that collisional coupling is dominant. The gas temperature still being coupled to the CMB temperature, we have T S = T K = T CMB and consequently δT B = 0.

2. Collision dominated, gas-radiation decoupling regime : for 200 z 40 the gas starts to decouple from the CMB and cools adiabatically as (1 + z) -2 while collisions still dominate meaning that T S ∼ T K < T CMB and consequently δT B < 0.

3. Lyman-α pumping dominated regime : the expansion process has sufficiently decreased the gas density so that collisional coupling fails and the spin temperature decouples from T K . At the same time, absorption from CMB photons implies T S → T CMB and δT B → 0. However, as Cosmic Dawn begins, the first stars form and heat the gas progressively recoupling the spin temperature to the gas temperature, through the Lyman-α pumping mechanism, meaning that T K < T S < T CMB and δT B < 0.

4. X-ray heating regime : during this period, X-ray photons start to propagate. Depending of the astrophysical model (i.e. the value of the astrophysical parameters or the possible approximations) the Lyman-α pumping mechanism might saturates leading to T K ∼ T S . Eventually though X-rays heat the entire IGM implying that T S ∼ T K > T CMB and consequently δT B > 0.

5. The Epoch of Reionization : ionizing photons are emitted from halos and start ionizing the IGM leading to n HI → 0. During this period δT B fluctuations are dominated by the ionization fraction x H II modifications rather than by the spin temperature. In the end of the EoR n HI ∼ 0 effectively canceling the 21-cm signal.

As illustrated by figure 1.8 that shows the global 21-cm signal as a function of redshift for 193 different astrophysical models, the redshift at which each regime start as well as their duration strongly depend on the model.

Observational challenge

Radio interferometry basics

Why interferometry ?

For a telescope, the physical effect that intrinsically limits the angular resolution is diffraction. Considering diffraction through a circular aperture, one can compute the maximum angular resolution θ of a telescope as :

θ 3.5 × 10 3 λ D arcmin (2.1)
where D is the diameter of the telescope and λ is the observed wavelength. Thus, a telescope observing the 21-cm signal during the EoR λ = 2.1 × 10 -1 (1 + z) ∼ 2.1 m with an angular resolution of θ ∼ 1 arcmin 1 should have a diameter of D ∼ 7 km. Such a large structure is unfeasible and there is consequently no way to precisely observe the 21-cm signal with this method.

Fortunately, it appears that using multiples small antennas observing the same sources, and accounting for the short delay between the antennas, allows to combine all signals into a highly resolved one, equivalent to what would be received (with more sensitivity) by a single far larger array 2 . The simplest interferometer is called the short dipole antenna and is composed of only two antennas. Though very simple, it already contains all the most fundamental aspects of interferometry starting by the (u, v, w) coordinates system. For a pair of antennas with respective locations P 1 and P 2 , we can define a baseline b = P 2 -P 1 that is associated to a vector u ≡ b λ in the (u, v, w) space. When both antennas look at the same object in direction x = (x, y, z) in the sky, they will observe the same signal with only a delay of τ g = u * x c . For each frequency ν ≡ w 2π , the resulting output tension for the antennas will be:

Short dipole antenna, the simplest interferometer

V 1 = V cos(w (t -τ g )) V 2 = V cos(wt) (2.2) (2.3)
1 An angular resolution of θ ∼ 1 arcmin is already 10 times worse than the Hubble Space Telescop resolution 2 Quantitatively, the theoretical angular resolution of an interferometer is obtained using the largest distance between two individuals antennas as D in Equation 2.1 meaning that an equally resolved one antenna telescope should have a diameter of this maximal distance. This result is to be tempered though as the interferometer architecture and many observational challenges tend to reduce the interferometer resolution where V is proportional to the electric field produced by the source. These signals are then sent to an unit called a correlator that multiplies them and makes a time average3 , hence sending as its output :

R c = V 2 2 cos (wτ g ) (2.4)
Note that in equation 2.4 the correlator is an even cosine function, thus only sensitive to the even part of an arbitrary source brightness distribution. To detect the odd part, it is possible to add a correlator that insert a π/2 phase delay to the output of one antenna. Using the trigonometric formula cos(wτ g -π/2) = sin(wτ g ) we see that the second correlator outputs R s = (V 2 /2)sin(wτ g ). For computational purpose, we can combine the two correlators into a complex value, named the visibility V (u, v) :

V (u, v) = R c -iR s = I(x)e -2πiν u * x c dx (2.5)
where I is the spatial sky brightness distribution. One deduction of paramount importance from Equation 2.5 is that an interferometer effectively measures the two dimensional Fourier transform of the spatial sky brightness distribution I that can thus be retrieved by applying an inverse Fourier transform on the visibilities. In this case, the reconstruction is however very poor as only one Fourier mode corresponding to the baseline b is sampled. Thus, the more antennas in the radiointerferometer, the more densely the (u, v) plane (e.g. the Fourier space) is sampled. Furthermore, all quantities in Equation 2.5 are time-dependent and while it can be neglected for most of them, the projection of the baseline vector with respect to the source direction u * x varies significantly with Earth rotation, greatly improving the (u, v) coverage for observations of multiple hours as discovered by [START_REF] Ryle | [END_REF]. Also, an interferometer can theoretically observe several lines of sight at the same time making various combination of the input signal from its multiple antennas, a process called multi-beaming. In practice, the interferometer has to be designed to enable such parallel observations feature as it is impossible to store all input signals from all antennas to reconstruct all possible lines of sight in post-processing.

Tomography

One critical consequence of the cosmological expansion is that photons that were at the 21 cm wavelength at one point in time t 21cm will slowly see their wavelength increase, no longer being able to interact the hydrogen atoms later in time. Such photons thus keep information about the universe (state of ionization ...) at this time t 21cm without significant degradation due to their subsequent travel history. Consequently, observing the 21-cm signal through a wide frequency range allow us to look at the Universe evolution across time. One can easily imagine the amount of significant information about the reionization history that such observation would bring, giving access to the speed of reionization, the ionized bubble distribution or the fluctuations of primordial heating for instance.

In this context, imaging the spatial distribution of the 21 cm signal as a function of redshift, a process named 21-cm tomography, has been the primary long-term goal of the Epoch of Reionization field. The 21-cm tomography process implies very accurate observations with a sufficient signal-to-noise ratio for each line of sight. During the EoR, the 21-cm signal spatial distribution has fluctuations of various scales starting at a few arcminutes up to degrees (Majumdar et al., 2012;Datta et al., 2012;Mellema et al., 2013;Kakiichi et al., 2017). In the EoR 21-cm spectral window, arcminutes resolution can be reached using an instrument with a maximum baseline of a few kilometers. However the (u, v) plane needs to be densely sampled to have the sensitivity to observe such faint signal. To this day though no instrument has such configuration4 and only the Square Array Kilometer, detailed in Section 2.7, can pretend to achieve a 21-cm tomography in the near future.

21-cm Power Spectrum

Definition

If a full imaging of the spatial distribution of the 21-cm signal is still unreachable due to an insufficient signal to noise ratio (and observational challenges such as foregrounds removal detailed in section 2.5), statistical quantities about the signal are currently accessible. Especially, observing and constraining the Fourier transform of the auto-correlation function, also known as the power spectrum, is the main goal of most of the current EoR instruments. For an intensity field T B , the power spectrum of the 21-cm signal is defined using the relation :

δT * B (k) δT B (k ) = (2π) 3 P (k)δ 3 (k -k ) (2.6)
where represents the Fourier transform, * is the conjugate operator, is the average over multiple realizations (i.e. ensemble average), k is the Fourier space coordinate and δ is the Dirac delta function. Equation 2.6 can be re-written in a form that is more reachable observationally. Using the ergodocity hypothesis and the cosmological principle of large-scale isotropy and homogeneity, that respectively enables to replace the ensemble average by a volume average and allows P to only depends on the norm k of k leads to equation :

P (k, z) = 1 4πk 2 k | δT B (k, z)| 2 dS (2.7)
where dS is the Fourier space surface element. It is worth noting that the large-scale isotropy assumption is especially questionable along the line of sight. Observationally, there are two ways to obtain the power spectrum. The first one is to compute it after obtaining the interferometric image I through inverse Fourier transform of Equation 2.55 . The second, and more straightforward option, is to compute it directly through the interferometer response (e.g. the visibility), hence staying in the space of interferometric observations : the Fourier space. For that, we note that the power spectrum is proportional6 to the delay transform (Parsons et al., 2012b) obtained by applying a Fourier transform along the frequency axis to the visibility of Equation 2.5:

P (k) ∝ V (u, v, τ ) ≡ B V (u, v, ν)e -2πiντ dν (2.8)
where B is the observing bandwidth, and τ is a delay7 defined as the Fourier conjugate of ν. Equation 2.8 also outlines a straightforward mapping that exists between the k modes perpendicular and parallel to the line of sight (k ⊥ , k ) and the observer units (b ≡ λu, τ ) (Morales & Hewitt, 2004) where k ⊥ ∝ b and k ∝ τ .

Facility

Location

Actual instruments constraining the 21-cm Power Spectrum

Multiple existing instruments have already measured, analyzed and published constraints on the power spectrum. Before summarizing the current results, it is relevant to briefly present these instruments and their configurations8 .

PAPER and HERA

The Precision Array for Probing the Epoch of Reionization (PAPER9 ) (Parsons et al., 2008(Parsons et al., , 2010) ) is an array of 128 dipole antennas initially designed to develop new analysis techniques. Since 2012, the main focus of PAPER has been on developing an approach for measuring the CHAPTER 2. OBSERVATIONAL CHALLENGE 21-cm power spectrum called "delay spectrum" where visibilities from individual baselines are Fourier transformed and then cross-multiplied (Parsons et al., 2012a,b). The PAPER instrument has been designed into a "maximum redundancy" configuration shown in Figure 2.3 where antennas are placed on a grid to create multiple copies of the same baseline spacing that can be averaged together before squaring speeding the noise time averaging. The maximum redundancy configuration also comes with drawbacks as the number of different baseline lengths greatly decreases and the (u, v) plane is less evenly sampled but one can argue that this will not affect the final power spectrum quality as the visibilities are not combined and transformed into an image with this method.

Following the recent decommissioning of PAPER experiments, a new array, the Hydrogen Epoch of Reionization Array (HERA10 ) has been built in South Africa (Deboer et al., 2017;[START_REF] Koopmans | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF]. It uses the same approach as PAPER of a maximum redundancy configuration with more than 200 14 m diameter dishes positioned following a 350 m size hexagonal grid. Due to its configuration, the main goal of the instrument is to provide strong constrain on the power spectrum on a restricted mode range of k ∈]0.2 Mpc -1 , 0.4 Mpc -1 [ for redshift between 6 and 12. Foreground modeling is limited in such a redundant configuration. First results from HERA have been released recently in The HERA Collaboration et al. (2021).

GMRT

The Giant Metrewave Radio Telescope (GMRT11 ) (Swarup et al., 1991) is a telescope based on a reduced number of large (45 m in diameter) dishes instead of more numerous smaller antennas. It is composed of 30 such dishes arranged in a Y-shaped configuration for a maximum baseline of around 30 km. Such a design, lacking short baselines and coverage in the (u, v) plane, is especially powerful to construct foreground models as well as foreground angular correlation function (Rana & Bagla, 2019). To study the sky signature (both spectral and spatial), it uses cross-correlations of visibilities, an approach that has been enhanced (Datta et al., 2007) to estimate a multi-frequency angular power spectrum. GMRT was able to produce the first upper-limits at the time12 with (248 mK) 2 for (k = 0.5h. Mpc -1 , z = 8.6) [START_REF] Paciga | [END_REF].

MWA

The Murchison Widefield Array (MWA13 ) (Bowman et al., 2013;Tingay et al., 2013;Jacobs et al., 2016) has been built in the Western Australian desert, where human-made low radio frequencies interferences are at their lowest. It is the low band precursor of the future SKA array (detailed in Section 2.7) and as such has not only been built for EoR observation purposes (Bowman et al., 2013). It thus balances redundant and non-redundant configurations, longer baselines for imaging and high surface brightness sensitivity on EoR scales. Of the 256 apertures array composing this telescope half, gathered under the name "Compact Array", are focused on EoR science and have already provided multiple upper limits (Dillon et al., 2015;Beardsley et al., 2016;Ewall-Wice et al., 2016). MWA has also been used to produce the first sky model for calibration of EoR observations, encompassing ∼ 3 × 10 5 southern sources (Hurley-Walker et al., 2014;Wayth et al., 2015;Hurley-Walker et al., 2017). The early power spectrum limits published by this telescope (as well as LOFAR) also reveal the highly systematics-dominated nature of the current data, setting the focus on handling these systematics rather than acquiring more observations.

LOFAR

The Low Frequency Array (LOFAR14 ) ( Van Haarlem, 2005;Patil et al., 2016) is a composite aperture array designed to observe at low frequencies. Though only its core, built in the Netherlands, is used to study the EoR and Cosmic Dawn, multiple stations of LOFAR are scattered throughout Europe allowing it to have baselines of more than 10 4 km. The 36 Dutch-based, core stations of LOFAR, spread around an area of about 100 km diameter, are each composed of multiple smaller antennas, either Low-Band or grouped together. Its latitude is one of the main asset of this telescope as it can perform circumpolar observations. Early on, the LOFAR collaboration, like many other, tried to tackle the observational issues that plague the EoR observation, like foreground handling (Jelic et al., 2008(Jelic et al., , 2010;;[START_REF] Chapman | [END_REF], with a focus on quantifying the impact of polarization leakage15 (Asad et al., 2015(Asad et al., , 2016(Asad et al., , 2018)). Thank to this thorough study, the LOFAR collaboration published the most constraining upper limits to this date, summed up in Section 2.3.3. Beside the power spectrum, it is worth noting that LOFAR is also believed to be able to produce low angular resolution images (Zaroubi et al., 2012) and the 21-cm forest (Ciardi et al., 2013), as it is the only functional instrument to this date to have sufficient sensibility to try such experiments. NenuFAR (New Extension in Nancay Upgrading LOFAR) (Bondonneau et al., 2021) is a new array in its early-science phase closely related to LOFAR. NenuFAR is composed of 96 stations each one consisting of 19 dipoles. It is designed to observe frequencies between 10 and 85 MHz. Thanks to a new GPU-based correlator, the aim is to correlate all stations, enabling a state of the art sensitivity, sufficient to reach the predicted magnitude of the 21-cm signal during Cosmic Dawn. Being connected to LOFAR, it can be used by the latter as an efficient extension for low-frequency observation. Due to its characteristic, NenuFAR has been selected (French SKA white book, 2017) as an official pathfinder for SKA.

NENUFAR

OVRO-LWA and LEDA

The Owens Valley Radio Observatory Long Wavelength Array (OVRO-LWA) is a low-frequency interferometer composed of 288 dual-polarization dipole antennas. Like MWA, it serves multiple scientific purposes and provides data about, for example, radio transients surveys, exoplanet searches, emission of compact binaries or the properties of the ionosphere. Among OVRO-LWA antennas, 251 lie in a 200 m diameter core while 32 others are outside this core increasing the maximum baseline length up to 1.5 km. The five last antennas have radiometric front-ends for total power measurements of the sky. They are part of the Large-aperture Experiment to detect the Dark Age (LEDA) (Price et al., 2018) that attempts to constrain both the 21-cm power spectrum and the 21-cm global signal (detailed in section 2.4). OVRO-LWA and the LEDA experiment that it hosts already released multiple constrains on the 21-cm power spectrum (Eastwood et al., 2019;Garsden et al., 2021). In Figure 2.5 we show the latest results for modes k ∼ 0.1 -0.4 hcMpc -1 from GMRT (Paciga et al., 2013), PAPER (Kolopanis et al., 2019), MWA (Dillon et al., 2015;Beardsley et al., 2016;Barry et al., 2019;[START_REF] Trott | [END_REF], HERA (The HERA Collaboration et al., 2021) and LOFAR (Patil et al., 2017;Mertens et al., 2020). We focus on given modes and compare to 21cmFAST (Mesinger et al., 2011) (dotted black) and 21cmSSD (Semelin et al., 2017) (plain black) simulations to highlight the gap between the best up to date upper limits and the expected amplitude of the power spectrum in reasonable models. Other upper limits have also been predicted for other modes like ∆ 2 < 2.39 × 10 3 mK 2 at (k = 0.59 hcMpc -1 , z ≈ 6.5) in Li et al. ( 2019) and higher redshifts like ∆ 2 < 10 8 mK 2 at (k = 0.10 Mpc -1 , z ≈ 18.4) (Eastwood et al., 2019). Yet, almost all current upper limits are two orders of magnitude higher than simulated power spectra. For now, it thus only allows to discard the most exotic theoretical models. However, the upper limits steadily improved with better understanding of the foregrounds and systematics over the last years and are expected to be more and more constraining as time will pass. (Pritchard & Loeb, 2012).

Constrains on the Power Spectrum

Global 21-cm signal

If the measurement of the 21-cm fluctuations through imaging or power spectrum is unreachable, one can observe the mean 21-cm signal, also known as the global signal (Shaver et al., 1999;Furlanetto, 2006;[START_REF] Greenhill | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF]. In Figure 2.6 we show the expected evolution of the global 21-cm signal (Mellema et al., 2013), that has been described in Section 1.4.4 and can be summarized as follow :

1. During the Dark Ages there is a collisional coupling due to the density of the gas. At first the gas and radiation are still strongly coupled and kinetic and radiation temperatures are thus the same, hence the absence of signal. Gradually though gas and radiation decouple and the gas cools down adiabatically, the global signal is thus seen in absorption compared to the CMB signal.

2. When entering the Cosmic Dawn period, the first stars begin to form producing Lyman-α photons, which in turn couple gas and spin temperature leading to another larger absorption.

3. At the end of Cosmic Dawn, sources that can emit X-rays have begun to form and these X-rays start to heat the IGM, that will therefore be at a higher temperature than the CMB and start to emit more 21-cm photons than it absorbs.

4. Then the main part of the reionization process occurs and through UV emission, ionizes most of the hydrogen atoms, hence progressively canceling the 21-cm signal.

Even if the 21-cm global signal is an averaged signal, some information can be derived from its measurement. First and foremost, from its various critical points, it is possible to infer approximate values for the beginning and the end of reionization (Furlanetto, 2006;Pritchard & Loeb, 2010). Using this knowledge and going into further detail about the signal shape, it might be possible to constrain the redshift evolution of the star formation rate, the UV photon emissivity and the X-ray luminosity of the early galaxies.

To measure the global signal, it only requires a single radio dipole whose wide beam will obtain the average signal over the whole sky. Unlike 21-cm tomography, sensitivity is not an issue to measure the 21-cm signal but the same substantial need to calibrate the instrument exists. Moreover, foregrounds mitigation is an even harder task for the global 21-cm signal as it is difficult to isolate each contribution to this average signal. As instruments measuring the global signal are substantially cheaper, simpler, and easier to build than the one measuring the 21-cm fluctuation, several instruments are currently operating, being constructed or planned. We can for example cite SciHi (Voytek et al., 2014), LEDA (Greenhill & Bernardi, 2012;Bernardi et al., 2015;Price et al., 2018), SARAS (Singh et al., 2017), PRIZM (Philip et al., 2019), EDGES (Bowman & Rogers, 2010;Monsalve et al., 2017) and the proposed lunar experiment DARE [START_REF] Burns | [END_REF].

Among them, only EDGES has reported a possible first detection to this date (Bowman et al., 2018) with a significant and unexpected absorption peak at redshift ∼ 17. More precisely, the absorption profile is centered at a frequency of 78 MHz, has a full-width at half-minimum of 19 MHz and an amplitude of 500 mK. The most surprising feature of this observation is its amplitude. If we make the assumption that, at z ∼ 17, atomic collisions are sufficient to imply T S ∼ T K , we can express the spin temperature as T S ∼ T CMB,0 1+znr (1 + z) 2 , where z nr ∼ 200 is the redshift where matter and radiations temperatures start to differ significantly. Using equation 1.22 and assuming δ ∼ 0 and x H II ∼ 0, we can estimate that δT B ∼ -250 mK. The absorption profile observed by EDGES is higher than the maximum theoretical absorption value. If confirmed, this observed feature can be explained either by a gas colder than expected or an excess radio background above the CMB radiation. A colder gas might be the probe of a previously unknown interaction between dark matter and baryons (Barkana, 2018;Fialkov et al., 2018;Muñoz & Loeb, 2018;Yang et al., 2019;Lawson & Zhitnitsky, 2019;Liu et al., 2019). The possibility of an excess radio background has also been explored with studies assuming an empirical synchrotron-like spectrum (Feng & Holder, 2018;Fialkov & Barkana, 2019;Reis et al., 2020) or a short period of radio-loud accretion on black holes (Ewall-Wice et al., 2020). Other studies also consider non-astrophysical sources like light dark matter decays (Fraser et al., 2018), decay of relic neutrinos into sterile neutrinos (Chianese et al., 2019) or superconducting cosmic strings (Brandenberger et al., 2019). However, this measurement is still highly debated and in need of independent confirmation from other facilities.

Foregrounds

When observing the 21-cm signal, a critical issue is the impact of the foregrounds (Di [START_REF] Matteo | [END_REF][START_REF] Trott | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF][START_REF] Chapman | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF][START_REF] Weltman | [END_REF] that vastly dominate the signal. For the 21-cm signal, foregrounds are four order of magnitude brighter than the actual signal from the EoR. Such foregrounds can be divided into two main categories, believed to occupy relatively disjoint regions of the (k ⊥ , k ) space (Chapman et al., 2016).

Foreground categories

Figure 2.8: Illustration of the observed signal and its decomposition between various foregrounds and the redshifted 21-cm signal. Taken from [START_REF] Chapman | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF].

Galactic source

The main foreground source is the Galactic diffuse synchrotron emission that represents ∼ 70% of the total foreground emission (Shaver et al., 1999) and is the dominant foreground source from 10MHz to 10GHz, largely encompassing the EoR observing range and more. It is mainly produced by relativistic cosmic-ray electrons originating from supernovae close to the Galactic plane. These electrons are accelerated as they spiral around the strong supernovae magnetic fields and consequently emit radiations over a wide range of frequencies in all directions [START_REF] Pacholczyk | Radio astrophysics. Nonthermal processes in galactic and extragalactic sources[END_REF][START_REF] Rybicki | Radiative Processes in Astrophysics[END_REF]). If such synchrotron emission is theoretically well-known, it is still hard to accurately model the Galactic diffuse synchrotron emission that undergo multiple transformations before being observed. Indeed, as the radiation travels, it encounters different astrophysical contents such as matter, magnetic field and other radiation.

The interactions with such contents dramatically modify the expected frequency distribution of the synchrotron emission. Furthermore, there is a spatial fluctuation of the sources density because of the supernova distribution that is peaked along the galactic plane.

Along with the galactic diffuse synchrotron emission, around 1% of the total foreground emission lies in the galactic free-free emission, i.e. the thermal emission of ionized gas inside the Milky Way. Such emission being masked by the more important galactic synchrotron emission, it cannot be seen directly and is often constrained through the analysis of the Hα line (Lian et al., 2020).

Extragalactic sources

Extragalactic compact sources are also non-negligible foreground sources accounting for about 27% of the total emission (Mellema et al., 2013) and are composed of multiple components different in nature.

• Radio-loud Active Galactic Nuclei (AGN) : the synchrotron emission is here issued from the accretion of matter around the central supermassive blackhole of the AGN that produces narrow jets perpendicular to the plane of accretion. These jets usually have diffuse endings called radio lobes.

• Star-forming galaxies (SFG) : the emission of SFG are of strictly the same type than the galactic synchrotron of our own galaxy, namely synchrotron emission around supernovae and free-free emission from HII regions. The population of SFG at low-radio frequencies are still poorly constrained however16 and have to be extrapolated from high-radio frequencies populations.

• Radio-quiet AGN : the emission of such AGN present the characteristics of both previous types as radio-quiet AGN are likely SFG that possesses an active nucleus but without jets (Delvecchio et al., 2017;Ceraj et al., 2018). Its contribution is negligible compared to the two others types.

• Galaxy cluster radio halos and relics : they are extended radio sources that produce a diffuse non-thermal radio emission over Mpc scales. Such sources are defined as radio halos when their morphology is regular and they are at the center of the clusters. Conversely, when they are located at the periphery of the cluster and their morphology is irregular they are named radio relics. Radio halos are found in ∼ 30-40% of X-ray luminous galaxy clusters [START_REF] Matteo | [END_REF].

Due to the sheer difference in amplitude between foregrounds and the 21-cm signal, it is of paramount importance to create catalogs of all these sources, identify their signals and subtract them from the observed one. Complementary to the source identification, it is necessary to understand the spatial and spectral distribution of radio sources in the sky. It allows to account for still unconstrained sources or sources that cannot be cataloged due to a lack of sensitivity. If some source clusters are neglected, it can lead to false detection of the cosmological 21-cm signal (Murray et al., 2017a,b).

Mitigation strategies

The main way to mitigate these overwhelming foregrounds is through their measurement. This is done in various ways, creating diffuse sky models or observing bright compact extra-galactic sources in the field of view, and has been one of the primary focus of all instruments presented in Section 2.3.2 [START_REF] Bernardi | [END_REF](Bernardi et al., , 2010;;Ghosh et al., 2011;Yatawatta et al., 2013;Jelic et al., 2014;Asad et al., 2015;Remazeilles et al., 2015;Offringa et al., 2016;Procopio et al., 2017;Line et al., 2017). Once measured, known foregrounds components can be subtracted from the received signal. However due to their predominance even the few residual foregrounds, either due to miss-subtraction or real unmodelled sources, already significantly alter the signal and efficient mitigation strategies on the remaining signal are needed.

The key to mitigate the signal and the residual foregrounds lies in their statistical difference. While the foregrounds are mainly broadband emissions with smooth spectral signature, the 21-cm EoR signal is supposed to show structure at the MHz scale. The numerous mitigation strategies, used by each telescope, can be split into two different categories : subtraction and avoidance.

Residual foreground subtraction

Foreground subtraction methods are critically based on the assumption that the foreground spectral signature is smooth. For each angular pixel along the frequency axis a smooth function is fitted. Once the best-fit model is found, it is straightforwardly subtracted from the signal and the result is believed to be mostly the EoR signal. Such methods dramatically rely on an accurate fit and consequently on accurate fitting models that can be further separated into two groups.

• Parametric models : they were the first developed models. In their simplest form, parametric models are smooth polynomials of varying order (McQuinn et al., 2006;Bowman et al., 2006). More complex parametric models follow the "correlated component analysis" (CCA) (Ricciardi et al., 2010) approach where each foreground component is approximated by an individual empirical parametric form and then all functions are blended together through a linear mixing algorithm. Parametric models are thus very efficient to directly include physical knowledge into the foreground model but conversely suffer from possible over-fitting that alters the signal or degeneracies in the parametrization.

• Non-parametric models : they are more complex models based on the assumption that the multiple elements making up the mixed signal are statistically independent. No parametrization is proposed and the source separation process is made by adjusting the mixing matrix so that the statistical independence of the estimated components is maximized. Multiple implementations of this approach have been developed and some are used to this day in observation pipelines, like the fast independent component analysis (Chapman et al., 2012, FastICA) for LOFAR. Such models have the exact opposite characteristics compared to parametric models as they alleviate the possible over-fitting and parametrization ambiguity problems at the cost of the ability to directly add physical knowledge to the model.

Residual foreground avoidance

Another way to escape the influence of residual foregrounds in the signal is to discard modes where they are dominant. Conveniently, when representing the signal in the (k ⊥ , k ) space (described in Section 2.3.1) the foreground contribution mostly lies in a low-k , high-k ⊥ region called the "wedge" [START_REF] Bernardi | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF][START_REF] Chapman | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF][START_REF] Weltman | [END_REF], while its complement, the EoR "window", is reasonably free from foreground contribution (Liu et al., 2014a,b). Consequently, in principle, when averaging only over the EoR window modes, the resulting power spectrum only correspond to the 21-cm signal. The major drawback of this strategy is that many (k ⊥ , k ) cells are ignored and their underlying information lost.

Figure 2.9: Schematic representation of the (k ⊥ , k ) space with an example power spectrum from early MWA data. Taken from (Dillon et al., 2014).

Noise

Thermal noise

When observing a signal as faint as the 21-cm signal, any perturbation in the observation can critically affect the result. As such, even with a perfect foreground removal, an instrument can be unable to detect the 21-cm signal solely because of its thermal noise. One source of this noise comes from transformation of the radio signal into electric current. This current physically corresponds to the thermal motion of electrons inside a transistor, a motion that will add noise to the signal. Even if the time average of this noise is zero, its variance is not. The standard deviation in antenna temperature can be expressed [START_REF] Wilson | Tools of Radio Astronomy[END_REF]:

∆T A = T sys √ ∆νt int (2.9)
where T sys is the system total temperature, ∆ν the frequency band and t int the integration time. T sys can be further decomposed into T sys = T sky + T R where T R is the noise temperature of the receiver and T sky the brightness temperature of the sky. For the 21-cm frequency range, T sky T R and T sky can in turn be approximated (Mellema et al., 2013) leading to :

T sys ≈ T sky ∼ 400 ν 150MHz -2.2 K (2.10)
Using equation 2.9, the density flux S ν = 2k b T A Ae [START_REF] Wilson | Tools of Radio Astronomy[END_REF] noise for an interferometer composed of n antennas is :

∆S ν = 2k b T sys A e √ 2N ∆νt int (2.11)
where A e is the effective collecting area of an antenna and N = (n(n -1))/2 is the number of pair-wise correlations.

To obtain the noise temperature, we then apply the Rayleigh-Jeans relation of equation 1.18 to the flux density (S ν = I ν cosθdΩ). Assuming that the beam is gaussian (i.e. Ω ≈ θ 2 with θ the field of view17 ) we can express the noise temperature as :

∆T B ≈ λ 2 T sys A tot θ 2 √ ∆νt int (2.12)
where A tot = A e √ N is the total collecting area of the interferometer.

Noise on the power spectrum measurement

Sample Variance

The field of view of a real telescope is finite meaning that smaller modes (in terms of |k|) will massively be sampled whereas larger modes will be sampled only a few times. This lack of statistics with growing modes leads to another uncertainty that impacts the power spectrum measurement apart from the already existing thermal noise. This uncertainty is named the sample variance and its variance can be expressed as (McQuinn et al., 2006):

C SV (k) = P (k) λ 2 B 2 A e D M (z) 2 y (2.13)
where B is the telescope bandwidth, D M (z) is the comoving distance to redshift z and y is the depth18 of the observation.

Thermal noise for a given mode

As stated in Section 2.3.1, the finite number of antennas composing a telescope implies a finite number of baselines and consequently an imperfect sampling of the (u, v) space. For a total observing time of t int , the average observing time t k of a given mode k will vary according to the number of baselines that can observe such mode. More precisely

t k ≈ Aet int λ 2 n(b)
where n(b) is the baseline density. One can express the thermal noise variance of mode k as (McQuinn et al., 2006):

C N (k) = B∆νσ 2 A = λ 2 BT sys A e 2 1 Bt k (2.14)

Total uncertainty on the power spectrum

To reduce the noise on the power spectrum, one can use its property of symmetry around the polar angle to sum all measurements over a spherical annulus in Fourier space, hence decreasing the noise by a factor 1 √ Nc where N c = 2πk 2 sin(θ)∆k∆θ λ 2 yD M (z) 2 Ae(2π) 3 is the number of Fourier cells in such annulus. Then, using Equations 2.13 and 2.14 we can express the total error on the power spectrum measurement (McQuinn et al., 2006) :

δP (k, θ) = N -1 2 c A e yD M (z) 2 λ 2 B 2 C SV (k, θ) + C N (k, θ) (2.15)
Finally, to obtain the error on the spherically averaged power spectrum defined in Section 2.3.1, it is necessary to sum all pixels with the same | k |= k. The error on the power spectrum from a measurement of all pixels in a shell with constant k can be expressed19 :

δP (k) = θ (δP (k, θ)) -2 -0.5
(2.16) In the domain of radioastronomy, the community demand for more accurate observations has led to the design and now construction of a new instrument. The Square Kilometer Array (SKA) telescope is the product on an international cooperation20 and as such, will involve the whole community with operating, storing and administrative sites spread around the world. As shown in figure 2.10, this telescope is divided into two distinct and distant instruments. In South Africa, the first part of the telescope, named SKA1-mid and composed of 197 dishes separated from each other by at most 150 kilometers, will make sky observations between frequencies of 350 MHz and 15.3 GHz. The second part named SKA1-low currently starts its construction in western Australia and will be composed of around 131.000 antennas packed in 512 stations with each station separated by at most 65 km. It will conduct sky surveys between frequencies of 50 MHz and 350 MHz, theoretically outmaching the sensitivity of the current leading telescope, the european telescope LOFAR, in this frequencies.

SKA : Toward a radio-bright future

The previously detailed architecture compose the first phase of the SKA telescope, named SKA1. Its construction is currently starting for possible first observations in a few years. In Figure 2.11 we present a comparison in terms of sensitivity (left panel) and survey speed (right panel) as a function of the observed frequency between the current generation of telescopes in these frequencies (LOFAR, uGMRT and JVLA) and the predicted capabilities of SKA for its two phases. We see that SKA1 will be more sensitive by a factor of at least 2 than any telescope of the current generation and might be an order of magnitude more sensitive at some frequencies. In term of survey speed SKA1 will also outperformed this structures by an order of magnitude allowing astronomers to collect more accurate data. Though already impressive, SKA1 performance might eventually be outmatched by an order of magnitude by the second iteration of the telescope SKA2. This comparison most noticeably do not include the HERA telescope introduced in section 2.3.2. Like SKA, this telescope is a new generation telescope. Recently operational HERA is, in some cases, as sensitive as SKA1 will be.

SKA is also a perfectly fit instrument to study our signal of interest, the 21-cm signal. Having a rest frame frequency of 1420MHz, the 21-cm signal will gradually be redshifted as we look further back in time and its frequency will decrease accordingly. The lowest observation limit of SKA instrument is 50 MHz, which corresponds to the frequency of the 21-cm signal when emitted at redshift z = 27 hence making observation of the 21-cm signal from the Cosmic Dawn accessible. The central region of SKA1-low will contain, within the central 1km region, more than 200 stations giving good surface brightness sensitivity for early times, e.g. the Epoch of Reionization [START_REF] Trott | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF]. The stated aims of SKA1-low is to obtain the 21-cm power spectrum, the 21-cm Forest, and even more importantly direct imaging, the tomography being beyond the capability of current instruments [START_REF] Koopmans | [END_REF].

More than a huge technical challenge for precise engineering companies, SKA will also be a tremendous computational challenge. The cumulative data flux collected by SKA1 will be of around 15 Tb s -1 . Such flux is not meant to be stored but, even after reduction, the data flux will still be of the order of the Tb s -1 . It is then transferred to the Science Data Processors, to be further reduced. This process requires powerful numerical algorithms and methods yet to be developed. It is indeed necessary to live filter, denoise, discard, select, analyse the observations to only keep the relevant information without discarding potentially previously undetected signals revealing new science cases or adding an unwanted bias to the data. Once processed, the observations which now represent a mean flux of 20 Gb s -1 will be stored in SKA Regional Centres around the world. Even with such reduction the amount of data stored each year will be approximately 700 Po.

Though very promising, and endorsed by the participation of a large number of countries, SKA is currently only at the very beginning of its construction and only time will tell if this instrument will really revolutionize our vision of the Universe.

Numerical simulation techniques

Even in the numerical area, cosmology faces the same problem as observations, a problem one can schematically describe as "how to handle the Universe". A plain simulation of every physical particle in the Universe and their interaction is impossible. In numerical simulations we consider a group of physical particles as a single entity, a numerical particle that is characterized by parameters describing its average content. If one wants to have relevant results, the smaller the numerical particle is, the closest to reality the simulation remains. This implies handling a lot of interactions between numerical particles, with N 2 possible interactions per force for N numerical particles. However numerical particles, being physical stars, clouds, galaxies or clusters, all exchange energy not only in terms of gravity but also in terms of radiation that we must therefore take into account.

In this chapter, a focus will be set on two codes, one semi-analytical (21cmFAST) and the other that considers fully-coupled radiation-hydrodynamics (LICORICE). We focus on them because both are used in the context of this thesis. However, there are many other seminumerical (Geil & Wyithe, 2008;Choudhury et al., 2009;Zahn et al., 2011;Fialkov et al., 2013) and numerical (Mellema et al., 2006;Trac & Cen, 2007;Reynolds et al., 2009;Petkova & Springel, 2009;Rosdahl et al., 2013;[START_REF] Gnedin | [END_REF]Aubert et al., 2018;Ocvirk et al., 2020) codes that are simulating the Epoch of Reionization and one might be interested in reading the reviews of Trac & Gnedin (2011) and Mesinger (2017) for further details.

21cmFAST : a semi-numerical code

21cmFAST is a semi-numerical code that was designed to be as computationally cheap as possible at the cost of some approximations (Mesinger & Furlanetto, 2007;Mesinger et al., 2011;Sobacchi & Mesinger, 2014;Mesinger et al., 2016;Park et al., 2019). Even with such approximations, some studies argue that it gives results similar to fully numerical ones (Mesinger et al., 2011;Zahn et al., 2011).

One key characteristics of 21cmFAST which saves computation time is that the effect of radiation is included in an effective way: the photons are not followed and their effects on matter are handled through excursion-set algorithms that will extract collapse fractions and the ionization process as will be detailed in this Section.

Linear Density and velocity fields computation

The density field in 21cmFAST is generated using the same method as in some numerical Nbody codes, namely assuming that the density field δ(x) ≡ ρ(x)/ρ -1 can be approximated by the growth of a Gaussian random field in the linear regime because density perturbations evolve as δ(z) = δ(0)a(z) with a the expansion factor. All statistical properties of this field are defined by its power spectrum P (k) ≡ V -1 |δ(k)| 2 where δ(k) is the Fourier transform of δ x . In Fourier space, the density value for each independent wavenumber is:

δ(k) = V P (k) 2 (a k + ib k ) (3.1)
where a k and b k are drawn from a normal distribution. In 21cmFAST the power spectrum used in Equation 3.1 is taken from (Eisenstein & Hu, 1999). It is then straightforward to apply an inverse Fourier transform to δ(k) to finally obtain the linear density field δ(x).

Once the linear density field is computed, a standard (Zel'dovich, 1970;Efstathiou et al., 1985;Sirko, 2005) approximation is used to compute the corresponding velocity field. Using linear approximation (δ(x) 1) and continuity criterion the velocity modes of the simulation can be expressed as:

v(k, z) = v z (z)v k (k) = ȧ(z) ik k 2 δ(k) (3.2)
Note that the computation is made in Fourier space as it is computationally faster than in real space. Also, as emphasized in Equation 3.2 the temporal and spatial components of the velocity field can be separated meaning that the spatial component has only to be computed once and can be reused for any redshift.

Collapsed fraction f coll

In 21cmFAST, halos are not identified. Instead, to handle the reionization process, a collapsed fraction f coll is computed when necessary. Following Barkana & Loeb (2004), f coll at redshift z in a region of radius R centered at position x is (Mesinger et al., 2011):

f coll (x, z, R, M min ) = f ST f PS erfc   δ c -δ R nl 2[σ 2 (M min ) -σ 2 ( 4 3 πρR 3 )]   (3.3)
where σ is the mass variance, δ c = 1.686 is the critical density δ R nl is the evolved density field smoothed on scale R 1 , and f ST and f PS are the mean collapsed fractions (averaged over δ R nl ) following respectively Sheth et al. (2001) and Press & Schechter (1974) formalisms. The normalization factor f ST f PS ensures that the mean collapse fraction will match results from the Sheth et al. ( 2001) formalism, specifically developed so that results are more in agreement with numerical simulations (e.g. Trac & Cen, 2007).

Ionizing sources

One key feature of 21cmFAST is its handling of the reionization, a process driven by photons which are not directly propagated because of the high computational cost. To alleviate this problem, only regions of mass above a spatially variable minimum M min are considered as ionization sources. This minimum represents the threshold above which stars are efficiently formed and is expressed as a maximum between photoheating, cooling and supernovae feedback effects (Sobacchi & Mesinger, 2014;Mesinger et al., 2016) :

M min = max(M cool , M photo , M SNe ) (3.4)
1 δ nl is related to the linear density field through δ(δ nl ) = δc 1.68647 1.68647 - -Martín et al., 2008) The first term of the right-hand side of Equation 3.4 M cool is a tunable parameter defined as the mass below which the halo virial temperature2 is below the minimum temperature that gas can reach through atomic cooling, and thus below which the gas content will not easily fragment and form stars. Following some studies that tend to show that molecular cooling is suppressed at the very beginning of reionization (Haiman et al., 1997), the fiducial value of M cool corresponds to a case where molecular cooling is negligible. It is worth noting that, in the following study, we use a version of 21cmFAST that only consider M cool .

1.35 (1+δ nl ) 2/3 -1.12431 (1+δ nl ) 1/2 + 0.78785 (1+δ nl ) 0.58661 (Rubiño
The second term of the right-hand side of Equation 3.4 M photo is the minimal mass for halos inside HII regions to keep enough gas to form stars. Indeed, it has been emphasized in numerous works (Shapiro et al., 1994;Hui & Gnedin, 1997;Mesinger & Dijkstra, 2008;Okamoto et al., 2008;Pawlik et al., 2009) that the additional heating from the UV background in ionized regions can deplete halos of their gas. In 21cmFAST the expression used for M photo is a theory-driven functional form whose parameters have been empirically determined in Sobacchi & Mesinger (2013).

The third term of the right-hand side of Equation 3.4 M SNe accounts for the supernovae feedback, especially important for high-redshift galaxies [START_REF] Springel | [END_REF]. The amplitude of this feedback effect is yet to be constrained by observation and as such M SNe is a tunable parameter.

Recombination process

Before the major update of (Sobacchi & Mesinger, 2014), the inhomogeneous recombination effect was not considered by 21cmFAST. To introduce this fundamental notion without increasing significantly the computation time, the code relies on various empirical fits to numerical simulations. First, to describe the small-scale structures of the IGM at a subgrid level (Cole & Kaiser, 1989), a gas density distribution P V is computed following the empirical formula calibrated in Miralda-Escude et al. (2000). Then, assuming ionization equilibrium the neutral fractions at each overdensity ∆ ≡ n b /n b with n b the baryonic density are computed. The self-shielding is then taken into account in the local photoionization rate term using the empirical fit to radiative transfer simulation from Rahmati et al. (2013) and a self-shielding overdensity threshold established in Schaye (2001). The recombination rate dnrec dt per baryon in an ionized cell can then be obtained by integrating over the density distribution as follows:

dn rec dt (x, z) = +∞ 0 P V (∆, z)∆n H α b (1 -x H I (∆)) 2 d∆ (3.5)
where α b = 2, 6 × 10 -13 cm 3 s -1 is the case B recombination coefficient for gas at T∼ 10 4 K, x H I is the neutral fraction and n H is the mean hydrogen number density. Once the recombination rate for the given ionized cells is computed, it is integrated over time and averaged over the considered scale R of the region to finally reach the average total number of recombinations in the region n rec (x, z, R).

Reionization computation

Once all the above quantities are computed, the reionization process is handled using an "excursionset" method (Press & Schechter, 1974;Bond et al., 1991;Lacey & Cole, 1993;Sheth & Tormen, 1999;Furlanetto et al., 2004) applied to the non-linear density field. At each step of this approach a region of size R is considered around a given cell of position x and redshift z. If, during the sources lifetime, the number of ionizing photons produced inside the region is greater than its number of baryons plus the cumulative recombinations, the central cell is tagged as fully ionized. Mathematically, this condition can be written (Sobacchi & Mesinger, 2014;Mesinger et al., 2016) :

ζ ion f coll (x, z, R, M min ) ≥ 1 + n rec (x, z, R) (3.6)
where ζ ion is the ionizing efficiency, f coll the fraction of collapsed matter (computed in Section 3.1.2) and n rec (x, z, R) the average cumulative number of recombination computed in Section 3.1.4. In this computation, n rec is averaged over the considered scale R. The algorithm then looks at a region of the same size centered around another cell and the process is done iteratively over all cells. More precisely, this process starts for large regions of size R mfp and once all possible center cells have been looked at, analyzes again cells centers not already ionized with a shorter size R until reaching R cell where cells that are not tagged as fully ionized have their ionization fractions set to

ζ ion f coll (x, z, R cell , M min )/(1 + n rec (x, z, R cell ))
. Equation 3.6 encapsulates a wide range of physical effects in a small number of parameters, two of which have already been detailed. For the last one, the ionizing efficiency ζ ion it can be expressed as (Mesinger et al., 2016) :

ζ ion = 20 N γ 4000 f esc 0.1 f * 0.05 f b 1 (3.7)
where N γ is the number of ionizing photons per stellar baryons, f esc is the fraction of photons that effectively escape into the intergalactic medium, f * is the fraction of gas in stars and f b the fraction of baryons inside the galaxy with respect to the cosmic mean Ω b /Ω m . The normalization values for the parameters are fiducial choices in agreement with luminosity functions of high redshift galaxies (Barkana & Loeb, 2005b;Robertson et al., 2013).

LICORICE: a fully coupled radiation-hydrodynamic code

LICORICE is a code developed to simulate the EoR that solves simultaneously gravity, hydrodynamics and radiative transfer (Semelin & Combes, 2002;Semelin et al., 2007;Iliev et al., 2009;Baek et al., 2009Baek et al., , 2010;;Vonlanthen et al., 2011;Semelin, 2016;Semelin et al., 2017;[START_REF] Bolgar | [END_REF]. It is a High Performance Computing (HPC) code parallelized with both OpenMP and MPI.

In practice the simulation box, with periodic conditions applied to its edges, is split into (2 n )3 n>1 sub-boxes 3 , where n can be set to any integer value depending on the available computing power. As the code uses a Lagrangian approach, it works with a fixed number of particles4 , often set to (2 n ) 3 , half being dark matter and the other baryons. All particles of one type have the same mass and no new particles are introduced during the star forming process as explained in section 3.2.3.

Gravity computation: Tree Code

To compute the gravitational force applied on each particle in the simulation, the naive approach would be to look at each particle, compute the gravitational interaction between this one and every other particles in the simulation, one at a time, which corresponds to a complexity of O N 2 . For state-of-the-art simulations, that consider around 10 9 particles, it would take 10 18 gravitational force computations thus at least 10 8 s 3 years on a single CPU, for one computational step. We thus need to rely on tree algorithms, the one used in LICORICE being the Barnes&Hut algorithm (Barnes & Hut, 1986;[START_REF] Hernquist | [END_REF]Springel et al., 2001), where the force exerted by distant particle groups is approximated by their lowest multipole moments.

Figure 3.1: Representation of the box decomposition process that isolates each source into a box and discard empty boxes to form a tree. Given a star, the tree is then walked: if the cube is seen with an angle θ < θ 0 (yellow lines case) we only consider its multipole expansion, else it will be opened and its composing sub-cubes will be analyzed (orange line case).

First, we put our particles into a tree structure. The whole box (root node) is decomposed into eight smaller cubes (nodes) each with a size of half the box size. Then, cubes that contain more than one particle are subdivided again into eight more sub-cubes and this process continues iteratively until there is only one particle in each created sub-cubes (which are schematically the leaves of the tree)5 .

The gravitational force computation then uses this tree. Considering one particle, the algorithm will walk the tree and only look into the child nodes of a given node if it is close enough to the particle. Quantitatively, we do not look into a cube and consider only the multipole expansion of it, if the distance between the particle and the cube's center-of-mass is greater than

r lim = l θ 0
where l is the size of the cube and θ 0 , the angle of view, is a fixed accuracy parameter of value ∼ 0.7. If the cube is closer than r lim , it is opened and its sub-cubes are similarly analyzed. A 2D illustration of the box decomposition and the walk is shown in Figure 3.1.

One feature of the LICORICE code is that it has periodic boundary conditions. It is mandatory in any cosmological simulation as free boundaries will results in the whole box collapsing towards its center of mass. The periodic replicas of particles, therefore, have to be taken into account during the gravity computation. Fortunately, the formula to sum the monopolar contribution of periodic particles is given by Ewald (1921) and was implemented in the code. With this process the entire computation of gravitational forces on all the particles has a numerical complexity of O(N logN ).

Hydrodynamics solving: TreeSPH

SPH and TreeSPH principles

In hydrodynamics, two approaches can be followed to consider a fluid. First, the Eulerian approach looks at the properties of the fluid from a fixed position which numerically implies to have a grid on which we compute the properties of the fluid. On the other hand, the Lagrangian approach follows the fluid elements, which numerically implies that the fluid is represented as numerous particles that each possesses a relative motion compared to one another. The LICORICE code uses the latter approach.

The LICORICE code is based on a Smooth Particle Hydrodynamics (SPH) method. All particles in the fluid are characterized by various quantities (such as mass, position, velocity, temperature, ...) and thus to obtain the value of a physical quantity f at one precise point r in space we have to make our estimation based on the neighboring particles. In its simplest definition, to get f (r) the SPH method is simply making a weighted average on the value f i associated to each particles. The closer from the considered point one particle is, the higher its weight. This can be mathematically expressed as [START_REF] Hernquist | [END_REF]:

f (r) f (r) = f (r )W r -r , h(r) d 3 r (3.8)
where W is the smoothing kernel attributing weights to each particle and h is a smoothing length, the typical size of the smoothing kernel. Thanks to Equation 3.8, it is possible to compute the hydrodynamic evolution of some of the particles characteristics like temperature. However to compute this evolution with a reasonable noise, we need to consider a minimum number of neighboring particles to the one we analyze. This minimal number can be reached at short distances in gravitationally collapsed regions or at very large ones for the most underdense areas. Therefore, if in regular hydrodynamics the smoothing parameter h is considered constant, in astrophysics however it has to be adapted, depending on the considered particle, to reach this minimum number of neighbors, scaling h over two orders of magnitude6 .

In LICORICE, we use a variation of Equation 3.8:

f (r) f (r) = f (r )W r -r , h(r) + h(r ) 2 d 3 r (3.9)
The main difference with the previous equation is that we use the mean smoothing length between the two interacting particles ensuring that a force of the same magnitude will be applied to both particles through this interaction (i.e. respecting the action-reaction principle). In any case, a straightforward requirement of Equations 3.8 and 3.9 is that f (r) → f (r) when h → 0. This constrains the smoothing kernel function W to be normalized : W (r, h)dr = 1. Furthermore, we usually select W such that it is continuously differentiable, spherical and positive so that the error made in approximating f (r) by its smoothed estimate f (r) is in ∼ O(h 2 ). This also implies that for two quantities A(r) and B(r) [START_REF] Hernquist | [END_REF]:

A(r) B(r) = A(r) B(r) + O(h 2 ) (3.10)
In simulations framework, we only know the value of property f at a finite number of discrete points, e.g. our particles, whose distribution can for example be described by the density ρ(r) = N j=1 m j δ(r -r j ). Therefore, evaluating Equation 3.9 at position r i , multiplying its integrand by ρ(r j ) ρ(r j ) = 1 + O(h 2 ) and using Equation 3.10 we get:

f (r i ) ∼ f (r i ) ρ(r j ) ρ(r j ) = N j=1 m j f (r j ) ρ(r j ) W r i -r j , h i + h j 2 (3.11)
With the same approach we can also estimates the gradients:

∇f (r) = ∇f (r )W r -r , h(r) + h(r ) 2 d 3 r (3.12)
which, integrated by parts7 and then evaluated at position r i and reformulated for a finite number of discrete points gives:

∇f (r i ) = N j=1 f (r j )m j ρ(r j ) ∇W r i -r j , h i + h j 2 (3.13)

Reaching the dynamical system of equations

Using the SPH method detailed in the previous section, we can obtain smoothed forms for the hydrodynamical conservation laws, that can be turned into the dynamical system of equation that will rule particles motion. The basic dynamical evolution equations of a particle i are given by:

         dr i dt = v i dv i dt = - ∇P i ρ i + a visc i -∇Φ i (3.14) (3.15)
where Φ i is the gravitational potential that has been computed in section 3.2.1, a visc i is a viscosity term whose presence enables formation of shockwaves [START_REF] Hernquist | [END_REF]) and P i is the pressure. The pressure contribution can be re-written in a symmetric computable smoothed form, using the identity ∇P i = 2 √ P i ∇( √ P i ) and Equation 3.13 as:

∇P i ρ i = N j=1 m j 2 P i P j ρ i ρ j W r i -r j , h i + h j 2 (3.16)
Concerning the viscosity term a visc i it can be transformed into a viscous contribution to the pressure gradient Π i,j and merged with the first term as shown in Equation 3.17.

The internal energy u i of particle i follows the first law of thermodynamics

du i = -P i dV i + T i ds i
where V i = ρ -1 is the specific mass and ds i is the specific entropy which account for all nonadiabatic effects. To close our system of equations that rule the particle motion, and to describe the evolution of the fluid, we add the ideal gas law P = (γ -1)ρu where γ is the adiabatic index equal to 5 3 for monoatomic gas. Applying the same methods to all equations of our system, we can finally write the smoothed form of our dynamical equation system:

                     dv i dt = -∇Φ i - N j=1 m j 2 P i P j ρ i ρ j + Π i,j W r i -r j , h i + h j 2 du i dt = N j=1 m j P i P j ρ i ρ j + Π i,j 2 v i -v j 2 W r i -r j , h i + h j 2 + Γ -Λ ρ P i = (γ -1)ρ i u i (3.17) (3.18) (3.19)
where Γ is a heating term, Λ is a cooling term.

Stars

Star Formation

LICORICE uses a constant number of particles implying that star formation takes place inside the baryonic particles. When the overdensity becomes greater than a given threshold (usually 100), star formation begins following Schmidt law with exponent 1:

dρ s dt = c eff ρ g (3.20)
where c eff is a calibration coefficient and ρ s and ρ g are respectively the local stellar and gas densities. Having baryonic particles composed of a few percents of star content is computationally expensive because we still compute the dynamics of the stars with the gas dynamics. Consequently, when a particles stellar content exceeds 2%, all the neighboring particle stellar content is gathered. For all particles to keep the same mass, the gas content of the particle is then given to its neighbors. Eventually the gathered baryonic particle stellar content will reach 100% and this particle will be turned into a star particle that is collisionless.

Emissions

Any particle with a stellar content is considered a source. To compute the average emissivity, we assume a black body emission for the stars and Salpeter IMF with lower cutoff at 1.6M and upper cutoffs at 120M (Salpeter, 1955;Baek et al., 2010). LICORICE being designed to simulate large boxes ≥ 100cMpc, even a few percent of the mass of a baryonic particle represents ∼ 10 6-7 M . Star forming regions are thus unresolved. In this framework, it is the cumulative ionizing radiation more than the timing that impacts the medium. Thus only two parameters, namely the luminosity and lifetime, are needed for the UV emission of a source. For simplicity, all sources have constant luminosities over their lifetimes. If stellar content is added to the particle, both its luminosity and remaining lifetime are increased in order to conserve the underlying energy to emit. See Baek et al. (2010) for more details.

Ionizing radiative transfer

Radiative transfer handling is a salient requirement to accurately model the Epoch of Reionization. Some codes like 21cmFAST derive radiation effects from density fields. However, Figure 3.2: Brightness temperature maps for six models produced for the 21SSD database (Semelin et al., 2017) based on LICORICE. The parameters respectively account for the Xray emissivity (f x ), the Lyman band emissivity (f α ) and the ratio between hard (from X-ray binaries) and soft (from AGN) X-rays (r H/S ). Taken from Semelin et al. (2017).

to directly compute the radiative transfer, several methods are used by numerical codes. For example, there are full ray-tracing algorithms that propagate photons from every source of the simulation often during post-processing treatment due to the expensive computational cost (Mellema et al., 2006;Ahn & Shapiro, 2007) sometime mitigated by split and merging method (MORAY [START_REF] Wise | [END_REF] or SCORCH (Trac & Cen, 2007;Trac et al., 2015;Doussot et al., 2018;Chen et al., 2020)). One can also mention moment-based methods based on the computation of the moments of the specific intensity equation (Gnedin & Ostriker, 1997) among which are the photon number density and the flux (ENZO-RT (Reynolds et al., 2009), GADGET-RT (Petkova & Springel, 2009), RAMSES-RT (Rosdahl et al., 2013), ART [START_REF] Gnedin | [END_REF], EMMA (Aubert et al., 2018)).

The radiative transfer that is used in LICORICE is based on a Monte Carlo method. In this method, one needs to compute on an adaptive grid the values of the density, ionization frac-tions and other relevant quantities for the radiative transfer. Then, at each timestep, photon packets are distributed among sources based on their luminosities. Randomly chosen directions are attributed to these packets that start to propagate at the speed of light. At each cell that a photon packet crosses, it "deposits" a fraction e -τ (τ being the optical depth of the cell) of its content, until the photon packet is almost depleted, at which point it is erased. Though less costly than full ray-tracing, this method requires cells and their ionized fractions to be regularly updated, which is a particularly costly step. For a number N of photon packets crossing a given cell per timestep, the noise due to the discrete number of realizations scales as N -0.5 and we can adjust the number of photon packets emitted to shrink this noise down to a desired threshold. In practice, as the number of sources increases significantly during the reionization process, one needs to set a fixed number of overall photons packets (not per source) and distribute it among all sources: decreasing the number of photons packets per source while source number increases. Simulation comparisons works (Iliev et al., 2006(Iliev et al., , 2009) ) show that, though noisier than full raytracing algorithm, such Monte Carlo methods gives competing results.

The Monte Carlo method for radiative transfer that is used in LICORICE has been adapted to the code specificities. We build an adaptive grid following Barnes & Hut (1986) algorithm (see section 3.2.1), but we restrict the tree construction when nodes contain at most 32 particles8 . The photon packets have random direction and frequencies, and are either UV or X-ray photon packets at a tunable proportion. For a given type (UV or X-ray), photon packets have the same energy (i.e. number of photons).

• UV photon packets have their frequencies randomly drawn from a computed spectra. At our resolution one star particle corresponds to a star cluster, so we assume a Salpeter luminosity function (Salpeter, 1955). We also use [START_REF] Aller | Numerical Data and Functional Relationships in Science and Technology -New Series[END_REF] table to get the radius and effective temperature of stars as a function of the mass. The spectral energy distribution and total luminosity are then computed by integrating over the luminosity function.

• X-ray photon packets have a significantly longer mean free path than UV photons, they can cross the simulation box multiple times before being depleted, particularly at the end of the reionization where possible absorbers are scarce. To alleviate this computationally costly effect, after a sufficiently large distance9 has been crossed, we consider the information about the location of their source as irrelevant and the photon packet as "background photon". Regularly some of the background photons are merged to keep their number reasonable (Semelin, 2016;Semelin et al., 2017). The energy of X-ray photon packets is linked to the star formation rate using the parameter f x (see Semelin et al. (2017) for more details). X-ray photon packets can be of two types at a tunable ratio (given by the simulation parameter r H/S ) :

-"Soft" X-rays that represent X-rays emitted by AGN. Their production is modeled using a spectral index of 1.6, a lower cutoff at 100 eV and an upper cutoff at 2 KeV (Semelin et al., 2017).

-"Hard" X-rays that represent X-rays emitted by X-ray binaries. They are modeled using the spectral properties tabulated in Fragos et al. (2013).

The heating efficiencies of these two types of X-rays are different (Fialkov et al., 2014).

Lyman-α line radiative transfer

A key feature of the LICORICE code is that it considers the Lyman-α radiative transfer (Semelin et al., 2007;Baek et al., 2010) required to accurately determine the Wouthyusen-Field effect used in the computation of the spin temperature 1.4.3. Lyman-α photons also heat the gas through their scattering (Furlanetto & Pritehard, 2006;Chuzhoy & Shapiro, 2006). This heating is of the order of a few Kelvin (see equation 7 in Baek et al. (2009)) while halos have virial temperatures of the order of 10 4 K. The Lyman-α transfer thus has a negligible effect on the gas dynamic and, as such, can be computed in post-processing on the particles snapshots.

To accurately compute the spin temperature, x α (defined in equation 1.26) must be evaluated everywhere in the box. This requirement comes at a high computational price. Computing the Lyman-α radiative transfer is indeed much more computationally costly than computing ionizing radiative transfer. Indeed, the X-rays photons and the UV ionizing photons are respectively traveling across the whole box and absorbed mostly on a 2 dimensional ionization front. For their part, the Lyman-α photons are scattered hundreds of times across multiple cells while their wavelengths are in the wing of the Lyman-α line. However, they are eventually redshifted toward the Lyman-α line core and undergo millions of scattering in the same cell. Lyman-α photons therefore effectively contribute to one cell only. Consequently, much more Lyman-α photons than ionizing or X-rays photons need to be propagated to reach the same level of noise using the Monte Carlo scheme. For these reasons, even if run in post-treatment, in LICORICE the Lyman-α transfer is computed using similar scheme as for the ionizing radiative transfer but on a fixed grid for efficiency purposes.

The computation of Lyman-α radiative transfer is one of LICORICE salient features as its computational cost leads semi-numerical models to avoid its direct computation. For comparison, in 21cmFAST the Lyman-α line (as well as the X-ray heating) effect is considered through a global rate computed by summing contribution from sources in a given region. The implementation is therefore similar to the ionization algorithm presented in 3.1.5 (Mesinger et al., 2011). The computation in 21cmFAST is thus on average quantities whereas, in LICORICE approach, the local values of the fields are considered.

Part II

Parameter Inference

4

Artificial Intelligence

This chapter is an introduction to the artificial intelligence field, the new materials developed during our study are presented in the next chapter.

The Advent of Artificial Intelligence

A long development

Though widely thought as a cutting new technology, Artificial Intelligence (AI) in its current sense1 is a field as old as computer science itself. Alan Turing releases its first research on what will become computer science in 1937 (Turing, 1936) while the first mathematical model to describe a basic artificial neuron is published in 1943 (McCulloch & Pitts, 1943). Artificial Intelligence then starts to progress academically the following years leading to the game-changing article Rosenblatt (1958). It presents a way to connect binary neurons into a proto-network named Perceptron. Follow two decades of interest in the field by the industry with large amounts of money invested in the hope of developing technological products. However practical applications fail to come, mainly because of the lack of computational power available (Minsky & Papert, 1969).

The AI field continues to develop in academic spheres for several decades and while groundbreaking methods are implemented, like Hopfield (1982) and Rumelhart et al. (1986) presenting respectively new ways of connecting neurons and new learning algorithms2 , scientists still do not have access to the computational power and large data set to transform these concepts into results. Eventually, the technological improvement reaches the level to satisfy such requirements. One of the most resounding successes of these practical applications is the chess games between the specifically trained system Deep Blue (Campbell et al., 2002) from IBM and the then world champion Garry Kasparov. This put back AI on the front stage and technology juggernauts invest massively in its development creating the technological arms race that is known today.

An unavoidable tool in this new era

One key feature of AI and neural networks in particular, is that it performs better on larger data sets. Previously a fatal hindrance to its development, this specificity is now one of its assets as we enter an era, often referred as "Big Data era", high dimensional data are continuously created and exchanged. It is markedly the case for astronomy, where observational data are more and more abundant, as detailed for example for the new SKA telescope in section 2.7 where the observation data flux will be of the order of 1Tb.s -1 and needs to be filtered live, an area where neural networks are especially efficient whereas other methods tend to be overwhelmed.

In astronomy, AI is a tool that has first been used in the domain of extra-galactic large surveys where millions of galaxies and observations must be classified based on their morphology (Odewahn et al., 1992;Bazell & Peng, 1998;Lintott et al., 2008;Huertas-Company et al., 2011;Cornu & Montillaud, 2021). It shows that, once trained, neural networks perform classification tasks faster and more reliably than "classical" algorithms. Even for numerically well resolved problems it can output a result with a speed orders of magnitude faster (Bonnett et al., 2016;Shimabukuro & Semelin, 2017;Gillet et al., 2019;Doussot et al., 2019). This past years, other machine learning algorithms were extensively used with good results in all field of astrophysics like in molecular cloud clustering using unsupervised Meanshift method (Bron et al., 2018) or in wide-field optical surveys simulating galaxy morphology image through deep generative models (Lanusse et al., 2021). We can also cite catalog outliers detection using unsupervised random forest (Baron & Poznanski, 2017) or real-time gravitational wave detection and parameter inference (George & Huerta, 2018). See (Ball & Brunner, 2010;Fluke & Jacobs, 2020) for more comprehensive reviews. This diversity shows that machine learning is a tool that will become one of the most commonly used in the near future.

Categories of Artificial Intelligence algorithms

Artificial Intelligence is a field that evolves at a very fast pace. Its diversity makes it hard for anybody to establish an exhaustive list of all Machine Learning methods that exist3 . It is however possible to classify all methods depending on how they learn.

• Supervised methods : they use a data set of examples (also known as the learning set)

where both the input and target output are specified. Such methods learn by comparing their computed outputs to the reference ones and are eventually able to make predictions for inputs whose outputs are unknown. Using another data set with known input-output pairs (also known as the test set) is enough to assess the performances of such methods.

• Unsupervised methods : they use a data set where no expected output is specified. As such, these methods will try to categorize their inputs based on given proximity estimators.

• "Reward-based" methods : they also use a data set where no expected output is specified but use a predefined reward function to estimate how appropriate their predicted output is. They are particularly useful for task where the optimal result is unknown but some basic rules are known and two solutions can be ranked. It can be further subdivided into two subclasses :

reinforcement : with a given initial state that learn by trials and weights modification.

evolutionary : these methods mimic biological evolution. Instead of one algorithm starting at a given initial state, a population of numerous networks are randomly initialized and trained in parallel. Only the ones that give best results are used to create the next population of networks that will be tested and the process repeats until convergence. As in natural evolution, some "mutations" are included to insure a thorough exploration of the weights space.

The previous classification remains theoretical as many complex machine learning algorithms combine methods. For example, it is frequent to use what can be called "hybrid" methods that starts with a unsupervised algorithm that typically reduces the dimensionality of the input data, before using a supervised method to link this intermediary result to the desired output.

In astrophysics, as in any other fields, it is of paramount importance to accurately relate one peculiar task to its corresponding problem type. Once the type is identified, the adapted machine learning methods can more easily be selected in the otherwise confusing diversity of algorithms. In figure 4.1 we thus present some of the most popular machine learning algorithms as well as their application domains.

Artificial Neurons

Definition

In its simplest definition, a neuron is a unit of calculus that takes a given number of input values and outputs a resulting value that mathematically depends on the inputs. More precisely, the neuron starts by making a weighted sum S of the n input values :

S = n i=0 ω i x i + b (4.1)
where x i is the i-th input, b a bias parameter4 and ω i its associated weight. Following this sum the neuron applies a function a, often non-linear and called activation function, to the weighted sum. The most basic activation function used for binary output is the step function but smoother functions can be used for regression problems.

Single neuron training

Initially the weights ω i of the neuron are chosen randomly and the output result is thus random as well. The strength of the neuron is that it can "learn" by adjusting its weights following a gradient descent process. This gradient descent algorithm finds the local minimum in the weights space of a given function. In the supervised learning case, the weights modification is made by comparing the computed output Y = a(S) to the reference value Y ref associated to the input in the learning set through an error function. This error function (also called cost function or loss in the machine learning field) can be the straightforward square error

C(Y, Y ref ) = 1 2 (Y -Y ref ) 2
or more complex ones as detailed in section 4.6.5. Usually the cost function is computed for multiple data of the set and then summed. The weights are then slightly modified following :

ω i ← ω i -η ∂C ∂ω i (4.2)
where η is a learning rate that will be considered constant in this example and quantifies the amplitude of the weight modification. Once the weights are updated, the next learning step starts and another computation is done by the neuron. The new output is once again compared to the reference value and the process continues iteratively until no more improvement can be achieved.

Perceptron : the simplest network

A single neuron using a step function as an activation function is only able to perform a linear separation in the output space. For more complex problems, a far more flexible separation may be required. For that, multiple neurons can be combined into networks. The first and simplest architecture is the Perceptron (Rosenblatt, 1958). For the Perceptron, neurons are not linked together, they are all receiving the full input information and separately perform their computation. All their output values are then combined into an output vector. This vector being a combination of 0 and 1, it can encode more complex information, for example integers expressed in term of bits. Each neuron can thus learn to predict its corresponding bit of the target output integer following the same scheme detailed in the previous section (Rumelhart et al., 1986).

The Perceptron algorithm represents a progress in term of performance compared to a single neuron. Unfortunately, it has also strong limitations. First, one has to find a proper way to encode the output information into a binary vector. Each neuron only being able to make a linear separation, this implies that the encoding must be done such that each possible output values are linearly separable from the others in the output space. Also, for complex information the required number of neurons can increase exponentially.

Complex neural networks : linked neurons

Multi Layer Perceptron

The limits of the Perceptron algorithm can all be lifted using deep artificial neural networks like the Multi Layer Perceptron (MLP). Compared to the Perceptron, the MLP is based on activation functions that are continuous and differentiables. These functions are also respecting the constraints of having two separate states as their limits. It follows that even one neuron can now output a real number instead of just a boolean value, simplifying the network architecture when handling regression problems. The most famous activation function typically used in MLP is the sigmoid function (Rumelhart et al., 1986) :

a(S) = 1 1 + e -βS (4.3)
where β is an tunable parameter often set to 1. More critically than the modification of the activation function, the main MLP improvement compared to the Perceptron comes from its multiple neural layers. More precisely, neurons are now divided into layers with neurons from one layer taking as inputs all the outputs from the neurons from the previous layer5 . In terms of semantic, the last layer of neurons that computes the desired output is called the output layer while all intermediary layers are called hidden layers. Also, though not composed of neurons, the input data is known as the input layer. Due to its multiple layers, the MLP can effectively apply a sigmoid function on a linear combination of outputs of sigmoid functions, generating non-linear approximations. With each layer, the mathematical complexity of the MLP approximation function increases and allows to fit the reference relation between input and output with as much precision as necessary. Alternatively, it is proven by the Universal Approximation Theorem (Cybenko, 1989) that with only one hidden layer and a sufficiently large number of neurons, a MLP can approximate any function as accurately as desired.

Backpropagation algorithm

With the addition of a layer structure, the weights actualization procedure needs to be modified as only neurons of the output layer directly possess the information about the target output Y ref . The primary algorithm, from which any other is derived, is called the "Backpropagation" algorithm (Rumelhart et al., 1986;[START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF][START_REF] Marsland | Machine Learning: An Algorithmic Perspective[END_REF] and enables a recursive error gradient descent computation from the output layer to the first hidden layer. The algorithm depends on the choice of cost function. A simple and usual choice is :

C(Y, Y ref ) = 1 2N N i=0 (Y i -Y i,ref ) 2 (4.4)
with N being the number of neuron in the output layer. For a given layer L the weights are still modified following the gradient descent process of equation 4.2. The underlying idea behind the backpropagation algorithm is that it is easier to compute the weights updates of one layer when the weights updates of the next layer have already been computed. Accordingly, the backpropagation algorithm starts by computing weights updates for the output layer, for which the partial derivatives of the cost function are straightforward to compute and uses the results to compute weights updates of the previous layer. It then applies the same process iteratively. Mathematically, for the weight ω L i,j that is applied by the j-th neuron of layer L on its i-th input data, the gradient ∂C ∂ω L i,j can be subdivided into :

∂C ∂ω L i,j = ∂C ∂a L j ∂a L j ∂S L j ∂S L j ∂ω L i,j (4.5)
where S L j and a L j are, for the j-th neuron of layer L, respectively, the weighted sum of the inputs defined in equation 4.1 and the activation function. Let us define δ L (j) ≡ ∂C ∂S L j the local error term for the j-th neuron of layer L. We can re-write the update of weights ω L i,j as :

ω L i,j = ω L i,j -η ∂S L j ∂ω L i,j δ L (j) (4.6)
where δ L (j) can be computed following :

δ L (j) = ∂a L j ∂S L j ∂C ∂a L j = ∂a L j ∂S L j N L+1 k ∂S L+1 k ∂a L j ∂C ∂S L+1 k = ∂a L j ∂S L j N L+1 k ω L+1 j,k δ L+1 (k) (4.7)
In the previous equation N L+1 is the number of neuron in layer L + 1. For the reference case where the activation function of each layer are the sigmoid, the error function is the sum of the squared errors and there is N t layers, the previous equations can be simplified as :

∂S L j ∂ω L i,j = a L-1 j , ∂C ∂a N t j = a N t j -Y j and ∂a L j ∂S L j = βa L j (1 -a L j ) (4.8)
With this particular choice of activation and error functions, increasing the number of hidden layer quickly raises a problem, named "vanishing gradient" problem. Using equation 4.5 to propagate the error, the local error δ L is multiplied by

∂a L j ∂S L j
which for a sigmoid is mostly inferior to 1. Thus the amplitude of the error gets smaller with each layer and weights of the first hidden layer might not be updated at all. Two approaches are usually used to alleviate this problem. The first one is to restrict the network to one hidden layer, which can still be efficient when remembering the Universal Approximation Theorem. To avoid the exponential increase of neurons in the single hidden layer and also to develop more refined architectures with peculiar (convolutional, locally-connected ...) layers, the second approach is to use other activation functions than the sigmoid that prevent this vanishing gradient issue.

Learning "Hyper-parameters"

Over-training problem and multiple set

When a machine learning algorithm has been correctly built to fit its application case and shows an improvement of its prediction capabilities, it is necessary to consider one last issue referred to as over-training. Over-training happens when, after approximating the general relation between the inputs and outputs, the network starts to learn the specificities of the data set it is trained on. Usually, even large data sets are not large enough to correctly account for the highly complex relation between the input and the output. It is, in essence, a problem similar to the interpolation by a high order polynomial. The function will fit with a tremendous precision each point of the data set but will have totally erratic fluctuations between each one.

To stop the learning process before over-training when its generalization capability is the best, the data set is split into three subsets. The first set is the learning (or training) data set on which the network is trained and the weights updated. The second is the validation set, used regularly during the learning process to compute an error but has no influence on the weights update. Finally the test set is used after the learning process to better quantify the generalization ability of the network.

Usually all three data sets are created before the learning process and do not change after. Sometimes though, the validation and training sets can be repeatedly merged and their data randomly re-attributed to either of them following a method called cross-validation. When the number of data is small, only two subsets can be built : the training set and a test set that takes the role of both previously defined test and validation sets.

To avoid over-training, both the prediction error on the learning and validation sets are monitored during the learning process. When the validation set error stops decreasing and starts increasing again while error on the learning set continues to decrease, it means that the network is now learning the training set specificities and is worsening its generalization ability. One thus needs to use the network weights when the error on the validation set was the lowest to avoid this over-training issue. Other methods to prevent over-training will be detailed in section 4.6.2.

Learning rate

When considering the various parameters that rule the learning process of a neural network, also known as hyper-parameters, the most obvious one is the learning rate η, shown in Equation 4.2. It quantifies the amplitude of the modification that will be applied to the weights at each weights updates during the learning process. Consequently, if η is too high, the network will greatly modify its weights with each update. The learning process will be fast but it might True Function Over-trained model entirely miss a narrow but deep minimum, and will in any case jump around a well without settling at the bottom. On the contrary, if η is too low, the learning process will be gradual and more stable. However the network might be stuck in a local minima and, in any cases, the learning process will be significantly longer.

To solve this issue of determination of η most learning algorithms nowadays adapt the value during the learning process. Usually, η starts with a high value to quickly advance toward the area where the error is seemingly low. The value of η is then gradually decreased to allow convergence into the possible narrow minima of such regions of interest. This adaptive learning rate will be discussed in section 4.6.3.

Weight initialization

Another important issue that might critically affects the learning process if not addressed is the initialization of the neurons weights. If we consider a naive initialization where all weights of all neurons are set to be equal, then the output a(S) of each neuron will be the same. Consequently, for any value of the error function, the backpropagation algorithm will apply the same modification on all weights and they will stay equal, dramatically decreasing the effective number of parameters of the network and its efficiency.

To avoid this issue, the weights have to be randomly initialized but this will not ensure by itself an efficient learning process. When looking at equation 4.2, we see that if the amplitudes of the weights are too large, a divergence may happen in the learning process when the error is large too. Conversely, if the weights are too small, the modification will be negligible and the learning capabilities of the network will hugely decrease. Many studies have been published on what the exact amplitude of the weight initialization should be but there is no universal solution as it strongly depends on the activation function and the depth of the neural network. A general tendency is however to follow the prescription from [START_REF] Glorot | Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics[END_REF] (see also [START_REF] He | Proceedings of the IEEE International Conference on Computer Vision[END_REF]) where, for a given layer, the random weights initialization follow a random uniform law in the range [-1/ √ N , 1/ √ N ] where N is the input dimension for this layer.

The effect of the weights random initialization and of the stochastic learning of section 4.6.1 is to introduce some randomness in the learning process. It means that, with the same network architecture and the same data sets, two trained networks can have performances that marginally differ. It is especially the case when no regularization is included to the learning process (see section 4.6.2). To really compare two neural networks with different hyper-parameters, one must therefore perform several independent training of each architecture to average out this noise.

Data set size and normalization

The Universal Approximation Theorem states that with only one hidden layer and a sufficiently large number of neurons, a MLP can approximate any function. It might thus be tempting to increase the number of neurons when looking to improve a network performance. However, more than the increased computational time, the real limiting factor is the data set size. Indeed, if there are more weights (i.e. free parameters) in the network than data (i.e. constrains) in the data set, the network will be widely under-constrained. The resulting trained network might be unstable and be particularly sensitive to over-fitting problems, detailed in section 4.5.1. It might therefore make widely inaccurate predictions for data it has not been trained with. A general rule to avoid this effect is to have an effective6 number of weights in the network at least one order of magnitude smaller that the number of examples in the data set. To increase performance for a fixed number of weights, deeper architecture might be of use at the risk of having a vanishing gradient issue.

The previous section outlines that the quality of a learning process mostly depends on the amplitude of the weights update. In this regards, the values taken by the input data of the data set are as important as weights initialization. These inputs must be scaled such that they are of the same order of magnitude as the range where the activation function has a linear behavior. The usual scaling, associated with the sigmoid activation function, is to normalize the data so that they have zero mean and are included in the interval [-1, 1]. It is especially relevant for input data of various order of magnitudes. In such case, the higher input will numerically be dominant in the final result. Thus, most of the weights update will be made to better link this peculiar input to the output at the cost of the other inputs.

Learning refinements

Improved gradient descent algorithms

Standard Gradient Descent

The regular Gradient Descent algorithm (also known as Batch Gradient Descent) uses the information from all the data set. It first makes a prediction for every element of the data set. All individual errors of the batch are then summed and only one weight update is performed using the resulting global error. As all elements of the data set are considered at the same time, their order is not a problem with such algorithm. Using regular Gradient Descent allows to perform one weights update by learning step and use all the available information. However, it is computationally costly because the error function must be computed for all examples of the data test before a single weights update can be applied. Furthermore, with the regular gradient descent algorithm, there is no exploration of the weights space as it will strictly move according to the local slope of this space.

Stochastic Gradient Descent

Conversely to regular Gradient Descent, the Stochastic Gradient Descent (SGD) algorithm performs a draw to select only one element of the data set to learn on. It then computes the error function for this element and performs a weights update based on this error.

The random draw mechanism prevents ordering problems. Indeed, if the elements of the data set are ordered in some ways then the first data to be considered will represent a biased subset of the overall set and the network will first learn to predict this biased subset rather than the more general relation. With most learning methods decreasing their effective learning rate with learning iterations, the network might quickly converge to the configuration that only favors the predictions of the first data and not be able to find the global minimum for the whole data set in the weights space.

The SGD algorithm is a very noisy scheme because the minimum of the error function in the weights space is different for every element of the data set. At each iteration, the algorithm therefore makes the weights converge to the minimum for the current example rather than to the global minimum when the error is computed on the entire data set. By design, the SGD algorithm reduces the computational cost of one weights update but, due to its noisy nature, it needs more weights updates before converging.

Mini-batch Gradient Descent

Mini-batch Gradient Descent is a scheme that mixes both the previous approaches into one that avoids their drawbacks. It first splits the data set into subsets of a tunable size, named minibatch. For a given mini-batch, it makes a prediction of all the elements and sums their individual errors and uses the sum to perform a weights update. The algorithm then selects another minibatch and repeats the process until all mini-batches are viewed. To improve convergence stability, the algorithm can either look at all mini-batches in a random order or re-construct new minibatches at each learning step by randomly picking elements. With either of this methods, the algorithm can prevent data set ordering issue like SGD algorithm. However, it is a considerably less noisy process than SGD because of the error averaged on multiple elements that is closer to the global error averaged over the whole data set than an error computed on only one element. Due to its overall efficiency in term of computing time and converging capability, the Mini-batch Gradient Descent is by far the most popular scheme in the machine learning field.

Regularization

Regularization methods were developed as a way to prevent over-training issues described in section 4.5.1. The goal of regularization methods is to ensure that the training performed on the provided learning set produces a network with performances that remain good for any other input (i.e with good capacity for generalization). It is especially relevant for deep learning cases where the number of weights is high and problems are often under-constrained. Thus regularization methods usually either add further constraints to the weights, decrease the effective number of weights or artificially increase the data set size.

Weight decay and L1/L2 regularization

As established in section 4.5.3 the learning process is more efficient when weights are small enough for the activation function to be used in its linear regime. For the sigmoid activation this linear regime is around 0. Even if the weights are initialized with a well-behaved distribution, they can increase during the training process and lead to use the activation function outside this linear regime. It is, thus, common practice to add a weight decay (Hanson & Pratt, 1989;Kingma & Ba, 2014) to the learning process that equally decreases the value of all weights.

Weight decay effectively adds constraints to the problem, hence acting as a regularization. It can simply be implemented through a factor λ < 1 that is applied to all weights at the end of each learning step, in such case equation 4.2 is modified into :

ω i ← (1 -λ) ω i -η ∂C ∂ω i (4.9)
This process also prevents over-training to some extent [START_REF] Krogh | [END_REF][START_REF] Hastie | The Elements of Statistical Learning[END_REF]. Usually, weight decay is based on the addition of the weights norm (either L1 or L2) into the cost function, the quantity to minimize becoming :

C(Y, Y ref ) + λ i j (ω 2 i,j ) Ln (4.10)
with λ the hyper-parameters specifying the importance of the regularization term and L n = 0.5 L1-regularization and 1 for L2-regularization. This type of regularization is used extensively due to its affordability. It can easily be applied to other supervised learning methods than neural network as it only consists of adding a sum of weights to the overall quantity to minimize. For example the regression techniques detailed in chapter 5 also uses the same principle to add constraints to the problem. Data augmentation is a process that artificially increases the number of examples in the data set. It can only be applied to inputs that have an intrinsic symmetry (e.g. scale-invariance, rotational symmetry, translational symmetry, ...). For example, in image recognition, there is often a rotational invariance as the picture represents the same object even when it has been rotated. In this case, it is thus possible to create new data by rotating the existing ones. With data augmentation, the number of examples in the data can be multiplied many times, greatly increasing the number of constraints to the problem. It is extensively used for classification problems but significantly less for regressions where the inputs often have less symmetries. For example, some astrophysical signals or statistical quantities can hardly be dilated, scaled or translated while still corresponding to the same source.

Data augmentation

Dropout

Dropout refers to temporarily discarding some of the weights of the network and only performs the learning process on the remaining ones. At each learning step, all weights are re-activated and new ones are selected to be temporarily discarded. Dropout can usually be implemented in two different ways. First, for a given layer each neuron has a probability p to be dropped out, p being the given dropout fraction. The other dropout method consists of keeping all neurons but a fraction p of their weights are dropped out at each iteration.

Whatever the dropout method selected, the effective number of weights during the learning process is decreased by a significant amount. With fewer weights, the problem is better constrained and thus regularized. Also, the learning speed is increased as less weights updates are effectively done. Regularization by dropout prevents having few weights that convey most of the information while the effects of the other weights are negligible. It leads to a more stable learning, as the prediction relies on multiple weights computations that can averaged out an unexpected output from one neuron.

Adaptive learning rate

Momentum conservation

To obtain an effective adaptive learning rate, one solution is to add "momentum" to the learning process (Qian, 1999). It consists of considering the weight update of the previous learning step in the computation of the weight update of the current one. More precisely, if ∆ω t i,j is the weight update at step t, at step t + 1 we will have :

∆ω t+1 i,j = (1 -α)η ∂C ∂ω t i,j + α∆ω t i,j (4.11)
where 0 < α < 1 is the relative amplitude of the momentum. Such momentum allows the learning process to keep a global direction of modification during the first steps of the learning process where it might be far from the global minimum in the weights space. The high momentum during the first steps also allows the training to converge faster while also helping to avoid local minima. It is useful for gradient descent algorithms using small mini-batches before weights updated. The global direction and amplitude is kept even if it trains on a biased subset with a minimum far from the global data set minimum.

RMSProp : a momentum-based learning algorithm

The momentum characteristic is usually further refined in common learning algorithms. The RMSProp algorithm (Dauphin et al., 2015;Murugan & Durairaj, 2017) that is used in our following study adds spatial considerations about the weights space into the learning process.

More precisely, at each step t + 1 and for each weight w i,j it will independently compute a momentum function M t+1 i,j following :

M t i,j = αM t-1 i,j + (1 -α) ∂C ∂ω t i,j 2 (4.12)
where α is an hyper-parameter that can again be considered as the momentum amplitude. The associated weight update at step t + 1 is then expressed as :

ω t+1 i,j = ω t i,j - η M t i,j + ∂C ∂ω t i,j (4.13)
with an arbitrary low parameter to avoid divergence. From the two previous equations, we can see that

η M t i,j + ∂C ∂ω t i,j ∝ sign( ∂C ∂ω t i,j
). RMSprop algorithm thus effectively normalizes the gradients, which is a key asset in two learning situations. will be high. Therefore this weight update for a regular SGD algorithm will be high and the algorithm will jump around the well without settling at the bottom or else very slowly. However, as RMSprop does not depend on the gradient values but only its sign the weight update will be moderate and the algorithm will come closer to the minimum along this axis. Conversely, if the minimum is shallow but far away along a given weight axis, ∂C ∂ω t i,j will be low. The SGD algorithm will thus perform a low amplitude weight update and only slightly comes closer to the minimum. RMSprop, not being affected by the low ∂C ∂ω t i,j value, will perform a moderate weight update, converging faster toward the minimum.

A comparison between RMSprop and SGD for a toy model with only two weights is shown in figure 4.5. In a general case, RMSprop will thus converge faster to the minimum than regular gradient descent method, requiring fewer learning steps.

Batch Normalization

The regular normalization of the input data ensures that the first neural layer has normalized inputs on which the learning process is efficient. However, the outputs of this first layer have no reasons to still be normalized and we can expect a degradation of the normalized distribution when going forward into the network. To go further, and provide all layers with a scaled or normalized input, one can apply what is called a batch normalization (Ioffe & Szegedy, 2015). With batch normalization, after each layer there is a re-normalization (computed on the currently considered data batch) of the outputs. Furthermore, the output is then re-transformed (re-scaled and re-shifted) using learnable parameters. This learnable parameters will be adjusted during the learning process to provide each layer with a distribution whose mean and variance is the

Cost function

The cost function (also known as error function or loss) quantifies the difference between the predicted output and the target one from the data set. It is one of the building blocks of the backpropagation algorithm. As presented before, the most commonly used cost function is the mean squared error, in equation 4.4. Such an error function shows limits in regression problems when the various outputs to predict are of different orders of magnitude. The intrinsic problem is that the computed error is absolute, meaning that an error of 10% when predicting an output of the order of 1 will negligibly increase the loss compared to an error of 10% when predicting an output of the order of 10 5 . During the learning process, the algorithm will thus focus on reducing the prediction error on the latter output at the expense of the former that will be more and more inaccurately predicted. Intuitively though, in such a case we want all output values predicted with the same order of relative error. One way to do it is to use, like in our following study, the logarithmic squared error defined as :

C(Y, Y ref ) = 1 N N i=0 [log 10 (Y i,ref + 1) -log 10 (Y i + 1)] 2 (4.14)

Improved supervised learning methods for EoR parameters reconstruction

This Chapter strictly and uniquely contains the article "Improved supervised learning methods for EoR parameters reconstruction" whose authors are Aristide Doussot, Evan Eames and Benoit Semelin. It has been published in September 2019 in The Monthly Notices of the Royal Astronomical Society (Doussot et al., 2019). As such, some elements of context described in this chapter might have already been described in part I of this manuscript.

Abstract

Within the next few years, the Square Kilometer Array (SKA) or one of its pathfinders will hopefully detect the 21-cm signal fluctuations from the Epoch of Reionization (EoR). Then, the goal will be to accurately constrain the underlying astrophysical parameters. Currently, this is mainly done with Bayesian inference. Recently neural networks have been trained to perform inverse modelling and, ideally, predict the maximum-likelihood values of the model parameters.

We build on these by improving the accuracy of the predictions using several supervised learning methods: neural networks, kernel regressions or ridge regressions. Based on a large training set of 21-cm power spectra, we compare the performances of these methods. When using a noise-free signal generated by the model itself as input, we improve on previous neural network accuracy by one order of magnitude and, using a local ridge kernel regression, we gain another factor of a few. We then reach an accuracy level on the reconstruction of the maximum-likelihood parameter values of a few percents compared the 1σ confidence level due to SKA thermal noise (as estimated with Bayesian inference). For an input signal affected by a SKA-like thermal noise but constrained to yield the same maximum-likelihood parameter values as the noise-free signal, our neural network exhibits an error within half of the 1σ confidence level due to the SKA thermal noise. This accuracy improves to 10% of the 1σ level when using the local ridge kernel. We are thus reaching a performance level where supervised learning methods are a viable alternative to determine the maximum-likelihood parameters values.

Introduction

It has been recognized for more than 20 years that the neutral hydrogen in the Inter-Galactic Medium (IGM) before and during the process of reionization of the universe must have emitted radiations at 21 cm that, redshifted at meter wavelengths by cosmic expansion, should be observable nowadays with adequate radiotelescopes (Madau et al., 1997;Furlanetto et al., 2006;Pritchard & Loeb, 2012;Mellema et al., 2013;[START_REF] Koopmans | [END_REF]. The main difficulty in detecting this signal is to separate it from various types of foreground emissions (galactic synchrotron, extragalactic point sources, etc., see e.g. Di [START_REF] Matteo | [END_REF]Jelic et al., 2008). Single dipole instruments can measure the intensity of the signal as a function of frequency integrated on the sky (global signal). Such observations have the advantage that the signal-tonoise ratio does not depend on the collecting area, but the drawback that the limited amount of collected information gives us less leverage to separate the signal from the foregrounds and encodes less knowledge about the underlying astrophysical processes. A tentative first detection of the global signal has been reported by Bowman et al. (2018) with the EDGES instrument.

It is likely that the cosmic origin of the detected feature can only be ascertained with future interferometric observations that would quantify the angular fluctuations in the detected feature (in the form of a power spectrum).

A number of instruments have been attempting to measure the power spectrum of the signal, although mostly at higher frequency (and thus lower redshift) than the EDGES detection. Only upper limits have been established so far, at various wavenumbers and redshifts (Paciga et al., 2013;Beardsley et al., 2016;Patil et al., 2017;Ali et al., 2018). With this type of observations, unambiguous separation of signal and foregrounds should be possible but the calibration of the instrument is a much more difficult task. Also, the higher information content of the measured quantity comes with the requirement of a large collecting area to improve the signal-to-noise ratio. As the methods to perform the calibration improve, we may see a first interferometric detection in the next few years. Then, next generation instruments such as SKA and HERA should be able to measure the power spectrum much more accurately, detect it at lower frequencies (higher redshifts) where the foregrounds are stronger, and even, in the case of SKA, image the signal in three dimensions.

There is obviously a trade off between the amount of information in a type of observation (global signal, power spectrum, imaging) and the collecting area and integration time required to perform it with a good enough signal-to-noise ratio. But in all cases, transforming this information into astrophysical or cosmological knowledge is not a straightforward process. Indeed, the local intensity of the signal depends, in some cases non-linearly, on the hydrogen density, ionization fraction, velocity, kinetic temperature and on the local Lyman-α radiation field (see Furlanetto et al., 2006). These quantities are in turn correlated in a non-trivial manner through the process of structure formation. Thus the first crucial step in interpreting the signal is to build a model that can compute the signal from such basic processes as growth of density fluctuations, formation of sources of radiations (stars and AGN), and radiative feedback of the sources on their direct environment and on the IGM. These models can be analytical (e.g. Barkana & Loeb, 2005a;Pritchard & Furlanetto, 2007), semi-numerical (e.g. Thomas et al., 2009;Santos et al., 2010;Mesinger et al., 2011;Fialkov et al., 2014;Ghara et al., 2015), or the result of radiative transfer cosmological simulations (e.g. Gnedin & Shaver, 2004;Mellema et al., 2006;Valdes et al., 2006;McQuinn et al., 2007;Baek et al., 2009, and subsequent works). In all cases the models will requires the use of astrophysical parameters to describe processes either not implemented ab initio and/or below the resolution of the computation. A simple example is the efficiency of star formation, that would require a mass resolution below 1 solar mass and an extremely short time step to be modelled self-consistently.

The second important step in extracting astrophysical knowledge from the observation, is to use reliable statistical methods to put constraints to the models astrophysical parameters. A number of such methods exists. Pober et al. (2014) used the Fisher information matrix to derive confidence intervals for the parameter values, [START_REF] Greig | [END_REF], 2017b, 2018) use Bayesian inference enacted by Markov Chain Monte Carlo (MCMC) with the semi-numerical code 21cmFAST. As even using 21cmFAST for MCMC Bayesian inference is computationally expensive, Kern et al. (2017), Schmit & Pritchard (2018) and Jennings et al. (2019) build an emulator of the code, using Gaussian Processes and Neural Networks. Another approach to parameter estimation is to train supervised learning algorithms to perform inverse modelling, taking some representation of the observed signal as an input and directly predicting the param-eter values. As in the case of building an emulator, the training is specific to the chosen model, and typically requires a smaller number of modelling runs as MCMC inference does. For now, neural networks trained for inverse modelling have been implemented using the power spectrum (Shimabukuro & Semelin, 2017) or the full tomographic data (Gillet et al., 2019) as input, to predict best-fit parameters values. Predicting confidence levels could be done in various ways, for example using Bayesian neural networks. It is not obvious at this stage, however, that the predicted confidence levels would have the exact same meaning as in classical Bayesian inference.

When predicting best-fit parameters using neural networks (or other supervised learning algorithms) trained to perform inverse modelling, an error exists, due to the imperfect training of the network. This training error can be exactly computed if the test input signal was produced by the model itself (then we know the corresponding true parameters). Note that finding parameter values that do not perfectly match a test signal not produced with the model is an issue not specific to supervised learning methods: maximum-likelihood parameters with a low likelihood value indicate an imperfect model. In any case, for supervised learning methods to be actually usable, we need to ensure that the training error is much smaller than the typical 1-sigma confidence due to the thermal noise in the target observation, as estimated by Bayesian inference. This should of course be true if an noise-free signal is fed to the network, but also if a noised signal is considered. Such was not really the case in Shimabukuro & Semelin (2017) where the error is of the same order as the thermal noise, or in Gillet et al. (2019) where only a noise-free signal is considered. Thus we need to improve the performance of supervised learning method implementations, either by improving the implemented methods, or exploring new ones.

In this work we explore both of these possibilities. First, we improve substantially on the performances reached in Shimabukuro & Semelin (2017) using neural networks, by using a larger learning set, and optimizing several steps in the process. Then we explore another supervised learning method. Neural network have encountered great success when dealing with image classification. In this situation, the dimension of the signal space is huge, typically of the order of 10 6 , the number of pixels in the image. In our case, the signal is the value of the power spectrum at various wavenumbers and redshifts. The dimension of the signal space is much lower, typically of the order of 10 2 . In such comparatively low dimensions, advanced versions of the classical linear regression are known to perform well. Indeed, the linear regression, using the knowledge from a set of samples to approximate a model with a linear relation, can be classified as supervised learning. As 10 2 dimensions is still very large to apply the classical linear regression, kernel regression and ridge regressions have been developed [START_REF] Hastie | The Elements of Statistical Learning[END_REF]. In this work we combined these improved regression methods and push them as far as we can in terms of performance to compare them with the neural network approach.

The layout of this chapter is as follows. In section 5.3 we present the case to which we apply supervised learning: the input signal and thermal noise, the parameters to be predicted and the model that relates them in the case of forward modelling. In section 5.4 we detail the different supervised learning methods studied in this work. In section 5.5 we study the accuracy of these methods in term of the error on the reconstructed parameter values. In section A.1.5 we present our conclusions.

Framework

The model: 21cmFAST

The learning process of any supervised learning method requires a training set consisting of a sufficient number of cases (where both inputs and outputs are known). Generating such a number of cases is currently beyond the reach of full-numerical simulations designed to predict the 21-cm signal. Consequently, we have selected the semi-numerical code 21cmFAST (Mesinger et al., 2011) which is fast enough to provide the required number of cases in a reasonable amount of time. Let us briefly review some salient features of 21cmFAST numerical methods.

The main feature is that 21cmFAST does not include full radiative transfer, thus saving a lot of computation time. Instead, the ionization process is based on the "excursion-set" approach (Furlanetto et al., 2004;Mesinger & Furlanetto, 2007). The basic principle is that if the number of ionizing photon produced in a region is larger than the number of neutral hydrogen atoms in the same region, the region is considered ionized (in practice, only the region center cell is tagged as ionized, as regions centered on all cells will be considered). The photon production rate is assumed to be proportional to the collapse fraction (fraction of baryons in a collapsed object). At each location, the collapsed fraction smoothed on scale R, f coll (x, z, R), is compared to an efficiency parameter ζ ion . The comparison is performed for decreasing R values, from a large scale R mfp to the cell size

R cell . If f coll (x, z, R) > ζ -1
ion then the center cell of the region is flagged as ionized. Finally, at R cell , the ionizing fraction of the remaining cells that are not fully ionized is set to be ζ ion f coll (x, z, R cell ). See Mesinger et al. (2011) for further details.

Another cost-saving strategy implemented in 21cmFAST is to ignore baryonic dynamics, and use simplified dark matter dynamics. The dark matter density field is linearly extrapolated from the primordial field using the standard Zel'Dovich approximation (Zel'dovich, 1970). Baryons are simply assumed to track the dark matter exactly. See Mesinger & Furlanetto (2007) for further details. X-ray heating and Lyman-α contributions to the spin temperature of hydrogen are implemented in 21cmFAST, again using cost-saving strategies. However we deactivated these processes in our study, setting ourselves in the high spin temperature limit.

Selected EoR observables and model parameters

In our approach to supervised learning where our goal is to put constraints on model parameters using observables, the observables are the inputs of the method and the parameters values are the outputs. Let us specify which inputs and outputs have been used in this work.

EoR observable

In our study, we chose to focus on the power spectrum of the intergalactic 21-cm signal, assuming that the non-gaussianities (Shaw et al., 2019) of the signal are not necessary to accurately reconstruct the parameters. More precisely we chose to consider the values of the power spectrum at 12 different wavenumbers k, logarithmically sampled from 4.42×10 -2 cMpc -1 to 3.20 cMpc -1 , for integer values of the redshift z from 5 to 15. Then, the signal that is used as an input lives in a space of dimension 120. This choice allows us to work in relatively low dimension unlike, for example, Gillet et al. (2019) who deal with the full information from the lightcone using convolutional neural networks. While neural networks have shown their ability to deal with high dimensional signals (dimension 10 6 ) when analysing images for example, other supervised learning method, such as the different flavors of linear regression presented here, are well suited to lower dimensionality.

Choice of EoR parameters and sampling

Concerning the EoR parameters that we want to reconstruct, we have chosen three parameters that have often been considered in other works [START_REF] Greig | [END_REF], 2017b, 2018;Schmit & Pritchard, 2018;Eames et al., 2019):

• ζ ion accounts for the ionizing efficiency of high-z galaxies and can be expressed as:

ζ ion = 30 f esc 0.3 f * 0.05 N γ 4000 2 1 + n rec (5.1)
with f esc the ionizing photon escape fraction, f * the fraction of galactic gas in stars, N γ the number of ionizing photons produced per baryon in stars, and n rec the typical number of times a hydrogen atom recombines during the EoR

• R mfp is the mean free path of ionizing photons within the ionized regions, regulated by the existence of unresolved Damped Lyman-α systems.

• T vir is a mass threshold above which halos are allowed to form stars and begin ionizing their surroundings.

Detailed definitions of these parameters are given in [START_REF] Greig | [END_REF].

We based our study on a learning set of 2400 labelled cases, generated for our previous study in Eames et al. (2019), corresponding to the nodes of a logarithmic 20×6×20 grid in the parameter space (ζ ion ; R mfp ; T vir ) with the following boundaries :

-ζ ion ∈ [20, 200] -R mfp ∈ [5 cMpc, 35 cMpc] -T vir ∈ [8.0 × 10 3 K, 10 5 K]
Let us emphasize that this sampling method by no means ensures a maximization of the information. Methods that optimize the sampling (for a fixed number of cases and fixed explored volume in parameter space) to maximize the information are presented in Eames et al. (2019) and appear to lead to a better training of, at least, neural network methods. Further details on the setup of the 21cmFAST runs performed for each triplet of parameter values can be found in Eames et al. (2019).

SKA noise modelling

For supervised learning methods designed to constrain model parameters to be of any use, they have to be able to handle a signal affected by the observational noise. We will concentrate on the thermal noise from the SKA, neglecting other possible sources such as imperfect foreground removal, residual calibration errors, or even sample variance. To model the expected thermal noise we consider the SKA specifications as detailed in [START_REF] Dewdney | SKA1 system baseline design[END_REF]. Following McQuinn et al. (2006), we write the detector noise covariance matrix as:

C(k i , k j ) = 1 Bt k i λ 2 BT sys A e 2 δ ij (5.2)
where B is the bandwidth, t k i is the effective observing time of the instrument in the grided visibility cell corresponding to wavenumber k i , λ is the observed wavelength, T sys is the total system temperature and A e is the effective area of the station. For the system temperature we have used T sys = 100+300 ν 150MHz -2.55 K (Mellema et al., 2013). Lacking data from a definitive design of the future SKA-Low antennas, we have used an effective area for a station composed of 256 antennas of: A e = 256 × min(2.56, λ 2

3 ) m 2 . We have assumed a bandwidth B = 10 MHz, and a station diameter of 35 m determining the field of view. Finally we have computed the t k i by integrating in visibility space the trajectories of the baselines from the SKA specifications. We considered 8h runs for a total integration time of 1000h, and a target field with declination -30 deg (close to the zenith for SKA-Low).

From the detector noise covariance matrix we can compute the 1σ uncertainty on the power spectrum due to thermal noise as:

δP 21 ∆T (k) =   |k|=k 1 Aex 2 y λ 2 B 2 C(k, k) 2   -1 2 (5.3)
where the sum extends over Fourier-space cells in the spherical shell with radius k (and also thickness ∆k = k in our case, which is an usual but determining choice), x is the comoving distance to the observed redshift and y the depth of the field (a distance) as determined by the bandwidth and the cosmology. The resulting level of noise is very similar to that in [START_REF] Koopmans | [END_REF].

See McQuinn et al. (2006) for further details on establishing the formulas. Once δP 21 ∆T (k) is computed for our binned wavenumbers, we simply add a realization of this noise to the signal to get a noised power spectrum.

Supervised learning methods

A well established way of predicting underlying astrophysical parameters using observables is Bayesian inference associated to Markov Chain Monte Carlo sampling. However, it often requires numerous instances of forward modelling to predict one observable, like in 21CMMC [START_REF] Greig | [END_REF], 2017b, 2018;Park et al., 2019) where the forward modelling is performed using 21cmFAST (Mesinger et al., 2011). This inherently comes with a high computational cost, even if some attempts on designing a fast 21-cm power spectrum emulator using gaussian processes (Kern et al., 2017;Jennings et al., 2019) or support vector machine (Jennings et al., 2019) to replace 21cmFAST have significantly accelerated the process.

With supervised learning methods trained to perform inverse modelling and predict parameter values, a typically smaller number of forward modelling instances is needed to build the learning set, decreasing the required computational time compared to 21CMMC. We chose to focus on neural networks, as it appears to be the fastest method in term of computational time, and on ridge and kernel regressions which have not been explored for this purpose before.

Common features

Learning set and test set

Although different, the two classes of supervised learning methods studied in this work share common features. In essence, the supervised learning material consists of a set of cases, from which the algorithm can interpolate to successfully make predictions for cases not in the set. This set of labelled cases is usually called the learning set in the neural network field. To quantify the prediction quality of a method, a second set of cases, distinct of the first one, is used. In the neural network field, this sample is often referred as the test set. Performing the evaluation on the test set avoids being impacted by the well-know issue of over-fitting on the learning set.

In our study the learning set is either made of 2400 signals for the cases without instrumental noise added (that were already described in Eames et al., 2019) or of 20 noised realizations of each signal, which means 48000 noised signals, when instrumental noise is taken into account. The test set is composed of 512 signals generated starting from random values of the three astrophysical parameters taken within the bounds of the grid-based learning set. When noise is included, we generate 40 realizations of each signal which leads to a test set composed of 20480 noised signals.

Limitations to absolute performance evaluation

Any supervised learning algorithm includes, in various forms, adjustable quantities, often called weights, that encode the computation of the outputs. The learning process therefore consists of adjusting the weights to accurately recover the known (labelled) outputs of the learning set, based on its inputs, by minimizing a given error function. The function to minimize usually depends on various adjustable hyper-parameters like a learning rate η or the weight decay rate λ. Changing the values of these hyper-parameters thus leads to different error functions, different minimization results and therefore different predictions. Optimizing the values of the hyperparameters is of paramount importance to obtain the best possible predictions. However it is almost impossible to make sure that a set of hyper-parameters values is a global optimum, especially with neural networks where there is an infinite number of possible architectures. The comparison between methods can only be done with parameter values that are, at best, local optima in the hyper-parameter space.

Also, our learning and test sets are generated with the same semi-numerical model: 21cm-FAST (Mesinger et al., 2011). Any conclusion that we reach concerning the accuracy of parameter reconstruction using different methods will only hold when applied to a real observed signal if the model is able to reproduce the observed signal. More quantitatively, in the 120-dimensional signal space, the signals produced by our 3-parameters model occupy a 3 dimensional manifold. The observed signal will not lie in this manifold unless the model is perfect. Even when noise is included, the performances of our parameter reconstructions methods are only evaluated close to this manifolds (at distances typically corresponding to a 1-sigma thermal noise). If the observed signal is at a distance equivalent to many sigmas, our conclusions cannot apply.

Preparing the data

Labelling a noised signal: theoretical issue

A difficulty appears when using a noised signal. For a learning set sufficiently dense in the (noise-free) signal space, two neighbouring noise-free signals could be altered with two different realizations of the instrumental thermal noise in such a way as to lead to the same noised signal. This is of course why Bayesian inference predicts a distribution of possible parameter values instead of a single value. We show this problem in figure 5.1a for a toy model with a 2D signal, composed of the power spectrum values at two wavenumbers k 1 and k 2 , and produced by a model with only one parameter θ. The purple line represents the manifold of all the possible signals produced by the model, the model-manifold for short. We illustrate that the noised signal P has been obtained in two different ways, one starting from the signal corresponding to parameter value θ 0 with a noise N i and the other starting from the signal corresponding to parameter value θ 1 with a noise N j .

However, simple versions of neural networks cannot produce a distribution of parameter values as an output, only a single value (note however that Bayesian neural networks exist and could tackle the issue). To correctly evaluate the prediction ability of the supervised learning methods such as these single-value-output networks, it is therefore necessary to specify a single correct parameter value corresponding to a noised signal. The most logical answer is to decide that the correct parameter is the parameter corresponding to the noise-free signal on the modelmanifold closest to the considered noised signal. If a natural distance is chosen in the signal space (e.g L 2 norm), this minimal distance corresponds to adding the most likely realization of the thermal noise. This is in essence the maximum-likelihood value for the parameter (because the thermal noise is a Gaussian multivariate distribution). In the toy model of figure 5.1a, we note this most probable parameter θ . When we give to our methods the noised signal P , we thus expect them to predict the parameter θ .

Finding this most probable parameter value for a given noised signal in the general case is the very purpose of methods that derive maximum-likelihood parameter values. We do not know how to do it at low cost when building our noised training and test sets. We are however able to do so in specific cases: when the (noise-free) signal of the model manifold closest to the noised signal under consideration belongs to our learning or test sets (for which we do know the exact parameters values). Such a noised signal belongs to a 117 dimensional hyper-plane of the signal space which is orthogonal (as defined by the chosen distance in signal space) to the model-manifold at the corresponding noise-free signal's position. Graphically, we show a finite number of these perpendicularly noised signals for the two signals corresponding to θ 0 and θ 1 in P (k 1 ) In essence we are training our algorithms to perform an orthogonal projection onto the model manifold. In doing so, we have to restrict our sampling to specific noised signals for the learning and test sets. We have to trust the algorithm to interpolate when we give it a general noised signal and perform this same orthogonal projection. This ability to interpolate (or generalize, using the coined word in the machine learning community) is the very purpose of machine learning.

P (k 2 ) θ 0 P =θ 0 +N i = θ 1 +N j θ N i θ 1 N j (a) P (k 1 ) P (k 2 ) θ 0 θ 1 (b)

Disentangling the inversion algorithm error from the thermal noise uncertainty

The necessity to choose a correct label (i.e. parameter value) for the signals in our learning and test sets comes with a crucial beneficial side effect. If everything worked perfectly, our inversion algorithms would predict the exact same parameters values for a given noise-free signal affected by any different realization of perpendicularized noise. In practice this is not exactly the case as the learning process is not perfect. On the other hand, adding a general realization of the noise would displace the orthogonal projection on the model manifold, compared to the position of the original noise-free signal, thus changing the maximum likelihood parameter values.

Thus, using perpendicularized noise allows us to disentangle the error induced by the inversion algorithm itself on the reconstructed parameter value, from the intrinsic change to the maximum likelihood parameter values that the general-case thermal noise can induce. Moreover, the typical range of this intrinsic change, that is typically evaluated by Bayesian confidence intervals, gives us a scale: we want, if possible, the algorithm's error to be much smaller.

Generating the perpendicular noise

The perpendicularized noise realizations that have to be generated are perpendicular to the model-manifold at the position of the signal to which they will be added. Let us first consider the case when the signals have been generated at the nodes of a grid in the parameter space, like for our learning set. In this scenario, we can use finite differences to estimate the tangent hyperplane to the model-manifold, and thus the dual perpendicular space. For a signal P ζ i ion ,R j mfp ,T k vir corresponding to the parameters (ζ i ion ; R j mfp ; T k vir ), where i, j and k denote the indexes on the grid, we apply an algorithm whose main steps are the following :

1. Compute the vectors

V ζ ion ,i,j,k = P ζ i+1 ion ,R j mfp ,T k vir -P ζ i-1 ion ,R j mfp ,T k vir (5.4) V R mfp ,i,j,k = P ζ i ion ,R j+1 mfp ,T k vir -P ζ i ion ,R j-1 mfp ,T k vir (5.5) V T vir ,i,j,k = P ζ i ion ,R j mfp ,T k+1 vir -P ζ i ion ,R j mfp ,T k-1 vir (5.6)
that form a local basis in signal space generating the hyperplane tangent to the modelmanifold at signal

P ζ i ion ,R j mfp ,T k vir .
2. Orthonormalize the previous basis to obtain an orthonormal basis whose elements will be referred to as e 1,i,j,k , e 2,i,j,k and e 3,i,j,k 3. Generate a thermal noise N and compute

N ⊥ = N -N.e 1,i,j,k -N.e 2,i,j,k -N.e 3,i,j,k (5.7)
where . denotes the scalar product in signal space corresponding to our choice of distance (standard euclidean in our case, matching the L 2 norm).

Any such N ⊥ is locally perpendicular to the model-manifold. By construction, (ζ i ion ; R j mfp ; T k vir ) are the parameters values that we will teach the inversion algorithms to predict when fed with the noised signal

P ζ i ion ,R j mfp ,T k vir + N ⊥ .
The second case to consider is the case when to add noise to a signal that does not correspond to a node of a grid in parameter space, like in our test set. In this case we determine the tangent hyper-plane using a weighted average of the hyper-planes for the neighbouring nodes of the grid. The details of the procedure are described in the appendix.

It is worth noting that, with our method that use centered finite differences, only the local basis at signals which are not at the edges of our domain can be computed, therefore limiting the number of data that can be noised in our learning set to 1296 and in our test set to 258.

Data pre-processing

As already mentioned in section 5.4.2, considering the component of the noise perpendicular to the model-manifold does not completely lift the ambiguity in finding the signal on the modelmanifold closest to a given observed signal, because of the manifold curvature. If the distance to the manifold is larger than the manifold inverse curvature (radius of curvature in 2D) there may be several points (or even a continuity of points) on the manifold whose perpendicular hyperplane goes trough the noised signal, only one of them being the closest. Thus our method inherently has difficulty with observed signals far away from the manifold. If we are not careful, including a typical radio-interferometer thermal noise will automatically generate noised signals far from the manifold. Indeed, the model can easily (and does) generate the power-spectrum at large wavenumbers where the thermal noise is large. In the case of a typical SKA layout the thermal noise on the power spectrum typically increases as k 3 (see e.g. [START_REF] Koopmans | [END_REF]. Then, including large wavenumbers without caution will break our approach. Indeed, our orthonormalized basis does not match the basis consisting of the 120 Dirac functions centered on the (k, z) values where our power spectrum is evaluated. On the Dirac functions basis, the large thermal noise components are localized on a few vectors of the basis. But when projected on the orthonormalized basis it contaminates all components. This problem would be alleviated if the noise already had components of similar amplitude on the Dirac functions basis.

We arrive at the same conclusion by considering a general problem for supervised learning. If the fluctuations of a component of the signal are dominated by noise and not by variation due to changing values of the model parameter, this component will be of little help in constraining the parameters. The more relevant quantity to consider is of course the signal-to-noise ratio. This is equivalent to the traditional "inverse variance weighting" used in radio-astronomy imaging. With this operation, the contribution of the noise will be similar for all components of the preprocessed signal. Using this preprocessing step and feeding the result to the supervised learning methods, we give ourselves a better chance that the noised signal will remain close to the model-manifold. We indeed verified that this pre-processing step generally improves the accuracy of the predictions.

Kernel regression

Let us now describe advanced versions of linear regressions that we will use as supervised learning methods.

Linear regression

When addressing an interpolation problem, one obvious, yet useful method that exists is the simple linear regression, which consists of performing the following minimization:

min α,β j    N S i=1   y i -(α + N D j=1 β j x i,j )   2    (5.8)
where N S is the number of cases in the learning set, N D is the dimension of the input and y i is the output. This method results in a global linear approximation of the interpolating function which is then used to predict the outputs of the data of the test set starting from their inputs. It is also equivalent to approximating the model-manifold with a single hyper-plane.

It is therefore logical to think that, for each data of the test set, a local linear approximation of the interpolated function around the considered input will give better results. This is equivalent to locally approximate the model-manifold with an hyperplane, that varies depending on the location.

Kernel smoothing

Introducing locality in the regression leads to the class of supervised learning methods called Kernel smoothing regression methods [START_REF] Hastie | The Elements of Statistical Learning[END_REF]. To achieve this, to each case of the learning set participating in the regression we apply a weight:

K σ (x 0 , x i ) = 1 √ 2π e -D(x 0 ,x i ) 2 2σ 2
(5.9)

where σ is a scale hyper-parameter describing the desired level of locality and

D(x 0 , x i ) = N D j=0 (x 0,j -x i,j ) 2
is the distance in the signal space between the considered signal x 0 and the input signal x i of the learning set. The quantity to minimize is then min

α,β j    N S i=1 K σ (x 0 , x i )   y i -(α + N D j=1 β j x i,j )   2   
(5.10)

The parameters α and β j now depend on x 0 .

Global ridge kernel regression

If the kernel smoothing method considers the local information, it does nothing to deal with an eventual degeneracy of the problem. Imagine that two of the input values that constitute the signal are perfectly correlated in the noise-free case when varying one of the model parameters.

Then the prediction by the regression will be sensitive only to the average of the two β i coefficients corresponding to these two correlated values. The two β i could take very large values as long as the mean is correct. But then, when noise is added, which is uncorrelated at the two signal value, these two large β i value will induce a large variance in the predicted parameter values. What was here described in the case where noise is added can also occur when moving from learning set to test set as the two sets can exhibit different levels of correlation between two signal values. This problem can be alleviated by constraining the values of the coefficients, particularly relatively to one another (Hanke & Groetsch, 1998;Calvetti et al., 2000;[START_REF] Hastie | The Elements of Statistical Learning[END_REF]: this is what the ridge regression is about.

To implement this constraint we add a penalty term in our minimization which becomes min

α,β j    N S i=1 K σ (x 0 , x i )   y i -(α + N D j=1 β j x i,j )   2 + λ N D j=1 β 2 j    (5.11)
where λ is an adjustable hyper-parameter. This ridge regression basically shrinks the values of the coefficients by imposing a penalty on their size. The whole coefficient shrinkage process is comparable to the weight decay process used in neural networks and we can assimilate the hyper-parameter λ to a decay rate.

Local ridge kernel regression

One last step toward designing the most efficient regression is to consider the optimization of the two hyper-parameters σ and λ. It is likely that a global optimization of the hyper-parameters values on the overall domain of the signal space will result in a selection of mean values which enable most of the space to be correctly predicted but might critically fail in some area of the domain. One simple improvement is therefore to determine the best hyper-parameters values for each point of the test set, thus minimizing the quantity of Equation 5.11 for α, β j , σ and λ.

We implemented the local optimization by doing a simple grid search where we allowed our hyper-parameters to vary in a vast range of value:

-σ ∈ [2 × 10 1 , 2 × 10 6 ] -λ ∈ [1 × 10 -6 , 1 × 10 7 ]
This wide range appeared to be necessary as the hyper-parameters indeed took vastly different values depending on the location in the signal space. To illustrate this assertion, we present in figure 5.2 an histogram of the optimized hyper-parameters values computed from all the cases in the test set composed of noised signals. The brute-force grid approach to minimization is guarantied to find the global minimum in the explored domain. It is usable in our case because of the low number (two) of hyper-parameters. For a larger number of hyper-parameters, a minimization algorithm such as Levenberg-Marquardt should be used instead to achieve a reasonable computational cost, running the risk to converge to a local minimum. However, whether locally or globally, optimizing the value of the hyper-parameters when predicting the outputs of the test set requires us to already know the true value of the outputs to compute an error function between the predictions and the true results. This will not be possible with an observed signal whose associated model parameters are unknown. Still, by using this knowledge in the case of the test set, we exhibit the theoretical maximum accuracy of this method. The optimization of the hyper-parameters in a real case (i.e. with an observed signal) is an open problem but one reasonable solution is to adopt for the same hyper-parameters as for the closest signal (in signal space) in the learning set. An optimization of the value of the hyper-parameters for each point of the learning set is therefore needed. Obviously, the accuracy of the predictions on the test set will be worse when using hyper-parameter values optimized for the closest signal in the learning set than when optimizing on the test set signal itself. This gap may be reduced in the future by improving the strategy to chose the optimal hyper-parameters of an observed signal.

Artificial neural network

Network architecture

The principle of neural networks will not be discussed in depth as it has already been extensively described in various works (for examples in this field, see Shimabukuro & Semelin, 2017;Jennings et al., 2019) but we remind the reader that a neural network is basically composed of a set of calculus units, called neurons, that return an output which is the value of a function, called activation function, acting on the weighted sum of the inputs to the neuron. These units can be linked together in numerous fashion defining the architecture of the network. In our case, we use the Keras framework2 relying on Tensorflow3 as a backend to implement a fully-connected neural network with only one hidden layer of neurons, as shown in figure 5.3. Our hidden layer is composed of 80 neurons and our output layer of 3 neurons that each predicts the value of one of our three astrophysical parameters. With this simple architecture, a predicted parameter y pred can be explicitly written in term of the inputs to the network as :

y pred = f 2   80 i=1 W i f 1   N D j=1 w i,j x j + b i   + b   (5.12)
where N D is the dimension of the input, x j the value of the j-th component of the input, w i,j the weight given to x j by the i-th neuron of the first layer, b i the bias added by the i-th neuron, f 1 the activation function of the first layer, f 2 the activation function of the second layer, W i the weight given by the neuron of the second layer that predicts y pred to the result of the i-th neuron of the first layer and b the bias of the neuron of the second layer predicting the parameter y pred .

We chose a fairly simple architecture for our network as it is fully-connected and contains only one hidden layer. However it has been mathematically proven (Cybenko, 1989;Hornik et al., 1989) that neural networks with only one hidden layer can approximate with any accuracy any function if a sufficiently large number of neuron is used. The limit is rather in the size of the learning set describing the function to be interpolated. A network with many neurons but a small learning set will perform well on the learning set but weakly on the test set (overfitting phenomenon, or bias-variance trade-off). Evidences of the ability of this kind of network to handle the underlying denoising task have also been established [START_REF] Burger | 2012 IEEE Conference on Computer Vision and Pattern Recognition[END_REF].

Network characteristics

The learning process of a neural network is done by a gradient descent algorithm, referred as optimization algorithm. During the learning process, the optimization algorithm, using a prediction and its quality estimated by an error function, adjusts the weights given by each neurons to each inputs. The newly adjusted weights are then used to make another prediction during the next step and the weights are again re-adjusted to minimize the error function. This scheme is repeated over thousands of step, called epochs of learning, until the error function stop decreasing. For reference, we present all the characteristics of our chosen learning process in Table 5.1.

Characteristics Function

Cost Function :

Mean Squared Logarithmic Error C = 1 N TS N TS i=1 C i = 1 N TS N TS i=1 1 Np Np j=1 log 10
y pred i,j +1. y true i,j +1 2
Optimization Algorithm : RMSProp (Murugan & Durairaj, 2017)

w t+1 k,l = w t k,l - η E (∇ t w k,l C) 2 t +1×10 -7 ∇ t w k,l C with E (∇ t w k,l C) 2 t = 0.9E (∇ t-1 w k,l C) 2 t-1 + 0.1(∇ t w k,l C) 2
Activation function of the hidden layer : ReLU

f 1 (x) = x if x > 0 0 otherwise Activation function of the output layer : Linear f 2 (x) = x

Hyper-parameters

Learning rate η 5 × 10 -4 Batch size 128

Table 5.1: The detailed characteristics of our neural network. N T S is the number of data in the test set, N P the number of predicted parameters, here 3, y i,j is the j-th output parameter for the i-th data in the test set, t is the epoch of learning. w t k,l is the weight given by the l-th neuron of a given layer to its k-th input at the epoch of learning t

Results

Throughout this section, we will mainly quantify the quality of the predictions of our method through the root-mean square (rms) relative error, computed individually for each parameter and defined as :

χ y = 1 N T S N T S i=1 y pred i -y true i y true i 2 (5.13)
where N T S is the number of data in the test set and y i is the value of either ζ ion ,R mfp or T vir for the i-th case of this set. Tables and figures in this section are computed for the signals in our test set. We first compare our different supervised learning methods for the noise-free cosmological signals in the test set described in section 5.4.1, so we can compare our results to what has been done in Shimabukuro & Semelin (2017). We then compare our methods for predicting the astrophysical parameters from noised signals, affected by the perpendicularized noise described in section 5.4.3. When using these very specific realizations of the noise, a noised signal should correspond to the exact same (maximum-likelihood) model parameter values as the corresponding noise-free signal. Thus we are not exploring the intrinsic uncertainty on the predicted parameter values induced by thermal noise (i.e. the variance of the posterior distribution of the parameters values in the Bayesian approach), but rather the additional error introduced by the supervised learning algorithm in the parameter estimation. Our goal is to make sure that this additional error is small compared to the uncertainty due to thermal noise as estimated, for example, by Bayesian inference.

Noise-free cosmological signal

Uncovering an unintended degeneracy in the model

Testing our methods with a test set composed of numerous signals allows us to have the equivalent of multiple observations of the same phenomenon. This allowed us, during our investigation of the accuracy of the different methods, to "reverse-engineer" a feature of 21cmFAST that produces an unintended degeneracy in the R mfp parameter. This feature, compared to other approximations, has a small impact on the accuracy of the model. However, our goal is to prove that our method is able to invert the model accurately, whether the model itself is realistic or not. Absolute degeneracies make perfect inversion impossible. To push beyond the accuracy limit induced by the degeneracy, we had to correct for it.

Figure 5.4 depicts the predicted values of R mfp as a function of the real ones, the black line being a perfect prediction and the yellow dot the actual predictions of our local ridge kernel regression method. It clearly shows that the predictions take mostly discrete values and are not simply exhibiting random deviations around the value used to compute the signal. Looking into the 21cmFAST source code, it appears that our method has correctly reconstructed a feature of the code which we did not take into account when labeling signals. In this version of 21cmFAST, the radius of the regions to be tagged as ionized are investigated only in decrements of ×1.1. Consequently only a variation of R mfp by the same factor is guarantied to affect the results. More precisely, in our case, the influence of R mfp is the same for all R mfp in 4.8903 × 1.1 n , 4.8903 × 1.1 n+1 for n ∈ N, explaining the observed steps in figure 5.4.

For the rest of the study, we re-labelled our signals to correct for this degeneracy by setting the "true" value of any R mfp in a given interval to the geometrical mean value of the interval

R true mfp = 4.8903 × 1.1 n × 4.8903 × 1.1 n+1 = 4.8903 × 1.1 n+ 1 2 (5.14)
for n ∈ N. Not correcting for this feature may have affected previous works attempting to constrain model parameters using 21cmFAST. The resulting constraints may have been less tight than they could have been. Note that the latest version of 21cmFAST does not implement R mfp in the same way and may not be equally sensitive to this feature (although the discrete ×1.1 factor remains).

Without noise

χ ζ ion χ R mfp χ log(T vir ) Shimabukuro 2017
27.1 ×10 -2 22.8 ×10 -2 2.7 ×10 -2 Linear Regression 1.82 ×10 -2 8.00 ×10 -2 0.29 ×10 -2 Kernel Smoothing 1.19 ×10 -2 6.13 ×10 -2 0.28 ×10 -2 Neural Network 1.37 ×10 -2 2.53 ×10 -2 0.15 ×10 -2 Global Ridge Kernel Regression 0.68 ×10 -2 1.76 ×10 -2 0.09 ×10 -2 Local Ridge Kernel Regression 0.39 ×10 -2 0.19 ×10 -2 0.04 ×10 -2 Table 5.2: Rms relative error, χ, of the prediction for noise-free signals of the test set, computed for each parameter, for all the methods presented in section 5.4. The regressions have been optimized using the test set. Results from Shimabukuro & Semelin (2017) are shown for comparison.

Performance comparison

Table 5.2 displays the rms relative errors of the prediction, computed individually for each parameter, on noise-free signals of the test set, for all the methods presented in section 5.4 as well as results from Shimabukuro & Semelin (2017). We find that all methods of this work are better by one order of magnitude than the results presented in Shimabukuro & Semelin (2017). Beyond a more careful exploration and choice of the hyper-parameters of the learning process in the case of the neural network, the main reason for this improvement lies in the much larger learning set (justifying a network with more neurons) and the correction of the spurious degeneracy on R mfp . Our best supervised learning method for predicting a cosmological signal appears to be the local ridge kernel regression which improved Shimabukuro & Semelin (2017) results by at least a factor 50 for all three parameters, reaching a prediction rms relative error below 1%. However, remember that this result shows the theoretical maximum accuracy with perfectly optimized hyper-parameters.

Figure 5.5 shows the normalized distribution of the prediction relative error δ = y pred y true -1 as a function of the true parameter y true for our three astrophysical parameters ζ ion , R mfp and T vir for noise-free signals of the test set, as well as the uncertainty induced by the SKA thermal noise as evaluated using Bayesian inference by [START_REF] Greig | [END_REF] (green point) and the rms of relative error distribution computed at each bin (blue line). Note that each bin is defined for the value of one of the three parameters, the other two remaining unconstrained. Thus each bin holds a sample of typically > 25 cases. We present our three best supervised learning methods which are, from top to bottom : the neural network, the global ridge kernel regression and the local kernel ridge regression. For any of these methods, the rms relative errors of the predictions evaluated in different bins show only moderate fluctuations over the parameter range, and are inferior to a typical uncertainty induced by the SKA thermal noise. Moreover, for the local ridge kernel regression optimized on the test set the rms relative error of the prediction is only a few percent of the uncertainty induced by the SKA thermal noise. While this needs to be confirmed in the case of the noised signal, it opens the door to using supervised learning as a method to determined the maximum likelihood parameters associated with an observed signal.

Noised signal

When moving from a noise-free signal to a signal affected by noise, we are moving from a signal space that is effectively of dimension 3 (because we have 3 model parameters) to a signal space of dimension 120 (number of k bins times the number of redshifts). To improve the learning process, we need a finer sampling of the signal space. Thus, we generate 20 noised versions of the signal for each triplet of parameter values in the learning set. We do the same for the signals in the test set, generating 40 noised versions for each triplet of parameter values, to smooth our

With noise

χ ζ ion χ R mfp χ log(T vir ) Shimabukuro 2017
16.8 ×10 -2 17.2 ×10 -2 1.9 ×10 -2 Neural Network 3.70 ×10 -2 4.04 ×10 -2 0.41 ×10 -2 Global Ridge Kernel Regression 2.88 ×10 -2 2.84 ×10 -2 0.34 ×10 -2 Local Ridge Kernel Regression 1.10 ×10 -2 0.60 ×10 -2 0.16 ×10 -2 Table 5.3: Rms relative errors χ of the predictions for signals of the test set affected by a perpendicularized noise (see section 5.4.3), computed separately for each parameter. The rms values are given for all methods presented in section 5.4 where the regressions have been optimized using the test set. Results from Shimabukuro & Semelin (2017) are presented for comparison, but note that these did not use a perpendicular noise. estimation of the error distributions.

Perpendicularized learning set and maximum-likelihood prediction

As explained in section 5.4.2 using signals affected by a perpendicularized noise allows us to train the algorithm to predict the maximum-likelihood values of the parameters. Thus, using a learning set with signals affected by perpendicularized noise is the logical choice. Another possibility would be to trust the training of the learning methods to converge to an orthogonal projection of the signal space onto the model-manifold. This would likely happen in the case when the sampling of the signal space by the learning set is dense enough. We have no guaranty that this is the case in our situation. Thus we experimented with learning sets affected by noise either perpendicularized or not.

We found that using a learning set with a perpendicularized noise leads to a slight improvement of around 8% of the prediction accuracy for all methods. Even if the difference is not large, we decided to focus on a learning set with a perpendicularized noise, as it has a clearer theoretical interpretation. Let us now compare the performance of the different methods.

Performance comparison

Table 5.3 presents the prediction rms relative errors computed individually for each parameter on the signals of the test set affected by perpendicularized noise as described in section 5.4.3. We see that all our methods are reconstructing the astrophysical parameters with an accuracy of the same order of magnitude from a noised signal as from a noise-free signal, albeit worse by approximately a factor 3. As we used a perpendicularized noise, this factor of 3 is a measure of how much larger is the error introduced by the prediction algorithm in the noised case compared to the noise-free case. It is not a measure of the impact of thermal noise on the parameter uncertainty. We show in figure 5.6 the normalized distribution of the prediction relative error δ = y pred y true -1 as a function of the true parameter y true value along with the rms value of the distribution in each bin (blue line) for our three astrophysical parameters ζ ion , R mfp and T vir . We show the results for our best methods which are respectively from top to bottom : the neural network, the global ridge kernel regression and the local ridge kernel regression. We also show the 1σ uncertainty generated by SKA thermal noise as estimated with Bayesian inference [START_REF] Greig | [END_REF] (green bars) for comparison: we need our rms error to be, if possible, much smaller than this uncertainty. When considering the theoretical maximum accuracy of the local ridge kernel regression, which is when the hyper-parameters are optimized for the test set, we see that the prediction rms relative error is around 1%. Compared to SKA thermal noise, the reconstruction error is indeed smaller by one order of magnitude. Thus, for a perfect optimization of the hyperparameters, the local ridge kernel regression method enables a reconstruction of the highest likelihood astrophysical parameter with an error almost negligible compared to the size of the 1-sigma contour from Bayesian inference.

Hyper-parameters optimization strategy

Previous results for the local regression method should be considered as the theoretical maximum accuracy. As explained in section 5.4.4, we have optimized our hyper-parameters by using prior knowledge of the true value of the parameters, which is obviously not possible for an observed signal. Thus, we will now consider the case when the hyper-parameters have been optimized using only the information from the learning set, a method that can directly be applied to an observed signal. In this case, the hyper-parameters values assigned to a signal of the test set are the optimized values determined for the closest signal in the learning set. Let us note that the gap between the theoretical maximum accuracy and the accuracy of this optimization on the learning set may be narrowed in the future with better hyper-parameter optimization techniques.

Noise-free cosmological signal

The top part of table 5.4 displays the rms relative errors, computed for each parameter, for the noise-free signals from the test set described in section 5.4.1 using neural network and global and local ridge kernel regression optimized on the learning set (LS) or the test set (TS). For now, if we optimize the hyper-parameters with only the information from the learning set, the best method is the neural network. The local ridge kernel regression accuracy worsen by a factor ∼ 10, implying that it is sensitive to the value of the hyper-parameters, and that the optimal values are changing fast when moving away from the grid-point used in the regression.

Noised signal

We also present in the bottom table of Table 5.4 the rms relative errors χ, computed for each parameter, for a noised signal using neural network and global and local ridge kernel regression optimized on either the learning set (LS) or the test set (TS). The results are shown for signals of the test set noised with a perpendicularized noise. When not optimizing on the test set, the global ridge kernel regression is our most accurate way to reconstruct the astrophysical parameters with a prediction accuracy of a few percent or roughly half of SKA thermal noise. This method is barely affected by not using information from the test set, which is understandable since the optimization of the hyper-parameter is global. This simply states that the test set and learning set have comparable properties in this respect. We observe a decrease of the performance of the local ridge kernel regression when optimizing on the learning set similar to that in the case of noise-free signals.

To summarize, the error on the prediction of the parameters caused by the supervised learning methods, although much improved, is not yet quite negligible compared to the SKA uncertainty when considering the method that could be directly applied to a real signal. Algorithms to derive the optimal hyper-parameters to use on a real signal will have to be further improved.

Conclusion

In this work, we explored new supervised learning methods to constrain the underlying astrophysical parameters of the EoR. For this, we chose to base our reconstruction of the parameters on the power spectrum of the intergalactic 21-cm signal, measured at 12 wavenumbers and each integer redshift from z = 5 to z = 15. We used 21cmFAST to compute the power spectra, varying three different parameters. We chose to vary ζ ion which accounts for the ionizing efficiency of high-z galaxies, R mfp which is the mean free path of ionizing photons within the ionized regions and T vir which expressed the minimum virial temperature for halos to be allowed to form stars.

We used a learning set of 2400 signals produced by Eames et al. (2019). They are generated on a 20×6×20 grid in the parameter space (ζ ion ; R mfp ; T vir ). A test set of 512 signals whose parameters are randomly picked within the bounds of the former set was also generated. To be more realistic we also analysed the case where a SKA-type thermal noise is added to the signals. It leads us to our first main result :

• The signal in the test set which is used to evaluate the prediction accuracy cannot be modified with a generic noise. If this is done, the most likely parameters values associated with the noised signals are unknown: they are not, in the general case, those that were used to produce the noise-free signal. Thus the accuracy of the prediction (i.e. the systematic introduced by our inference method) cannot be computed. To circumvent this issue, we have to perpendicularize the noise in the signal space relatively to the model-manifold.

We mainly implemented two supervised learning methods for our comparison. We first improved the neural network method by using a better optimization of the learning algorithm and hyper-parameters, and by using a larger learning sample, but we kept the architecture from Shimabukuro & Semelin (2017) which is a fully-connected network with one hidden layer. Secondly, we studied another class of supervised learning methods which are different kinds of linear regressions and whose most advanced version is a ridge kernel regression with hyperparameters optimized locally in the signal space. Comparing the prediction accuracy of those methods, we get the following results:

• For a 21-cm signal with no added noise, considering only methods which do not use information on the true value of the parameters to be predicted to optimize their learning process, the best method is the neural network. We predict the parameters with an error of a few percent, which is an order of magnitude better than in Shimabukuro & Semelin (2017). On the other hand, if we focus on the theoretical maximum accuracy, the best method is the local ridge kernel regression whose hyper-parameters are optimized directly using information from the test set. This information would of course not be available in the case of an observed signal. We find that, when the hyper-parameters are perfectly optimized, this method leads to a prediction rms relative error below 1%, for a result 50 times better than in Shimabukuro & Semelin (2017).

• When considering 21-cm signal with an added SKA thermal noise, the most accurate operational method is the ridge kernel regression globally optimized on the learning set with a prediction rms relative error of a few percent which is approximately half the amplitude of SKA thermal noise such as predicted in [START_REF] Greig | [END_REF]. Again, from all methods the one with the theoretical maximum accuracy is the locally-optimized ridge kernel regression which reconstructs the astrophysical parameters with an accuracy of the order of 1% which is 10 times lower than the predicted SKA noise amplitude, meaning that, once optimized to its maximum, this methods will recover the maximum likelihood astrophysical parameters with an almost negligible error due to the supervised learning method.

As explained in section 5.4.1, our results are mitigated by the quality of our optimization of the hyper-parameters which we cannot prove to be a global optimization. Also, we have optimized the performance of a neural network with only one hidden layers and do not explore the wide possibility of deep learning architecture with several hidden layer, which can likely further improve the accuracy of the predictions.

Appendix : Perpendicularized noise generation at generic parameter space location

To generate a noise perpendicular to the model-manifold at a location that is not on our initial sampling grid, we will still use the set defined on the grid as a way to obtain the local basis generating the hyper-plane tangent to the model-manifold. For a signal P ζ x ion ,R y mfp ,T z vir , our algorithm is:

1. Identify the indexes i, j and k in the grid-generated set such that

ζ i ion ≤ ζ x ion ≤ ζ i+1 ion , R j mfp ≤ R y mfp ≤ R j+1 mfp and T k vir ≤ T z vir ≤ T k+1
vir which determine the eight signals from the grid-generated set that form the corners of the cell containing the considered signal

P ζ x ion ,R y mfp ,T z vir .
2. Using the algorithm for a grid-based set, compute the eight basis (e 1,α,β,γ ; e 2,α,β,γ ; e 3,α,β,γ ) with α = i or i + 1, β = j or j + 1 and γ = k or k + 1, corresponding to these corner points.

3. Compute the distances D x,y,z (i, j, k) between the considered signal and each of the eight previous points, based on the same definition of the scalar product in signal space. where

W (α, β, γ) = [D x,y,z (α, β, γ)] -1 i+1 α =i j+1 β =j k+1 γ =k [D x,y,z (α , β , γ )] -1
5. Orthonormalize the previous basis to obtain an orthonormalized basis whose elements will be referred as e 1,x,y,z , e 2,x,y,z and e 3,x,y,z 6. Generate a Noise N and compute N ⊥ using Equation 5.7.

Ionized bubble size: environmental variability

Environmental variability with LICORICE

When using a numerical code that solves simultaneously gravity, hydrodynamics and radiative transfer, two halos with the same dark matter mass can have different star masses (Rosdahl et al., 2018;Trebitsch et al., 2020). Accordingly, the luminosities of the two halos will be different too and there will be a dispersion in the relation that links the halo luminosity to the halo mass. This dispersion likely exists in our Universe too. It would be convenient to find a robust method to add this dispersion in semi-numerical codes as attempted by previous studies [START_REF] Sullivan | [END_REF]. In this chapter, we based our analysis on the results of the HIRRAH-21 simulation, detailed in section 8.3.1, that follows the evolution of 2048 3 particles, half of them baryons and the other half dark matter, in a 200 h -1 cMpc box. Figure 6.1 presents the halos luminosity as a function of their dark matter mass, where each point represents a halo of the HIRRAH-21 simulation and is colored according to its star mass content. This dispersion behavior is identical no matter the chosen global ionization fraction x H II . As expected, we see that the dispersion in this relation is mainly due to the number of star in each halo. To account for this effect, models that directly populate dark matter halos with stars could, for example, populate them following a distribution rather than a mapping. There are relations for which the dispersion is however harder to understand. Figure 6.2 shows ionized bubbles volume as a function of their inner halos luminosity for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. It is worth noting that we will focus on a low global ionization fraction of x H II ∼ 0.01 for the following study. This is because at this stage of reionization, bubbles can be reasonably thought as independent and isolated. We can therefore better interpret their properties than with a strong percolation. As explained in section 1.2 the ionization process mainly depends on the photo-ionization and recombination effects. The luminosity of the halos being the source of ionization, we would expect a direct (or at least less dispersed) relation between ionized bubble1 volume and halo luminosity but this is not reflected in figure 6.2. There are two possible explanations for this result : either the halo luminosity is not the most suited quantity to account for the photo-ionization effect or the environmental variability of recombination is the main cause of the dispersion of the bubble sizes and impacts in a non-negligible way the whole photo-ionization process.

Influence of the escape fraction

Escape fraction computation algorithm

The intrinsic luminosity of a halo does not directly lead the photo-ionization process. Only a fraction of the produced photons are not absorbed inside the halo and effectively escape into the intergalactic medium. It is therefore this escaped luminosity that has to be considered when considering the ionization of the IGM (as previously shown in equations 1.9 and 3.7). The HIRRAH-21 simulation has been computed using 4096 MPI domains. Based on the particles and halos catalog we thus use the following algorithm to compute the escape fractions of halos MPI-domain-wise:

1. The density ρ and ionization x H II fields are interpolated, following a SPH approach (see section 3.2.2), on a 10243 grid for a single domain. This means an effective 16384 3 grid, and a 12 ckpc resolution (compared to the 2 ckpc gravity smoothing of the simulation).

Except for this algorithm, all studies of the next chapters are based on fields computed on a 1024 3 grid for the whole box. However, here we need a far greater resolution to compute the escape fraction2 .

2. For each halo in the domain, we compute a radius following

R halo = 3M halo 4π (ρ halo ) 1/3
where M halo is the halo mass (i.e. the sum of the mass of its composing particles), ρ halo = 200ρ is the halo density with ρ the average density in the box. During this step, we also ignore halos that, based on their radius and center of mass, are not totally included in the domain because we do not have the information about the density and ionization field outside the current domain.

3. For a given halo, we propagate a given number of photon packets N phot (usually 10 6 ). We assign to each particle in the halo N γ,halo photon packets following

N γ,part = N γ,halo Lpart L halo
where L part is the luminosity of the particle and L halo the total luminosity of the halo. A particle can then be in two situations :

• If the particle is outside the halo radius 3 (i.e. |x part -x halo | > R halo where x halo is the location of the halo center of mass and x part the position of the particle), we consider that all the content of its photon packets escaped and add N γ,part to N γ,esc the total number of escaped photons.

• If the particle is inside the halo radius, we propagate each of its N γ,part photon packets following :

(a) The photon packet energy E γ is drawn from a Planck distribution (between E=13.7 and 100eV with a temperature of 5 × 104 K) and its direction is drawn with respect to spherical isotropy. (b) The photon packet propagates until it reaches the edge of the halos (i.e. |x γ -

x halo | > R halo where x γ is the position of the photon) and the optical depth τ is computed along the path following

τ = i σ(E γ )ρ i (1 -x H II ,i )dl i
where the summation is done over all cells crossed during the propagation, σ is the ionization cross-section and dl i the distance traveled in cell i. (c) We add the remaining content of the photon packet e -τ to N γ,esc . In Figure 6.3 we represent two slices of the ionization field of the HIRRAH-21 simulation centered around two halos along with R halo (black circle), x halo (white dot) and the position of the star particles (green dots). It illustrates the large variety of spatial environments that exist for two halos of approximately the same size and number of particles. The right panel also illustrates the high recombination that can exist around halos inside ionized regions and has motivated our peculiar construction of the ionized bubbles-halos matching algorithm detailed in section 7.2.2.

Escape fraction evolution

For the HIRRAH-21 simulation we observe that the halo escape fraction decreases with increasing halo mass. This behavior is in agreement with results from other numerical simulations (Wise et al., 2014;Kimm & Cen, 2014;Paardekooper et al., 2015;Kimm et al., 2017). At a given mass, the average escape fraction is higher in HIRRAH-21 than in other works (Paardekooper et al., 2015;Kimm et al., 2017). It is however difficult to directly compare these results as both the resolution and the numerical methods used are different. The relation between the escape fraction and the halo mass also presents an important dispersion that has already been observed in Kimm & Cen (2014). It likely comes from the fact that, for a given halo, there is a temporal evolution of the escape fraction that can vary by orders of magnitude (Kimm & Cen, 2014;Trebitsch et al., 2017). Supernova feedback seems to be the leading mechanism behind this temporal variation. The supernova removes gas from the clouds and increases the escape fraction for a time before the accretion of the surrounding gas decreases its value again. At a given time, various halos of a given mass can therefore be at various stage of this supernova feedback cycle and therefore have different escape fractions.

In Figure 6.4 we show the escape fraction as a function of the halo luminosity for halos of same dark matter mass in the HIRRAH-21 simulation. We see an anti-correlation between the escape fraction and the luminosity of a halo. Therefore, at a given dark matter mass, the more luminous the halo is, the lower its escape fraction is. An explanation of this effect might be that, as shown by the colorbar, the more luminous halos also contain more gas. Recombination is therefore higher and consequently the escaped fraction lower. In any case, the escaped luminosity is theoretically a quantity more directly linked to the ionized bubble volume. Depicting the ionized volume as a function of the escaped luminosity rather than the intrinsic luminosity might thus lead to less dispersion in the relation. relation is highly similar to the bubble volume as a function of the intrinsic luminosity of figure 6.2. Moreover, we see in the bottom panel of the two figures that the dispersion is also the same. The escaped fraction does not seem to be the source of dispersion when linking bubble volume and halo luminosity. This result is mitigated by figure 6.6 that shows the ionized bubbles volume as a function of their inner halos luminosity given a range of escape fraction for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. For halo luminosities greater than 10 43 erg.s -1 , we see that the mean ionized volume increases with increasing escape fraction. In this luminosity range, the difference of escape fraction thus contribute to the dispersion. However for the smallest bubbles (associated to halos with luminosity L 10 43 erg.s -1 ), where the dispersion is especially important, the escape fraction value does not seems to affect the mean ionized volume. Moreover, we see that splitting the data set by range of escape fraction does not lead to disjoint relations as one would expect if the escape fraction was the underlying parameter that solely causes the dispersion. The dispersion for each escape fraction range is also still important.

Impact on dispersion for ionized bubble volume

If splitting the sample using the escaped fraction does not totally alleviate the dispersion in the relation describing the bubble volume as a function of the halo luminosity, it is likely that the photo-ionization process is insufficient to explain the dispersion by itself.

Ionized bubbles properties

Overdensity evolution

To accurately account for the effect of the density field configuration around the sources on recombination, we use the clumping factor. Multiple clumping factors can be defined depending on the purpose (see Chen et al. (2020)) but the mass-weighted clumping factor for hydrogen (no matter its ionized state) can be defined as : Halo Luminosity (in erg.s -1 ) where n H is the number density of hydrogen, and M is the mass average. Computationally, as m cell = n cell V cell and all cells of the grid have the same volume V cell , we can rewrite C H for a given ionized bubble as :

C H = n 2 H M n H 2 M (6.1)
10 -1
C H = i n H,i i n 3 H,i i n 2 H,i 2 (6.2)
where the summation is done for every cell composing the ionized bubble. We see from the previous equation that only the density field is required for the numerical computation. When computed inside ionized bubbles, the volume-weighted clumping factor for the HIRRAH-21 simulation at x H II ∼ 0.01 is ∼ 2.5. It is in agreement with values from other numerical simulations (Shull et al., 2012;Finlator et al., 2012;Jeeson-Daniel et al., 2014;Kaurov & Gnedin, 2015;Chen et al., 2020) and analytical model (Kaurov & Gnedin, 2014;Madau, 2017).

An increase of dispersion linked to the clumping factor

In figure 6.7 we show the ionized bubbles volume as a function of their inner halos luminosity given a range of clumping factors for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. We see that splitting the data set by interval of clumping factor does lead to disjoint and less dispersed relations. One of the main causes of the dispersion in the relation between the bubble volume and their inner halos luminosity is therefore the configuration of the density field that alters the recombination amplitude. This conclusion is consistent with other studies like Mao et al. (2020) and Bianco et al. (2021a) that show the strong influence of the small-scale density fluctuations on the overall reionization process.

Interestingly, we also see in figure 6.7 that for a given halo luminosity, the ionized bubble volume increases with an increasing clumping factor. It might come from the fact that a higher clumping factor means a more heterogeneous environment. Therefore, there are very dense regions that recombine a lot but also very large regions that are almost empty, allowing the Ionized bubble volume (in cMpc 3 ) 1.00E+00≤CH≤1.33E+00 1.33E+00≤CH≤1.77E+00 1.77E+00≤CH≤2.35E+00 2.35E+00≤CH≤3.12E+00 3.12E+00≤CH≤4.14E+00 4.14E+00≤CH≤5.51E+00 5.51E+00≤CH≤7.32E+00 7.32E+00≤CH≤9.73E+00 Figure 6.8: Illustration of the increase of bubble volume with increasing clumping factor at constant halo luminosity. In the left panel, the clumping factor is low and the halo vicinity if homogeneous. In every direction, photons undergo a recombination and only one third of them get through. In the right panel, the clumping factor is high and the halo vicinity heterogeneous. In some direction, the gas clumps absorb all photons. However, in other directions the low gas density leads to almost no recombination. In total, half of the photons get through the halo direct vicinity and can thus ionize a larger volume.

photons to propagate for a long time before being absorbed. In total, the ionized volume might become significantly larger than in an homogeneous medium where photons undergo less intense recombination but in every direction. In figure 6.8 we illustrate our hypothesis.

Haloes and Ionized bubbles distribution : formalisms and computation

In this chapter, the basics of ionized bubbles identification is presented as well as the theoretical framework for the computation of their distributions and the reference theoretical model. It is a review of this field, the new materials developed during our study are presented in the next chapter.

Reference methods of ionized bubbles identification

Spherical-average (SPA)

To construct a bubble size distribution from numerical results, one can use the SPA method based on the excursion set theory (Zahn et al., 2007;[START_REF] Friedrich | [END_REF]Lin et al., 2016;Giri et al., 2018). At each location/cell, the ionization is averaged within a sphere of a given radius R. This smoothing radius is first of the order of the simulation box size. If the average ionization fraction inside the region is greater than a given threshold x th , the point is marked as ionized inside a bubble of radius R and another point is tested. Else, the average ionization fraction is computed within a sphere of a smaller radius. The process repeats iteratively until the minimum smoothing radius R (usually the size of a cell of the ionization field) at which point if the condition x R,H II ≥ x th is not respected, the point is marked as neutral.

The SPA method is diffusive and biased as outlined by [START_REF] Friedrich | [END_REF] and Lin et al. (2016). For a toy model of multiple non-overlapping bubbles of the same size R 0 , the produced distribution peaks close to R 0 3 and ranges from 0 to R 0 (Giri et al., 2018).

Mean-free-path (MFP)

Another category of processes to construct bubble size distribution from numerical ionization fields is the mean-free-path (MFP) (Mesinger & Furlanetto, 2007;[START_REF] Friedrich | [END_REF]Lin et al., 2016;Giri et al., 2019). During a step of this process, a random ionized location is selected then a photon is cast along a random direction. This photon is followed until a stopping criteria, which is usually when it encounters a cell with an ionized fraction below a given threshold x th . As the stopping criteria is, in essence, when the photon is absorbed, the distance that the photon travels is strongly links to its mean free path, hence this method name. This step is repeated multiple times (usually more than 10 6 times) leading to a histogram of photons travel distance that corresponds to the bubble size distribution.

MFP methods lead to different results depending on the selected threshold value [START_REF] Friedrich | [END_REF]. When using them on a toy model of multiple non-overlapping bubbles of the same size R 0 , they peak at R 0 but range from 0 (when the photon starts at the edge of a bubble go toward the neutral region) to 2R 0 (when the photon start at the edge of a bubble but travel toward the center of the ionized region and reach the other edge before being absorbed). They are therefore unbiased (Lin et al., 2016) but diffusive (Giri et al., 2018).

Ionized bubbles in this work : identification and properties

Refined friends-of-friends (FOF) algorithm

As detailed in section 3.2, LICORICE is a code based on particles that solves simultaneously gravity, hydrodynamics and radiative transfer. For predefined redshifts, it outputs a catalog listing all particles and their characteristics. Based on this catalog, one can therefore compute the value of the physical quantities on a grid following a SPH process detailed in section 3.2.2. To identify the ionized bubbles, we use a friends-of-friends method (FOF) where the algorithm (close from Hoshen & Kopelman (1976) algorithm) is based on the ionization field and proceeds as follows :

• For each cell of the grid, if the ionization fraction is greater than a given threshold x lim H II , we label the cell as being part of a new bubble. At this step, each bubble only consists of one cell.

• If two bubbles contain neighboring cells, they are merged into one bubble. This process continues until all remaining bubbles are disjoint. The threshold x lim H II is a tunable parameter of the algorithm. In figure 7.1 we show the bubble size distribution for three values of x lim H II (0.1, 0.5 and 0.7) at three different global ionization fractions (x H II = 0.01, 0.03 and 0.1). As expected, we see that a lower threshold leads to consider more cells as part of ionization bubbles and so as ionized. The total effective ionization fraction computed from the bubble catalog thus increases with a decreasing threshold. The evolution of the bubble size distribution as a function of the effective ionization fraction is the same for every threshold x lim H II . For the following study of chapter 8 we therefore set x lim H II to 0.5 to better match the simulation global ionization fraction and the effective ionization fraction computed from the bubble size distribution.

Halo-bubble matching

To better understand the astrophysical processes that rule the ionized bubbles evolution, we need to relate them to the characteristics of their source halos. To detect all possible halos contained in a bubble, we use the ionization field and bubble catalog detailed in the previous section and apply an algorithm composed of two independent process. 1. For each halo, if the center of mass of the halo is inside a cell that is part of an ionized bubble, the halo is included in this bubble.

2. Using the algorithm detailed in the previous section, we construct the independent "neutral regions" of the ionization field. If a neutral region has only one ionized bubble as a neighbor (i.e. this neutral region is inside the ionized bubble), we merge the neutral region with the ionized bubble and include all halos inside the neutral region in this ionized bubble. This process aims at detecting massive halos that belong to the ionized bubble but are effectively in neutral cells due to recombination. Such cases are not purely theoretical and exist in practice as illustrated by figure 6.3.

It is worth noting that, of all the halos that are tagged as inside an ionized bubble at the end of the process, more than 99% of halos have been included during the first step. However, as the other step targets halos that have a sufficiently large neutral vicinity so that their containing cell is almost neutral, these halos are particularly massive. Therefore, even if they are rare, they can non-negligibly affect the total content of their containing ionized bubbles.

Halo Mass function theory

Matter overdensity and variance

Following the simplest inflation models and assuming that there are no primordial non-gaussianities, the primordial density fluctuations can be approximated by a homogeneous Gaussian random field. There is however a correlation between the values of the overdensity δ at two different locations. Fortunately, the Fourier transform of a Gaussian random field is still a Gaussian random field. Furthermore, the cosmological principle of homogeneity at large scales implies that the Fourier coefficients of this field are independent realizations of the probability function 1 . This Gaussian random field can be characterized by the overdensity power spectrum. However, we can alternatively characterize it by the variance σ R of the Gaussian probability function at various scales R. This variance can be computed from the observed overdensity power spectrum defined by the relation δ(k)δ(k ) = P (k)(2π) 3 δ D (k + k ) with δ D the Dirac distribution and k is the Fourier conjugate of the location x. Using this overdensity power spectrum, we compute the variance at scale R through :

σ R = 1 (2π) 3 P (k)W 2 R (k)d 3 k (7.1)
where W R (k) is a smoothing kernel. The previous equation gives the variance corresponding to the actual density field at z = 0. It is often expressed as an astrophysical parameter : σ 8 which is the value of σ R for R = 8 h -1 Mpc at z = 0. This variance can also be computed for other redshifts starting from the observed overdensity power spectrum 2 and evolves as ∝ (1 + z) 2 .

1 In numerical simulation, it is thus easier to build such fields in the Fourier space 2 The exact expression of the variance is given in (Weinberg, 2008, chap 8) as

σ 2 R (z) = 1 2π 2 (1+z) 2 C((1+z) 3 Ω Λ /Ω M ) C(Ω Λ /Ω M ) 2 ∞ 0 P (k)|f (kR)| 2 k 2 dk where f is the top hat distribution function and C(x) ≡ 5 6 x -5/6 √ 1 + x x 0 du u 1/6 (1+u) 3/2 is a correction function normalized so that C(0) = 1.

Linear and non-linear overdensity

During the matter dominated era, on scales smaller than the Hubble length (i.e. the horizon), the matter density fluctuations evolves linearly with the expansion factor allowing us to define a linear overdensity. However, at small scales, we have to consider non-linearities through spherical collapse models and thus define a non-linear overdensity. This difference between linear and non-linear overdensities is especially relevant when comparing theoretical models to numerical one. Indeed, theoretical model often uses the linear overdensity whereas fully hydrodynamical simulations directly computes an overdensity that corresponds to the non-linear one. We can link linear and non-linear overdensities through (Sheth & Tormen, 2002) :

δ l (δ nl ) = δ c 1.68647 1.68647 - 1.35 (1 + δ nl ) 2/3 - 1.12431 (1 + δ nl ) 1/2 + 0.78785 (1 + δ nl ) 0.58661 (7.2)
where δ l denotes the linear overdensity, δ nl the non-linear overdensity and δ c = 1.68 the critical density for spherical collapse. The critical density δ c defines the threshold above which matter collapses into structures like halos.

Halo Mass function formalisms

Standard Press-Schechter formalism

The halo mass function (HMF) represents the number of collapsed structures per unit volume in a given mass bin. It has first been analytically computed in Press & Schechter (1974) whose logic we follow in this section. From linear theory, we can define a collapsed structure as a structure whose average linear overdensity on scale L is larger than the critical density δ c . To identify such structures, we have to compute the smoothed density field on a scale L following :

δ(L, x) = δ(x )W L (x -x )d 3 x (7.3)
where W L (xx ) is a smoothing kernel 3 . As said in section 7.3.1 the Fourier transform of the overdensity is a Gaussian random field and the Fourier coefficients δ(k) are independent realisations of gaussian probability functions. The Fourier transform of the smoothed overdensity field is δ(L, k) = δ(k)W L (k). Its coefficients are therefore independent realisations of gaussian probability functions too. The smoothed overdensity field δ(L, x) is also a Gaussian random field with a variance defined in equation 7.1. The smoothed density field on scale L being a Gaussian random field, its probability to be greater than δ c at a location x is :

P >δc (L) = ∞ δc 1 √ 2πσ 2 (L) exp - δ 2σ 2 (L) dδ (7.4)
For a top hat spherically symmetric smooth filter, the collapse mass of a collapse structure is accordingly larger than M (L) = ρ 0 (1 + δ c ) 4π 3 L 3 with ρ 0 the average density and we can make a change of variable L ↔ M in P >δc . For a cosmological volume, we have ρ 0 V M possible collapsed objects of mass larger than M. The number of collapsed object of mass greater than M in such volume will be N (> M ) = ρ 0 V M P >δc (M ). The number density of collapsed structure per unit of mass bin (i.e. the HMF) can thus be expressed :

n(M ) = - ρ 0 M dP >δc dM = - dσ dM ρ 0 M δ c √ 2πσ 2 (M ) exp - δ c 2σ 2 (M ) (7.5)

Extended Press-Schechter formalism

The Press-Schechter formalism can be extended using a random walk approach (Bond et al., 1991;Lacey & Cole, 1993). We recall that the overdensity field δ(t) grows linearly with the expansion factor up to the point where non-linearity becomes important. The overdense region then stops its expansion and collapses when δ(t) > δ c . The basis of the extended Press-Schechter (hereafter EPS) formalism is to alternatively consider δ at a given time and use a time-dependent density threshold δ c (t) that decreases with time. Therefore at a given time t all regions with δ > δ c (t) have collapsed. The time-dependent critical density δ c (t) is defined as (Lacey & Cole, 1993) :

δ c (t) = (12π) 2/3 10 t 0 t 2/3 (7.6)
where t 0 is the present day time. This approach is consistent with the previous one as δ c (t 0 ) = 1.686. We can compute the smooth density field at a given scale around a given location x using equation 7.3. It is also worth noting that any reasonable density power spectrum functional form implies that σ(L) → 0 and δ(L, x) → 0 when L → ∞. Any given particle at time t therefore corresponds to one of the two following cases :

• The smoothed density field around this particle is always lower than δ c (t) for any smoothing scale, in which case the particle does not belong to a collapsed structure.

• There is a scale L where δ(L, x) ≥ δ c (t), in which case the particle belongs to the halos formed by the collapse of the largest scale L max respecting the previous condition (i.e.

δ(L max , x) = δ c (t)).
Like in previous section, the smoothing scale L can also be expressed in term of its mass M . To find the halos containing a given point we can thus consider the smooth density field δ as a function of the smoothing mass M and find the largest M for which δ(M, x) = δ c (t). This evolution of δ at point x can be viewed as a trajectory in the space of the overdensity evolution.

The mass variance on mass scale M can be expressed as σ 2 (M ) = |δ(M, x)| 2 and decreases with increasing M . Mathematically, we can rather express the smooth density field δ evolution in terms of σ 2 (M ) rather than M . This change of variable leads to an interesting deduction.

As detailed in section 7.3.1, the density field is initially a gaussian random field that can be characterized by its variance and whose Fourier conjugate coefficients δ k are independent realizations of the probability function. At a given point x to go from δ(M ) to δ(M -dM ), the contribution of modes between the corresponding k and k -dk (Fourier conjugates of L(M ) and L(M ) -dL) in Fourier spaces will be drawn from a probability function. It will thus be equivalent to a step in a random walk of trajectory δ (Bond et al., 1991). Using σ as a variable instead of M or k ensures that every step of the random walk will have the same size. One can already guess that by counting the number of trajectories that effectively lead to a collapse in an interval of mass [M -dM, M ], it is possible to deduce a probability of collapse (i.e. forming halos) at such mass, mass variance and corresponding overdensity.

Using a top hat function as a smoothing kernel, the trajectories are strictly Brownian random walks and their evolution can thus be governed by a diffusion equation. For this diffusion process, the condition that collapsed structure will be formed for the largest M where δ(M, x) = δ c (t) is equivalent to an absorbing barrier δ c (t). The probability f (σ 2 , δ c (t))d(σ 2 ) that a trajectory will cross the barrier δ c (t) in the interval [σ 2 , σ 2 + d(σ 2 )[ (i.e. that a region of mass between M and M + dM corresponding to the σ 2 (M ) range will collapse into a halo) can be derived from the solution to the diffusion equation as (Lacey & Cole, 1993) :

f (σ 2 , δ c (t))d(σ 2 ) = δ c (t) √ 2πσ 3 exp - δ c (t) 2σ 2 d(σ 2 ) (7.7) σ δ Barrier δ c (t)
All trajectories Trajectories leading to collapse Finally the comoving number density of haloes of mass M at time t is (Lacey & Cole, 1993) :

n EPS (M, t) = 2 π ρ 0 M 2 δ c (t) σ(M ) dlnσ dlnM exp - δ 2 c (t) 2σ 2 (M ) (7.8)
It is worth noting that the EPS formalism for HMF differs from the Press-Schechter formalism only by a factor 2.

Sheth-Mo-Tormen formalism

The previous EPS formalism was based on a spherical collapse model. Its also possible to extend the description to build a halo mass function based on the gravitational collapse of homogeneous ellipsoids. Using the same random walk process as EPS, Sheth et al. (2001) (hereafter ST) shows that an ellipsoidal collapse induces a modification of the absorbing barrier (previously δ c (t)) in the diffusion equation. This new barrier has been empirically approximated by :

δ ec (σ, z) = √ aδ c (z)   1 + β aδ c (z) 2 σ 2 -α   (7.9)
where a = 0.707, β = 0.485 and α = 0.615. Critically, the barrier is not constant anymore and evolves with the variance σ. The halo mass function based on the ellipsoidal collapse is accordingly approximated by (Sheth et al., 2001) :

n ST (M, z) = A 2a π 1 + σ 2 aδ 2 c (z) p ρ 0 M δ c (z) σ 2 (M ) dσ dM exp - aδ 2 c (z) 2σ 2 (M ) (7.10)
where p = 0.3 and A ≈ 0.322. Despite being an empirical approximation, the ST formalism usually produces halo mass functions closer to numerical results than EPS that tends to overestimate the HMF (Tinker et al., 2010;Murray et al., 2013).

Conditional Mass function formalisms

Conditional mass functions (hereafter CMF) are essentially extension of the HMF formalism when starting in already overdense regions (Lacey & Cole, 1994;Sheth & Tormen, 2002;Rubiño-Martín et al., 2008). It thus expresses the comoving number of density of haloes of a given mass at a given redshift inside an overdense region of a given overdensity δ 0 and of size R 0 thus of variance σ 0 ≡ σ(R 0 ).

In the EPS formalism this overdensity condition of the medium can easily be considered. With the random walk approach, detailed in section 7.3.3, this condition only means that the trajectories start from the point (δ 0 , σ 0 ) instead of the origin. Mathematically, the CMF thus has the same expression than the HMF but with the change of variables δ → δ c -δ 0 and σ 2 → σ 2 -σ 2 0 . Within the EPS formalism, the CMF is therefore (Lacey & Cole, 1994) :

n c,EPS (M, z) = 2 π ρ 0 M σ(δ c -δ 0 ) (σ 2 -σ 2 0 ) 3/2 dσ dM exp - (δ c -δ 0 ) 2 2(σ 2 -σ 2 0 ) 2 (7.11)
Within the ST (Sheth & Tormen, 2002;Rubiño-Martín et al., 2008) formalism, the barrier depends on the variance σ as expressed in equation 7.9. A change of variable will therefore also affect the barrier and, as for ST HMF, the CMF must be approximated numerically. The CMF formalism for ellipsoidal collapse is thus given by (Sheth & Tormen, 2002;Rubiño-Martín et al., 2008) :

n c,ST (M, z) = 2 π ρ m dσ dm σ(m) |T (σ, σ 0 )| (σ(m) -σ 0 ) 3 2 exp - (δ ec (σ, z) -δ 0 ) 2 2(σ(m) -σ 0 ) 2 (7.12)
where

T (σ, σ 0 ) = 5 n=0 (σ 2 0 -σ(m) 2 ) n n! ∂ n (δ ec (σ, z) -δ 0 ) ∂(σ 2 ) n
with δ ec (σ, z) the barrier derived in 7.9.

Reference model of Bubble Size Distribution

Theoretical description

The reference model for computing the bubble size distribution (hereafter BSD) has been developed in Furlanetto et al. (2004). It relies on the assumption that each ionized bubble is associated to a single halo and that :

m ion = ζ ion m gal (7.13)
where m ion is the mass inside the ionized region, ζ ion is ionization efficiency and m gal the mass of the collapsed object. It is worth noting that this assumption strongly constrains the whole model which will not be usable for other, more complex relations, between the ionization source and its ionized region. From the previous equation, we can deduce that an isolated region of mass m reg is ionized if the collapsed fraction f coll ≡ m gal mreg is greater than ζ -1 ion . Using the EPS formalism for CMF of section 7.3.4, we can compute the collapsed fraction for this region of mean linear density δ reg and variance σ reg following :

f coll = 1 ρ 0 mreg m min n c,EPS mdm = σreg σ min 2 π σ(δ c -δ reg ) (σ 2 -σ 2 reg ) 3/2 exp - (δ c -δ reg ) 2 2(σ 2 -σ 2 reg ) 2 dσ = 2 √ π ∞ δc-δreg √ 2(σ 2 min -σ 2 reg ) exp -t 2 dt = erfc   δ c -δ reg 2(σ 2 min -σ 2 reg )   (7.14)
where m min is the minimum mass of an ionizing source and σ min = σ(m min ). We thus can deduce that a region can self ionized if :

f coll ≥ ζ -1 ion ⇔ δ reg ≥ δ lim ≡ δ c (z) - √ 2[σ 2 min -σ 2 reg ] 0.5 erf -1 1 -ζ -1 ion (7.15)
Based on the condition on the region overdensity expressed in equation 7.15 we can use the same approach as the one used for EPS formalism of section 7.3.3. At a given location x, we compute the overdensity smoothed at a given mass scale m starting from m → ∞. If the smoothed overdensity respects condition 7.15, the region of mass scale m has enough source to self-ionized. If not, we compute the overdensity for a lower mass scale and check the condition again until we find the largest region where logically δ reg = δ lim . Contrary to EPS approach, our barrier density δ lim depends on the mass scale (through σ reg ). As for ST formalism, to have an analytical expression of the bubble size distribution, we need to use a linear fit of the barrier that will be B(m, z)

≡ B 0 + ∂δ lim ∂σ 2 σ 2 (m) with B 0 = δ lim (m → ∞) = δ c (z) - √ 2σ min erf -1 1 -ζ -1 ion .
With this fit, we can now write the comoving number density of HII regions with mass in the range [m, m + dm[ (i.e. the bubble size distribution) as (Furlanetto et al., 2004):

m dn b dm = 2 π ρ m dlnσ dlnm B 0 σ(m) exp - B 2 (m, z) 2σ 2 (m) (7.16)

Comparison with simulations

In Mesinger & Furlanetto (2007) and Zahn et al. (2007), the previous model of bubble size distribution is compared to distributions computed from numerical results. To obtain a bubble size distribution from the simulation they respectively use the mean-free-path (MFP) (see 7.1.2) and spherical-average (SPA) (see section 7.1.1) methods. Figure 7.3 represents the bubble size distribution using the quantity R dp dR for numerical simulations (plain lines) and analytical computation (dashed lines) for two studies : Mesinger & Furlanetto (2007) (left panel) and Zahn et al. (2007) (right panel). The size distribution is expressed using R dp dR where p(r) is the number density of bubbles when assuming purely spherical bubbles and convolving with the volume-weighted distance to the edge of the sphere. It is thus defined as (Mesinger & Furlanetto, 2007):

p(r)dr = 2πr 2 dr x H II R dn b dR 1 - r 2R dR (7.17)
The quantity R dp dR allows to normalize the distribution to unity rather than to the mean ionization fraction. However, for FOF algorithms, like the one we use in our study, this representation makes unnoticeable the largest percolated bubble that contains most of the ionized volume (Giri et al., 2018). The left panel is adapted from Mesinger & Furlanetto (2007) and shows the size distribution at (z, x H II ) = (10, 0.11) (red), (9.25, 0.21) (yellow), (8.50, 0.39) (green), ( 8.00, 0.55) (blue), (7.50, 0.73) (cyan) and (7.00, 0.90) (pink). The right panel is taken from Zahn et al. (2007) and shows the size distribution at (z, x H II ) = (8.16, 0.11), (7.68, 0.20), (7.26, 0.33), (6.89, 0.52) and (6.56, 0.77).

We see in Figure 7.3a that at any redshift (or alternatively global ionization fraction) the analytical model underestimates the sizes of the bubbles compared to Mesinger & Furlanetto (2007) simulation. Conversely, in Figure 7.3b the analytical and numerical results are more in accordance. This can be explained by the assumption of spherical bubbles made in the analytical model. The numerical method of Zahn et al. (2007) detailed in section 7.1.1 is also based on this assumption. However, as developed in Mesinger & Furlanetto (2007), the first sources often form close from each other in the denser area of the universe. The ionized bubbles they create therefore quickly overlap with each other. This overlap implies a, somewhat debatable, increase of the size distribution detected by "mean free path" construction methods used in Mesinger & Furlanetto (2007), hence the discrepancy. Even if the numerical methods to detect bubbles of the two simulation have limits, we see that the reference analytical model gives results with different bubbles scales and distribution shape compared to the numerical ones. This difference is likely because of its first assumption of a linear link between the ionized gas mass inside the bubble and the collapsed mass of the supposedly single halo inside it.

A bubble size distribution model for the Epoch of Reionization

This Chapter strictly and uniquely contains the soon-to-be submitted article "A bubble size distribution model for the Epoch of Reionization" whose authors are Aristide Doussot and Benoit Semelin. As such, some elements of context described in this chapter might have already been described in part I of this manuscript.

Abstract

The bubble size distribution is a summary statistics that can be computed from the observed 21-cm signal from the Epoch of Renionization. As it depends only on the ionization field and is not limited to gaussian information, it is an interesting probe, complementary to the power spectrum of the full 21-cm signal. Devising a flexible and reliable theoretical model for the bubble size distribution paves the way for using it for astrophysical parameters inference.

The proposed model is built to the excursion set theory and a prescribed functional relation between the bubble volume and the collapsed mass in the bubble. Unlike previous models, it can accommodate any functional relation or distribution. Using parameterized relations allows us to evaluate the predictive power of the model by performing a χ 2 minimization best-fit to the bubble size distribution obtained from a high resolution, fully coupled radiative hydrodynamics simulations, HIRRAH-21.

Our model is able to provide a better fit to the numerical bubble size distribution at ionization fraction of x H II ∼ 1% and 3% than other existing models, especially when a dispersion is included in the volume-to-collapsed-mass relation. Moreover, the relation corresponding to the best-fit parameters, which is not an observable, is directly compared to the numerical simulation data. A good match is obtained, confirming the possibility to infer this relation from an observed bubble size distribution using our model.

Introduction

The Epoch of Reionization (EoR) is the era when the formation of the first stars and galaxies triggered a major transformation of the intergalactic medium (IGM): from the cold and neutral state that followed recombination (at z ∼ 1000), it transitions to a warm ionized state. This transition is brought about by the emission of ionizing radiations from the first sources that create ionized bubbles which first grow and then percolate until the last islands of neutral hydrogen finally disappear. While the first star in the observable universe may have formed as early as z ∼ 65 (Naoz et al., 2006), the beginning of the reionization process, defined as the time when it becomes observable (through non-negligible contribution to the Thomson scattering optical depth, the 21-cm signal or individual high redshift proto-galaxies) is likely in the z = 15 -30 range. A more accurate timing depends on the very uncertain efficiency of the formation of the first stars that rely on a number of physical processes such as molecular Hydrogen formation and dissociation, the dynamical feedback from the first supernovae explosions and the subsequent chemical enrichment of the gas. The end of the process, that is the complete ionization of the IGM, occurs around z ∼ 6 (e.g. Fan et al., 2006;McGreer et al., 2015;Gangolli et al., 2021). Among the many possible probes of the EoR (galaxy luminosity functions, quasar proximity effects, Gamma-ray bursts, Lyman-alpha emitters observations), the 21-cm signal emitted by the neutral IGM holds a unique place. Indeed, rather than sampling the universe on individual lines of sight, it can provide a full 3D tomographic mapping of the EoR (see e.g. Furlanetto et al., 2006;Mellema et al., 2013, for reviews). While 21-cm signal statistics such as the global signal or the power spectrum are in principle observable with current instruments if systematics are brought under control (e.g. [START_REF] Bernardi | [END_REF]Bernardi et al., , 2010;;Ghosh et al., 2011;Yatawatta et al., 2013;Jelic et al., 2014;Asad et al., 2015;Remazeilles et al., 2015;Offringa et al., 2016;Procopio et al., 2017;Line et al., 2017), only the Square Kilometer Array should have sufficient sensitivity to perform tomography at 5 angular resolution.

The difficulty with interpreting observations of the 21-cm line is that it depends on several local quantities: the gas density, ionization fraction, kinetic temperature and velocity, and the local flux in the Lyman-alpha line for the coupling of spin temperature to the kinetic temperature. These values are in turn determined by non-local and non-linear processes: mainly gravity and radiative transfer, but also all the processes that regulate the formation of the sources of radiations. Consequently, one must build parameterized models, theoretical, semi-numerical or numerical, that attempt to account for all these effects and explore the parameter space in the hope of matching the observed signal. Obviously, the methodology to explore the parameter space efficiently (beyond brute force griding) has been an active field of research. For example, Pober et al. (2014) use the Fisher matrix formalism in combination with the 21cmFAST seminumerical code (Mesinger et al., 2011) to estimate the constraints that can be obtained from future observations with the HERA interferometer. [START_REF] Greig | [END_REF], 2017b, 2018) deploy a full Bayesian Markov-Chain-Monte-Carlo (MCMC) framework also based on 21cmFAST to compute the full posterior distribution for the model parameters. As even using a semi-numerical model at low resolution in conjunction with a MCMC approach represents a significant computational cost, some authors have explored using emulators of the semi-numerical model, using Gaussian Processes (Kern et al., 2017) or Neural Networks (Schmit & Pritchard, 2018;Jennings et al., 2019;Cohen et al., 2020;Bye et al., 2021;Bevins et al., 2021). Another avenue of research consists in training supervised learning algorithms to perform the inverse modelling, that is retrieve the model parameters from the signal (Shimabukuro & Semelin, 2017;Gillet et al., 2019;Doussot et al., 2019). While these works were limited to maximum-likelihood type inference, the posterior distribution could in principle be accessed using Bayesian Neural Network (see, e.g. Hortúa et al., 2020) or with density estimators also using neural network for dimensionality reduction (Zhao et al., 2021).

Most of these parameter inference studies use the 21-cm signal power spectrum as a metric of the difference between the model and reality, either as a proof of concept, as in the above references, or based on the existing upper limits (Greig et al., 2021b,a). Others have used the global signal (e.g. Monsalve et al., 2019) or, as a proof of concept only, the full tomography (e.g. Gillet et al., 2019;List & Lewis, 2020;Zhao et al., 2021). When using the global signal or the power spectrum, fluctuations induced by the ionisation fraction, the X-ray heating and the Lyman-α coupling are mixed. Only at times when one type dominates over the other can we hope to isolate it. Tomography, on the other hand, offers the possibility to disentangle the different contributions, for example by identifying individual ionized bubbles. However, as it is not a summary statistics, it suffers from a high level of thermal noise. This is why a summary statistics derived from the tomography and associated to a single source of fluctuation such as the Bubble Size Distribution (BSD) may be an interesting metric for parameter inference. It would likely provide constraints on the parameter space with different degeneracies than the 21-cm power spectrum for example. It is also sensitive to the non-gaussian information in the signal. While instrumental limitations, starting with the increasing thermal noise at higher angular resolution (e.g. Mellema et al., 2013), will affect the process, methods to obtain the bubble size distribution from tomographic observations have been explored (Giri et al., 2018;Bianco et al., 2021b). Semi-numerical, or even numerical simulations could then be used to perform parameter inference based on this quantity. If an accurate theoretical model can be devised however, it stands a good chance of involving a lower computational cost during the inference.

To our knowledge, there is currently only one theoretical model: it is described in Furlanetto et al. ( 2004) (hereafter FZH04) and relies on the excursion set theory (see also Paranjape et al. (2016) for some improvements on the model). The FZH04 model has been tested against numerical simulations in Mesinger & Furlanetto (2007) and Lin et al. (2016) who find some level of discrepancy. The fundamental assumption in the FZH04 model is that the mass of the ionized gas in each bubble is uniquely determined and directly proportional to the collapsed halo mass in the bubble. This is actually a necessary assumption for the model to be able to deliver an analytic formula. While this assumption is natural, it is also simplistic. Indeed, environmental effects such as source clustering, inhomogeneous recombination or external photo-evaporation, are bound to introduce some scattering into the one-to-one relation, but also induce a non-linear relation. Another limitation of the model is associated to the percolation process. Indeed, the excursion set formalism evaluates the probability for a region to be ionised, but does not account for the possibility that such regions overlap. Furlanetto & Oh (2016) have studied the process of percolation for the ionization field and shown that an ionized region with infinite volume (that is extending from one side of the box to the other in simulations) appears as soon as the average ionized fraction reaches x H II ∼ 0.1. There is no framework in the FHZ04 model to account for percolation.

In this work, we present an alternative, more flexible theoretical model to describe the BSD. By reverting to the original Press-Schechter spirit rather than using the excursion set theory, we loose something in terms of formal rigor, but we gain much in term of flexibility. Not only are we able to consider any functional relation between the ionized gas mass and collapsed halo mass, but we can also move from a one-to-one relation to a distribution characterized by its average and dispersion. We will show that this more flexible representation offers a good match to the result of a high-resolution numerical simulation. Then, a choice of an astrophysically-motivated parameterization for the relation opens the way for parameter inference using the BSD as a metric.

In section 8.3 we will present our methods: the reference numerical simulation, the algorithm for computing the numerical BSD and our proposed theoretical model. In section 8.4 we test the predictive power of the model by performing a best fit of the theoretical model parameters to the numerical BSD. The resulting best-fit ionized-gas-volume to collapsed-mass relation is then compared to the same relation directly computed from the simulation data. Section 8.5 evaluates the improvements produced by including a dispersion in the relation. Finally section 8.6 and 8.7 are devoted to discussion and conclusion.

The bubble size distribution: Numerical computation and analytical models

The HIRRAH-21 simulation

The High Resolution Radiative Hydrodynamics for 21-cm (HIRRAH-21) simulation was performed using the licorice code, detailed in Baek et al. (2009) (see Baek et al. (2010); Semelin (2016) for subsequent upgrades). licorice computes the evolution of gas, dark matter and radiation on cosmological scales. It is a particle-based code modeling the dynamics using the Tree-SPH method (Semelin & Combes, 2002, and references therein). The dynamics is fullycoupled to the radiative transfer of ionizing UV and X-rays using a Monte-Carlo scheme described in Baek et al. (2009Baek et al. ( , 2010)). Notable features easily implemented in the Monte Carlo approach are that the photons are propagated with the correct speed of light on an adaptive grid, and that cosmological redshifting is included. This is especially relevant for handling X-rays that can travel long distances before absorption. HIRRAH-21 is meant to complement the 21SSD set of simulations presented in (Semelin et al., 2017) and whose 21-cm signal predictions are available at https://21ssd.obspm.fr. The initial conditions for HIRRAH-21 were produced with the music package (Hahn & Abel, 2011) using the same random seeds as for the 21SSD simulations for the shared resolution levels, and a new one for the additional resolution level in HIRRAH-21 (see below). Thus the initial density field in HIRRAH-21 differs from those in 21SSD only by fluctuations at the newly resolved scales. For reference, the X-ray production parameters are f X = 1 and r H/S = 0.5 (see Semelin et al. (2017) for definitions), although in this work, where we focus on the ionised bubbles size distribution, such moderate values have little impact on the results.

HIRRAH-21 follows the evolution of 2048 3 particles, half of them baryons and the other half dark matter, in a 200 h -1 cMpc box. A parallelised halo finder based on a friend-of-friend scheme resolves halos down to 4 × 10 9 M (that is 20 dark matter particles). The dynamical timestep is 0.5 Myr (divided by 3 at expansion factor smaller than 0.03) and the gravitational softening is = 2 ckpc. The parameters of star formation and the ionizing photon escape fraction are the same as in 21SSD, yielding an earlier reionization history due to the better mass resolution for halos. The spectral properties of the sources are also the same as in 21SSD, resulting from a Salpeter initial mass function truncated at 1.6 M and 120 M . Roughly 5 × 10 11 Monte Carlo photon packets were propagated during the simulation, with 7 × 10 10 in-flight photons by the end of reionization which occurs around z ∼ 6.9.

For information, in Figure 8.1 we show the volume-weighted ionized (purple) and neutral (orange) fractions as functions of the redshift from the HIRRAH-21 simulation. The optical depth obtained from this simulation is τ ∼ 0.0692. We use a standard Λ-CDM comoslogy with parameter values in accordance with Planck Collaboration (2016): H 0 =67.8km.s -1 , Ω M =0.308, Ω b =0.0484, Ω Λ =0.692, σ 8 =0.8149 and n s =0.968. HIRRAH-21 was performed on 16386 cores using 4096 MPI domains and requiring ∼ 3 × 10 6 CPU hours.

Numerical computation of the bubble size distribution

To identify ionized bubbles in HIRRAH-21, we use a friend-of-friend method, based on the ionization field interpolated on a 1024 3 grid. It means that we can detect bubbles with a characteristic size of 288 ckpc. We tag as ionized any cell that has an ionization fraction above a given threshold, in this study 0.5. A bubble is a connected set of ionized cells. A similar algorithm has been used and analyzed in previous studies [START_REF] Friedrich | [END_REF]Furlanetto & Oh, 2016;Giri et al., 2018).

When the bubble is identified, we compute its volume as :

V bubble = V cell N cell i=0
x HII,i where V cell is the volume of one cell, N cell the number of cells in the considered ionized bubble and x HII,i the ionization fraction of cell i. If necessary we can then compute an equivalent radius from the volume assuming a spherical shape. Other methods like the mean-free-path or the sphericalaverage methods give similar but not stricly equivalent results (Mesinger & Furlanetto, 2007;Zahn et al., 2007;Giri et al., 2018).

We consequently produce a list of bubble volumes which can easily be converted into a number density of bubble in an interval of radius [r, r + dr] : n b (r)dr. In accordance with previous works (Furlanetto & Oh, 2016;Giri et al., 2018), to depict the bubble size distribution we will use the quantity V 2 dn b dV . This quantity measures the fraction of space occupied by ionized bubbles with volume V in a given logarithmic bin of volume dlnV . This is equivalent to the given in some works (Furlanetto et al., 2004;Lin et al., 2016;Giri et al., 2019) up to a constant 3 Q where Q is the global volume-filling fraction of ionized regions.

In Figure 8.2, we show the bubble size distribution from our HIRRAH-21 simulation at average ionization fractions of x H II ∼ 1%, 3%, 10% and 20%, which in our simulation correspond, respectively, to redshifts z ∼ 9. 8, 9.3, 8.5 and 8.1. At large volumes (V 10cMpc 3 ), the distribution drops for x H II ∼ 1% and 3% because we consider very early stages of reionization when bubbles have not yet had time to grow. The distribution is more or less flat for x H II ∼ 10%, and drops again for x H II ∼ 20%. The decrease in amplitude after x H II ∼ 10% is theoretically expected because of the percolation process that merges all large bubbles into a single one (Furlanetto & Oh, 2016). For smaller bubbles (V 5cMpc 3 ), the finite resolution of our simulation affects the results leading to a depletion in the number of detected bubbles.

The excursion set based model of the bubble size distribution

An analytical model to compute the bubble size distribution has been proposed by Furlanetto et al. (2004) (already refered to as FHZ04), based on the assumption that the mass of the gas in an ionized region is directly proportional to the mass of the collapsed objects inside the region, the proportionality factor being called the efficiency factor ζ. We refer the reader to Furlanetto et al. (2004) for the full details on the model. The end result is that the bubble size distribution is expressed analytically as : where m is the ionized gas mass in the bubble, dn is the number density of bubbles with gas mass between m and m + dm, ρ is the mean density of the universe, σ(m) is the mass variance of the linear density field and B(m, z) = B 0 +B 1 σ 2 (m) is a linear fit to the overdensity threshold needed so that the collapsed fraction in the region is sufficient to ionize it. To enable a comparison with our own model, we will convert the mass-dependency of the distribution in Equation 8.1 into a size dependency using the relation :

m dn dm = 2 π ρ m d ln σ d ln m B 0 σ(m) exp - B 2 (m, z) 2σ 2 (m) (8.1)
m = V ρ (1 + B(m, z)) .
Using a formalism derived from the extended Press-Schechter theory, this model relies on the assumption that the mass of ionized gas depends linearly on the mass of collapsed object to produce an analytical formulation for the BSD. Thus it can hardly take into account another functional form or a dispersion in this relation.

A new, flexible model for the bubble size distribution

Computing the conditional mass function

Our model for computing the BSD aims at using a flexible physical prescription to relate the amount of collapsed objects in a given ionized region to its volume. So the first task is to describe the population of collapsed objects in a given region of the universe. Since ionized bubbles typically form in overdense regions, one cannot simply use the Halo Mass Function (HMF) but we must rely on the conditional mass function (CMF) formalisms. A region of given volume V 0 with given overdensity δ 0 , has at mass M 0 = V 0 ρ and mass variance σ 0 = σ(M 0 ). In the following study, σ(M ) is computed using TheHaloMod online calculator (Murray et al., 2013(Murray et al., , 2021)).

By construction, our model can use any form for the conditional mass function. In this study we will mainly use the conditional mass function derived from the empirical best-fitting halo mass function of Sheth & Tormen (1999) that can be written (Rubiño-Martín et al., 2008):

n c,ST = 2 π ρ m dσ dm σ(m) |T (σ, σ 0 )| (σ(m) -σ 0 ) 3 2 exp - (B ec (σ, z) -δ 0 ) 2 2(σ(m) -σ 0 ) 2 (8.2)
where

T (σ, σ 0 ) = 5 n=0 (σ 2 0 -σ(m) 2 ) n n! ∂ n (B ec (σ, z) -δ 0 ) ∂(σ 2 ) n
with B ec the barrier derived by Sheth et al. (2001) and given by

B ec (σ, z) = √ 0.707δ c (z)   1 + 0.485 0.707δ c (z) 2 σ 2 -0.615   .
In section 8.6 we compare with the results for the earlier CMF derived in the extended Press-Schechter formalism (Press & Schechter, 1974;Bond et al., 1991;Lacey & Cole, 1993, 1994) to highlight our model sensitivity toward this initial choice for the CMF. This conditional mass function can be written as :

n c,EPS = 2 π ρ m dσ dm σ(m)(δ c -δ 0 ) (σ(m) -σ 0 ) 3 2 exp - (δ c -δ 0 ) 2 2(σ(m) -σ 0 ) 2 (8.3)
where δ c = 1.686.

The collapsed mass probability distribution

As long as the non-gaussianities resulting from non-linear structure formation have not had time to develop on the scale of the considered region, the probability density function for a region of volume V 0 to have an overdensity δ 0 is given by:

P V 0 (δ 0 ) = 1 √ 2πσ 0 exp - δ 2 0 2σ 2 0 . (8.4)
The collapsed mass probability density for a region of volume V 0 can be expressed as a marginalization over δ: (8.5) where P V 0 (M coll | δ) is the conditional probability for a region of volume V 0 to have a collapsed mass M coll knowing that it has an overdensity δ. We explore two ways to estimate this probability:

P V 0 (M coll ) = δ l -∞ P V 0 (M coll | δ)P V 0 (δ)dδ
• The simplest approach is to consider that there is no sample variance and that a region of volume V 0 with an overdensity δ * will always have the same content in terms of collapsed objects. In this case we can directly compute the collapsed mass using a generic conditional mass function n c :

M coll (δ * , V 0 ) = V 0 M 0 M min n c (δ * )mdm. (8.6)
The minimum mass threshold M min is necessary in our case since we are interested only in halos that can efficiently from stars and thus produce ionizing photons. It can be set for example at the atomic cooling limit ∼ 1 × 10 8 M , or at the resolution limit of a simulation, allowing our model to fit results in various frameworks. For this study we set M min ∼ 4 × 10 9 M corresponding to the resolution limit of the HIRRAH-21 simulation. With this assumptions, the conditional probability P V 0 (M coll | δ) is a simple Dirac distribution :

P V 0 (M coll |δ) = δ M coll -M coll (δ * , V 0 )
and we can finally rewrite the collapsed mass distribution inside regions of volume V 0 of Equation 8.5 :

P V 0 (M coll ) = P V 0 (δ * ) dδ dM coll (M coll (δ * )). ( 8

.7)

• A more realistic approach to this problem is to consider that different regions of the same size and average overdensity can have different collapsed object populations because they have different small-scale density fluctuations. Indeed the CMFs only describe the average collapsed object population in such a region. A simple way to take this sample variance into account is to model the number of object in each mass bin using a Poisson distribution (Barkana & Loeb, 2004). Note that since we are not interested simply in the total number of collapsed objects but in the total mass in collapsed objects, we cannot work with a single mass bin, relying on the fact that the sum of two independent Poisson distributed random variables is Poisson distributed: indeed this property does not work for a linear combination of independent random variables (the total collapsed mass in our case).

The Poisson distribution of the number of halos in a region of volume V 0 and overdensity δ * , with a mass in the range [M * , M * + dM ], must have an expected value equal to the average number of halos in this range, i.e. for a general conditional mass function n c : V 0 n c (M * , δ * )dM . This is sufficient to fully define the Poisson distribution. Therefore we can numerically construct the distribution P V 0 (M coll | δ * ) with a simple Monte Carlo sampling. In each mass bin, we draw a random variable X from the correct Poisson distribution for the number of halos, then we combine these values to compute the total collapsed mass, and iterate the process many times to build the total collapsed mass distribution. Since these distributions depend on two parameters (V 0 and δ * ), they can be tabulated in advance if necessary. We can finally apply Equation 8.5 to get the collapsed mass distribution marginalized over the overdensity.

Figure 8.3 represents the collapsed mass distribution computed at redshift z ∼ 9.8 including or not the Poisson distribution to model sample variance, and using CMF either from Equation 8.3 (blue) or Equation 8.2 (yellow), for various ionized regions sizes. The main difference induced by sample variance is for values of collapsed mass 10 10 M , i.e. values close to our simulation mass resolution of 4 × 10 9 M . It is expected since the Poisson distribution only differs from a Gaussian for a low number of occurrences. This explains why the difference between the two distributions is attenuated with growing region size or growing collapsed mass values. Furthermore as, at a given size, ionized regions are likely to be regions with the highest amount of collapsed mass, only the right-end of the distributions are relevant for the BSD computation. In this range, the only expected difference between BSD computed with or without sample variance will be for the smallest ionized bubbles, where one can expect that the sample variance will help to describe the effect of resolution limit. The low mass oscillations that can be seen in the cases when the Poisson distribution is used are generated by using a hard lower mass limit for halos. Indeed, between M min and 2 × M min the region contains at most 1 halos, with decreasing probability, as dictated by the CMF, as the mass of the halo increases. When the collased-mass reaches 2 × M min , suddenly configurations with 2 halos become possible, creating a sharp rise in the probability. Such a phenomenon occurs again, with decreasing amplitude, when integer numbers of M min are reached. 8.3 (blue) or of Equation 8.2 (yellow). The results are shown for ionized regions of radius of : 0.4 cMpc, 3 cMpc, 10 cMpc and 20 cMpc.

In Figure 8.3 we also see that the collapsed mass probability distribution peak is shifted to lower collapsed mass when using the CMF based on Press & Schechter (1974) compared to using the CMF based on Sheth & Tormen (1999), thereby showing the effect on the collapsed mass probability distribution of this well-known difference between the two theories (Sheth & Tormen, 1999;Rubiño-Martín et al., 2008). This difference impacts the resulting BSD as will be discussed later.

Parametrizable bubble size distribution

Having computed the collapsed mass distributions for a wide range of region volumes using Equation 8.5, we can now express the probability of a region of volume V 0 to be ionized as :

P ion (V 0 ) = P V 0 (M coll ) dM coll dV (V 0 ) (8.8)
where dM coll dV is computed from the physical relation M coll (V ion ) that links the volume of a ionization bubble with the collapsed mass it contains. One of the key properties of our model is that it allows to use various physical relations based on theory or simulations results alike. It can also be theoretically reversed to infer physical relation based on bubble size distributions as will be discussed in Section 8.4.2. The final step to compute the BSD is to write the number per unit volume of ionized bubbles with volume between V and V + dV as: (8.9) This last step is very much in the spirit of the original Press-Schecher theory (even if we do not rely on the corresponding Halo Mass Function). What it lacks in rigour it gains in flexibility in the M coll (V ion ) relation. The FHZ04 model follows the more rigorous approach of the extended Press-Schechter formalism, but is limited to a linear M coll (V ion ) relation without which an analytical solution to the diffusion equation does not exist.

n b (V ) = 1 V P ion (V ) dV.

Accounting for dispersion in the M coll (V ion ) relation

If the physical relation between the ionized bubble volume and the collapsed mass in the bubble displays a non-negligible dispersion, like for our HIRRAH-21 simulation, we can account for it. Given the collapsed mass distribution for a region of a volume V 0 , we use the distribution caracterizing the collapsed mass dispersion at this same volume P V 0 ,ion (M coll ) to obtain a modified distribution encapsulating this scatter :

P ion (V 0 ) = ∞ M min P V 0 (M coll )P V 0 ,ion (M coll ) dM coll . (8.10)
It is worth noting that, although we will suppose a gaussian behaviour for our following study, the dispersion distribution can theoretically take any form, be it analytical or numerical. Applying Equation 8.9 to our new distribution we can now compute a bubble size distribution corresponding to a physical relation with dispersion.

Model summary

For clarity sake, we hereby sum up the main computation steps of our analytical model. For regions of volume V 0 we compute n b (V 0 ) as follows :

1. We first compute P V 0 (δ 0 ) the probability density function for a region of volume V 0 to have an overdensity δ 0 using Equation 8.4.

2. We compute P V 0 (M coll | δ) the conditional probability for a region of volume V 0 to have a collapsed mass M coll knowing that it has an overdensity δ. This can be done in two ways :

• Assuming that the numerical sample variance effect induced by the simulation resolution limit is negligible it can be computed using a Dirac distribution and Equation 8.6.

• Taking into account the numerical sample variance effect implies to numerically construct P V 0 (M coll | δ) using a Poisson distribution and a Monte Carlo sampling as detailed in section 8.3.4.

3. We then marginalize over δ to obtain the collapsed mass probability density P V 0 (M coll ) for a region of volume V 0 using Equation 8.5

4. We define a parametrisable physical relation M coll (V ion ) that links the volume of a ionization bubble with the collapsed mass it contains.

5. The probability of a region of volume V 0 to be ionized P ion (V 0 ) is then computed differently depending if the dispersion is considered :

• Using Equation 8.8 if no dispersion is assumed

• If a dispersion is assumed, the functional form of its variance has to be defined.

We then access P ion (V 0 ) by marginalizing over the collapsed mass distribution (now approximated by its mean and variance) using Equation 8.10.

6. Finally n b (V 0 ) the number per unit volume of ionized bubbles with volume between V and V + dV is given by Equation 8.9.

8.4

Testing the model for parameter inference

Physical relation between the collapsed mass inside an ionized region and its volume

To compare our model with the numerical results of the HIRRAH-21 simulation, we select two functional forms to parametrize the relation between the volume of an ionized region and its collapsed mass content. While the physical insights would be to consider M coll ∝ V 1 we will let the exponent in both our parametrizations vary for now, a liberty that will be withdrawn later.

Power-law relation

The first and most straightforward choice is to assume a power-law relation which can be expressed as : 8.11) where α and A are two parameters to be fitted. Though convenient, this first parametrization will not be able to handle the effect of the mass resolution and should be limited to a range of volumes which is not affected by this limit, i.e., volume

M coll = AV α (
V such that V < M min A 1/α
where M min = 4 × 10 9 in our case.

Logarithmic Softplus

To better fit simulations and observations, we want to be able to account for the effect of the lower mass limit for star forming halos, that will exist physically, although at a lower mass than our simulation mass resolution limit. This physical limit is set by the temperature floor that can be reached by hydrogen atomic or molecular cooling . This suggests another parametrization that will be able to account for this change of regime (at the cost of a higher number of free parameters).

A possible choice for a function that must be flat in a given range (as the collapsed mass cannot be smaller than M min ) and linear above a given threshold is the Rectifier Linear function, better known as ReLu in the artificial intelligence domain and defined as y : x → α min(xx th , 0) + y th with, α, x th and y th are three free parameters and, in our case y ≡ log 10 (R) and x ≡ log 10 (M coll ). However this function is piecewise linear and numerical computation around the breakpoint tends to result in a brutal and artificial drop in the amplitude of the output BSD. For this reason we chose to implement a variation of this function without any breakpoints: the logarithmic Softplus.

log 10 M coll M coll,th = α ln 1 + exp k log 10 V V th k (8.12)
where α is the slope of the relation at high volume, k = 10 is the smoothness of the transition from one regime to the other and M coll,th and V th are the collapsed mass and corresponding volume at the transition from the regime dominated by the mass limit to a regime where it is negligible. We set k = 10, as the sharpness of the transition is not critical for this proof-ofconcept study, meaning that this parametrization effectively relies on three parameters.

x H II ∼ 1% x H II ∼ 3% Case χ 2 -BSD χ 2 -M(V) χ 2 -BSD χ 2 -M(V

Parameter inference

One key application of the model developed in this study is that it can be used to infer the relation M coll (V ) from the BSD and thus to constrain the underlying astrophysical processes. For that, one has to assume a parametric form for the relation and apply optimization algorithms like gradient-descent or Markov-Chain Monte-Carlo (MCMC) to fit a given observed BSD. For this study, we apply a simple grid-search method to illustrate the capabilities of this model. We also focus on low global ionization fraction (x H II ∼ 1% and x H II ∼ 3%) for which the effect of percolation is negligible.

In Figure 8.4 we use both of our power law (purple lines) and Softplus (orange lines) parametrization, with (dash lines) and without (plain lines) sample variance modelled by the Poisson distribution. We fit the BSD from the HIRRAH-21 simulation (gray histogram) to infer the parameters. The method is also applied at two different global ionization fraction of x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel). We also show the best-fit FHZ04 model for reference. We see that all our best fits are close to the numerical distribution inside their respected domain of definitions. Table 8.1 presents the mean square relative error of the BSD fitting process (χ 2 -BSD) and of the following physical M coll (V ) relation reconstruction (χ 2 -BSD) for both x H II ∼ 1% and x H II ∼ 3%. For all the current models, it shows that the fit error is, at worst, 17% but always under 10% for the Softplus parametrization.

However the theoretical BSD seems to overestimate the total ionization fraction as illustrated by their respective maximums that have an amplitude ∼ 50% greater than the numerical one. As expected, the decreasing population with decreasing volume due to the mass resolution limit is better fitted when using the Softplus model and the overall shape appears to be closer to the numerical reference. Our model also gives results closer to the numerical distribution for all cases than the FHZ04 model, especially at volumes V 10 2 cMpc 3 . Fitting the relatively smooth shape of the numerical BSD with 2 and 3 parameters models is not in itself a strong result. That the parameters and the model are based on a physical understanding of the system is more interesting. But showing that the inferred values for the parameters actually match what we can directly compute from simulations data would tell us that when we apply this procedure to an observed BSD, we will learn something relevant about the process of reionization. We show in Figure 8.5, for both x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel), the inferred physical relation M coll (V ) for the power law (purple lines) and Softplus (orange lines) parameterizations, with (dash lines) and without (plain lines) considering the or not (plain lines). A numerical fit of the reference BSD using Equation 8.1 from Furlanetto et al. (2004) inferring the efficiency parameter ζ is shown as a comparison (yellow dash line). The fit is done at x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel). sample variance, and the distribution for the same relation computed directly with the data from the simulation. We see that all parametrizations infer relatively reasonable physical relations. However all our inferred relations overestimate the average amount of collapsed mass inside a region of a given volume, especially for the smallest volumes (V 2 × 10 1 cMpc 3 ), compared to the average value directly computed from the simulation. Referring to Table 8.1 it leads to a M coll (V ) reconstruction error of ∼ 5% for model considering the sample variance and ∼ 2.5% for the others. This difference must be tempered by the possible variability of both the simulation and the model results. Concerning HIRRAH-21 simulation, a Friend-of-Friend algorithm is run on the particle files to construct halos. This halo finder especially relies on a linking length that roughly quantify the maximal distance for two particles to be considered as neighbors inside the same halo (Davis et al., 1985). Various values of the linking length can be considered reasonable and choosing one rather than another impact the resulting M coll (V ) relation. For the analytical model, it has been specifically designed to be adjustable and we show in Figure 8.3 that the collapsed mass probability already differs by a factor 2 between the two halo mass function formalisms depicted. The inferred M coll (V ) can therefore vary even for two different CMFs.

In Figure 8.4, we see that the difference induced by including the sample variance is relatively negligible in our case. In term of physical relation inference, Figure 8.5 and table 8.1 shows that models including sample variance perform worse than the others, being less in agreement with the numerical mean relation. In consequence, though it is theoretically more comprehensive to account for this effect, it appears that it lessen the inference capabilities of our model and, for simplicity, we will only look at cases where the sample variance is not considered in the following.

Injecting dispersion in the model

In section 8.4, we showed that for both of our parametrizations, our best fits BSD have a moderate deficit of large bubbles and excess of mid-sized bubbles. This problem could be partially solved if we consider a dispersion in the physical M coll (V ) relation. In our numerical simulation this dispersion is non-negligible, as can be seen in Figure 8.5. The effect of this dispersion could be important considering that a variation of the mean M coll results in a strong variation in term of collapsed mass probability as we can see in Figure 8.3.

Dispersion parametrizations

We already detailed, in section 8.3.4, the required modification to the model in order to include dispersion in the M coll (V ) relation. We now need to choose a parameterization. As shown in Figure 8.5, the dependence of the standard deviation as a function of V exhibits two different trends depending on the dominance of the mass resolution effect. However, the slopes are not sharp and to avoid adding too many free parameters, we decide to use the simple parameterization:

σ (M coll ) M coll = BV β (8.13)
with B and β as our free parameters. Furthermore, since in our cases without dispersion the slope α of our mean relation in Equations 8.11 and 8.12 were always close to the expected value of 1, we decide to fix it at this value, effectively withdrawing one free parameter for both mean relations. Using equation 8.13 along with the mean relations of Equation 8.11 we will be fitting a total of three free parameters while with Equation 8.12 we will be using four free parameters in total.

Fitting the bubble size distribution with models including dispersion

In Figure 8.6 we use both of our parametrizations detailed in sections 8.4.1 (purple lines) and 8.4.1 (orange lines) with (dashdotted lines) and without (plain lines) considering dispersion. We fit the BSD from the HIRRAH-21 simulation (gray histogram) to infer the underlying parameters of this parametrizations. The method is also applied at two different global ionization fractions of x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel). For both parametrizations, considering or not (plain lines). A numerical fit of the reference BSD using Equation 8.1 from Furlanetto et al. (2004) inferring the efficiency parameter ζ is shown as a comparison (yellow dash line). The fit is done at x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel).

the dispersion results in a decrease of the global amplitude of the theoretical best-fit BSD. On the whole range however, we see in table 8.1 that the numerical BSD is not better match when considering dispersion than when not considering it. In Figure 8.7 we show the inferred M coll (V ) relation with (dashdotted lines) and without (plain lines) including the dispersion for both our power-law (purple lines) and our logarithmic Softplus (orange lines) parametrizations. The cases with dispersion highlight the fact that the inference capability of our model strongly depends on the choice of an adequate functional form.

Here, the simple power-law model being less flexible and unreasonable for volumes where the mass resolution effect is dominant (V 2cMpc 3 ), the resulting theoretical BSD is unable to match the numerical BSD. To be as close as possible to the numerical BSD within the constraints of this functional form, the best-fit model overestimates the dispersion amplitude by a factor ∼ 3. On the contrary, when using the logarithmic Softplus relation of Equation 8.12, a functional form that is flexible enough to account for the mass resolution effect, the inferred mean M coll (V ) relation is in very good agreement with the numerical one, though the dispersion seems to be somewhat underestimated in this case. In table 8.1 we see that using the logarithmic Softplus relation in case of dispersion leads to an error 0.03% on the inferred physical relation M coll (V ) for both x H II ∼ 1% and x H II ∼ 3%.

Discussion

The previous analysis was performed using the conditional mass function of Equation 8.2 but, as stated in section 8.3.4 our model can use various functional forms for the conditional mass functions which will logically affect the resulting BSD. In Figure 8.8 we represent the resulting BSD at x H II ∼ 1% for the same power-law relation (inferred with the model using Sheth & Tormen (1999)) between the collapsed mass inside an ionized region and its volume using the CMF of Equation 8.2 based on Sheth & Tormen (1999) (purple line) and the CMF of Equation 8.3 based on Press & Schechter (1974) (black line). We observe an uniform decrease of the amplitude on the whole range for the CMF based on Press & Schechter (1974) compared to the one based on Sheth & Tormen (1999). The amplitude decrease can be explained by the fact that, for a given volume, the region that are ionized are most likely the ones with a large collapsed fraction. In Figure 8.3 we showed that the collapsed mass probability distribution peak is shifted to lower collapsed mass for Press & Schechter (1974) compare to Sheth & Tormen (1999). Therefore, a given volume is less likely to have a high collapsed mass for the former than for the latter, which consequently leads to an overall lower appearance of bubbles in Press & Schechter (1974) case for the same physical relation.

It is worth noting that the previous result is not a performance comparison between the two CMFs. Indeed the physical relation have been inferred from the CMF of Sheth & Tormen (1999). The theoretical BSD using Sheth & Tormen (1999) has thus been fitted to match the numerical BSD while the BSD using Press & Schechter (1974) CMF has not.

This comparison strongly emphasizes the fact that the choice of mass function formalism has an effect which is non-negligible. Furthermore, it places strong limits on the quality of the BSD fit one can hope to produce : even a perfect match between the modeled curve and the numerical one for a given CMF will be degraded using another one.

Conclusions

In this work, we develop a new analytical model to compute the ionized bubble size distribution of the Epoch of Reionization (15 z 5) starting from the underlying physical relation between the collapsed mass inside an ionized region and its size. We compare it to both, the fiducial model of Furlanetto et al. (2004) and numerical results from the HIRRAH-21 simulation for total ionization fractions of x H II = 1%, x H II = 3% and 10%. The HIRRAH-21 simulation is composed of 2048 3 particles in a 200 h -1 cMpc box and was performed using licorice code, which fully couples the hydrodynamics with the radiative transfer of ionizing UV and X-rays.

The key features of our models are :

1. The underlying physical relation between the collapsed mass inside an ionized region and its size is totally parametrizable, whereas the fiducial model (Furlanetto et al., 2004) cannot be parametrized by construction. One can use any functional form according to its own data or theories and obtain the resulting BSD. In this study we use two functional forms, a power-law and a logarithmic softplus, both in strong agreement with the numerical BSD directly computed from the HIRRAH simulation.

2. More than just the functional form, our model is largely adaptable to a lot of theoretical frameworks. As it is based on the halo conditional mass function theory, it can use various mass function formalisms such as Press & Schechter (1974) and Sheth & Tormen (1999) formalisms. One can chose to include the sample variance effect if needed and functional form for the dispersion in the underlying physical relation can also be implemented.

3. Using various optimization algorithms our model can be used to predict the underlying physical relation between the collapsed mass inside an ionized region and its size, granted that a functional form has previously been chosen. Though this model has to be tested using other results from independent simulations, for our two functional forms the resulting physical relation is in accordance with the computed relation from the HIRRAH-21 simulation displaying a parameter inference capability.

Bubble size distribution with percolation

The reionization as a percolation process

In physics, percolation is a case of phase transition and has been studied extensively in the field of condensed matter physics (Essam, 1980;Stauffer et al., 1994;Saberi, 2015). During a percolation process, a medium that was previously in one state undergoes a transition into another state starting from multiple points until all the medium is in the new state. It is often studied as the connectivity of many systems modeled as lattices and was already used in astrophysics to describe the transition from deterministic to stochastic behavior in the structure of spiral galaxies (Seiden & Schulman, 1990). Percolation can be characterized by the appearance of an infinitely large cluster1 (e.g. ionized bubble) in the new state, that will then extend until it consists of all the medium. In random percolation the time-reversed process is identical to the process itself. It means that we can equally characterize percolation by the disappearance of a previously infinitely large cluster in the former state. In figure 9.1, we show the evolution of the largest ionized bubble (red) in the HIRRAH-21 simulation at three different values of the global ionization fraction (x H II ∼ 0.1, 0.5, 0.9). It strongly emphasizes the similarity with a percolation process. However, due to spatial correlation between the sources, and therefore the bubbles, the percolation of the reionization is not time-reversible. Describing the history of reionization using percolation theory, we can distinguish three phases (Furlanetto & Oh, 2016) :

• The pre-overlap phase happens before the appearance of the infinitely large ionized bubble cluster. It is the very beginning of the reionization process where the analytical description of spherically independent bubbles can be best applied.

• The overlap phase starts when the infinitely large ionized bubbles cluster appears. During this phase, two clusters are visible as the infinitely large neutral region still exists.

Most of the reionization process happens during this phase.

• The post-overlap phase happens when the previously infinitely large neutral region is fragmented into gradually decreasing part surrounded by the ionized bubble cluster. These last neutral area are likely the most underdense regions where no sources were formed. Through this description, we already see the limits of the analytical models assuming independent bubble growth. Figure 9.2 presents a slice of the ionization field of the HIRRAH-21 simulations for different global ionization fractions. We see that, even in the pre-overlap phase, ionized bubbles are hardly independent. Indeed, their sources are spatially correlated as they preferentially form in the large-scale structures of the universe. As a result, ionized bubbles form close from each other and quickly overlap. It implies that analytical theories are even more limited to the very beginning of reionization. Because they do not include percolation, they will differ from simulation results even if very slightly at the beginning.

200h -1 Mpc x H II ∼1% 200h -1 Mpc x H II ∼3% 200h -1 Mpc x H II ∼10% 200h -1 Mpc x H II ∼30% 0
Another consequence of the early effect of percolation is that, even for numerical BSD estimation, the measurement result can vastly depend on the method itself and how it handles percolated bubbles. The SPA method detailed in section 7.1.1 is particularly sensitive to such effects and refined algorithms have been developed to tackle this issue (see Paranjape & Choudhury, 2014).

The percolation theory is centered around the notion of a critical threshold x c at which the infinitely large cluster appears suddenly. Around this threshold, the fraction of the box volume occupied by the percolating cluster follows a power-law whose exponent depends on the dimensionality of the problem and of the spatial correlation2 . In Figure 8.2 of chapter 8 we see that this critical threshold is around a global ionization fraction of x H II ∼ 10%, a result in accordance with previous studies (Furlanetto & Oh, 2016) that cannot be seen in figure 9.2 due to the 2D nature of such slices.

Model with Percolation

In chapter 8 we saw that our theoretical model for bubble size distribution performs well for global ionization of x H II = 1% and 3%. However, we see that the shape of numerical and theoretical BSD match less for the latter case. It is even worse for x H II = 10% (see figure 9.3) where the two distributions are really different from each other. This growing discrepancy can be explained by the growing impact of percolations. Our actual analytical model does not consider percolation and is thus currently limited to the early stages of reionization, namely the beginning of the pre-overlap phase. To lessen this constrain and keep inference capabilities for higher global ionization fraction, we have developed an independent algorithm.

Empirical percolation routine

The percolation algorithm is called after the analytical model has run. It takes as input a BSD V 2 dn b dV and empirically emulates the effect of percolation on the non-overlapping bubbles. The algorithm acts on each volume V i in order starting from the lowest and proceeds as follows :

1. First, we assume random, uncorrelated locations for bubbles centers. If the center of a bubble of a given volume V j > V i is closer than R i + R j (where R x is the radius corresponding to V x ) to the center of a bubble of volume V i they are overlapping. There is therefore an overlapping area around each bubble of volume V j . Considering all bubbles of size V j , the total volume where overlap with bubble of size V j can happen is :

V i|j over = f f 4π 3 (R i + R j ) 3 -V j dn b (V j )
where dn b (V j ) is the comoving number density of bubbles of volume V j and f f is a fudge factor detailed in section 9.2.2.

2. The number of bubbles of volume V i in this overlapping volume is computed through :

n i|j b = V i|j over dn b (V i )
Assuming that each percolation only involves one bubble of volume V i and one of volume V j it thus is the number of bubbles of the two volumes that will percolate.

3. There are n i|j b bubbles of volume V i and of volume V j that have percolated, the comoving number density of ionized bubbles at these two volumes is decreased accordingly :

dn b (V i ) = dn b (V i ) -n i|j b dn b (V j ) = dn b (V j ) -n i|j b
4. The bubbles that result from the percolation process are of volume V i + V j , the comoving number density of ionized bubbles at this volume is thus increased by

dn b (V i + V j ) = dn b (V i + V j ) + n i|j b
When all volume V j have been considered for a given volume V i , the algorithm selects the next volume (in increasing volume order) and repeats the previous steps.

A fudge factor for spatial correlations

The simple percolation algorithm presented in the previous section directly acts on the BSD. It is useful when one wants to add percolation effects to an already existing model, like the one we develop in chapter 8. However, such a design comes with a strong limit : no spatial information is considered. The algorithm acts as if the comoving number density of ionized bubbles was spatially constant. In practice, ionized bubbles form where the density of sources is the most important : in halos clusters and filaments. There is therefore a non-negligible spatial correlation between bubbles.

Describing this spatial correlation is a subject of studies in itself but the effective consequences of this correlation is that ionized bubbles are packed around high density regions. The probability of two bubbles overlapping is therefore higher than what it would be if the bubbles were randomly distributed in the volume. To simply account for this, we introduced a fudge factor f f that empirically modifies the number of overlapping bubbles at each step. In essence, the higher the fudge factor f f is, the clumpier the ionized bubbles are. Our algorithm can therefore roughly adapt to the spatial correlation, even though one parameter is hardly enough to encompass all the spatial information.

Implementation

The simplicity of this empirical percolation algorithm is well outlined by its implementation that can be done in few lines. Despite not encompassing exhaustively the percolation effect, it is a fast and useful tool to give insights on the consequences of this effect. We therefore present below a ready-to-use implementation in Python3 . Figure 9.3 shows the BSD for the numerical simulation (gray histogram) and the analytical model either without (black line) or with the addition of the percolation routine for different fudges factors (colored plain lines). A numerical fit of the reference numerical BSD using Equation 8.1 from Furlanetto et al. (2004) and inferring the efficiency parameter ζ is shown as a comparison (yellow dash line). As expected, the analytical model from Furlanetto et al. (2004) that assumed independent non-overlapping bubble growth does not match the numerical BSD neither in range nor shape. The analytical model developed in chapter 8 suffers from the same limitations even though it is more adaptable.

However, when adding the percolation routine to the model and determining a suitable fudge factor, the analytical BSD is able to match the reference one with relatively good precision in shape, range and amplitude altogether. We also see that the analytical BSD modification is also more pronounced with increasing fudge factor. Lastly, the match is better for lower global ionization fraction. It is expected as the percolation process in the pre-overlap phase is simpler and therefore better described by our simple percolation algorithm that considers spherical bubbles. During the overlap phase however, the percolated cluster is spatially highly irregular and undergoes many percolation with small bubbles at the same time.

The percolation algorithm thus allows our model to reasonably extend its prediction of the BSD until x H II ∼ 0.1 and the appearance of the infinitely large ionized cluster. After that, it only gives insights on the distribution of the remaining ionized bubbles that have not percolated with the percolated cluster yet. However, this simple model also has strong limitations:

• The main percolated cluster is hardly fitted. It may be due to the spatial irregularities of the most voluminous percolated bubbles that allow them to be close to one another for a time without overlapping while our algorithm, considering spherical shapes, percolates them. It leads to a too large predicted percolated clusters compared to the numerical one.

Another difference is that the numerical BSD is constructed from a box of size 200h -1 Mpc so the percolated cluster cannot be infinitely large. On the contrary, there is no box size limitation for our percolation algorithm. It theoretically considers an infinite volume and the percolated cluster size is therefore not limited to the simulation box size. The predicted global ionization fraction is also higher than the numerical one.

• We can see from figure 9.3 that the value of the fudge factor f f that produces the better match between simulation and theory is different for x H II ∼ 0.1 and x H II ∼ 0.25. To some extent, it can be explained by the different typical percolation situation of the two cases. At x H II ∼ 0.1, the bubbles that percolate are still close to their sources in overdense region such as halo cluster. They are therefore close to one another and their probability to overlap is higher than if they were spatially randomly distributed, hence a high fudge factor. At x H II ∼ 0.25 the percolated cluster may contain all bubbles from the overdense regions of the universe and the remaining percolation process happens in the remaining underdense regions where bubbles are typically farther from one another than if they were spatially more homogeneously distributed, hence a low fudge factor. This explanation is highly hypothetical however and this problem mostly reveals the limitations of our percolation model. Whatever the reason, a varying fudge factor makes the percolation algorithm harder to use in practice as we would need an empirical fit of the fudge factor evolution. The previous studies have been made using a fixed M (V ) relation directly fitted on the M (V ) data from the simulation. Only the fudge factor was a free parameter to fit in order to match the numerical BSD. We now free all parameters and test the parameter inference ability of our model with the percolation algorithm. In figure 9.4, we use Softplus parametrization considering dispersion (orange dash line) and fit the BSD from the HIRRAH-21 simulation (gray histogram) to infer the parameters. The method is also applied at two different global ionization fraction of x H II ∼ 0.1 (left panel) and x H II ∼ 0.25 (right panel). We see that we are able to match the numerical BSD in both cases even if, as expected from the previous case, the percolated cluster in x H II ∼ 0.25 is not accurately fitted. In figure 9.5 we show (orange dash line) the inferred physical relation M (V ) (assuming a softplus function and a power-law dispersion) corresponding to the previous match. The physical relation directly fitted from HIRRAH-21 results (green line) is shown for comparison. In both cases, the inferred M (V ) is in strong agreement with the numerically fitted one while the dispersion is relatively over-estimated. However, when trying to infer the parameter values for x H II ∼ 0.5, the inferred relation is no longer in accordance with the numerical one despite a relatively good match in term of BSD (if we except the percolated cluster). The influence of the percolated cluster might be too important at this point to correctly account for percolation using our simple percolation algorithm. Adding the percolation algorithm and its fudge factor f f as an additional free parameter thus allows to keep good parameter inference ability even for higher global ionization fraction (at the beginning of the overlap phase) x H II 0.25.

For parameters inference

Conclusion

Throughout this work, we aimed at a better determination of the astrophysical parameters describing the process of reionization. We started our journey using state-of-the-art tools like supervised learning methods to constrain these parameters with both good precision and computational efficiency. Building upon the results of a numerical simulation, we also delved into theory of structure formation to develop an analytical model linking ionized bubble distribution and the relation between bubbles and their collapsed matter content. We will now summarize what has been accomplished as well as discuss the perspectives.

We first aimed at developing a supervised learning method to better constrain astrophysical parameters at a lower computational cost. After presenting the basic concept of supervised learning and neural network in chapter 4, we looked at our case of study. Supervised learning methods need as many examples as possible to adequately learn to perform their tasks. The more complex the task is, the more data are needed. With this constrain in mind, we chose to use a semi-numerical code, 21cmFAST (Mesinger & Furlanetto, 2007), able to produce thousands of signals with varying astrophysical parameters in a reasonable amount of time. Among the parameters of the simulation, we selected the ionizing efficiency, the photon mean free path and the minimal halo mass to form stars for reconstruction. To ease the task of the algorithms, we assumed that non-gaussianities were negligible for the parameter inference task and thus used as input the power spectrum (at 12 wavenumbers for 10 redshifts) rather than the whole information of the lightcones. It dramatically decreased the dimensionality of the problem, and therefore the number of free parameters and number of examples needed. Conveniently, it also allowed us to use other supervised learning methods than neural networks, which are almost unavoidable for high dimensional problems. After some trials with various algorithms, we settled on kernel regressions with and without regularization (so-called ridge regressions). We spent a fair amount of time, tuning our methods to perform well on our peculiar case of study and presented them, as well as their results, in chapter 5.

Using an unaltered power spectrum, we obtained a reconstruction error of less than 1% on the parameters values for the best methods. Interestingly, the methods were efficient enough to help us uncover an unintended degeneracy in the model. The semi-numerical simulation considers range of values for the photon mean free path rather than the values themselves, as it was emphasized by our results where a piecewise constant function appeared in the representation of the predicted parameter value as a function of the real one. We thus re-labeled our examples accordingly by using the centers of each range of values. Having good performance with an unaltered signal was only the first step as observations are in practice altered by thermal noise (along with other instrumental effects). We thus applied a model of noise, developed to mimic the predicted thermal noise of the SKA, to create numerous noised realizations of each unaltered signal given by the simulation. After training our methods with these noised examples, the performances were strongly degraded as expected. Even after weighting the input signal by the inverse of the noise variance, effectively setting to negligible values bins that were dominated by the noise, the reconstruction error was still of the order of 8%. To further improve our predictions, we had to address a theoretical issue. One noised signal can originate, with different probabilities, from a variety of unaltered signals with different probabilities. Ideally, for a noised signal, we would have to produce distributions of the possible values of the underlying parameters rather than one value. However, kernel regressions and classical neural networks can hardly produce such distributions. If only one value can be inferred, the maximum-likelihood value is the most relevant. It implied modifying the training and test data sets to only contain noised realizations whose maximum-likelihood parameters correspond to the underlying parameters of the unaltered signal they have been produced from. This modification did not strongly affect the learning set as it is possible that the methods were doing this maximum-likelihood process by averaging the results on multiple noised signal corresponding to the same unaltered one. However, it was critical for the test set and the performance evaluation as noised signal were previously associated to other values than their maximum-likelihood parameters leading to an increase of the numerical error even if the methods were effectively predicting the maximum-likelihood value. In other words, with this specifically noised data set, we can isolate and quantify the reconstruction error of the network rather than computing a combination of both the reconstruction error and the uncertainty due to thermal noise. With this specifically noised data set, we were able to train methods with a reconstruction error of the order of 1%. The reconstruction error will thus be four times lower than the already existing uncertainty in the signal measurement for a real observation with SKA.

We have then attempted to better understand the source of dispersion in the relations between the ionized bubble volume and the luminosity of the halos they contain. The goal was to identify the astrophysical parameters that best encapsulate the source of this dispersion and therefore ease the inclusion of this important feature into semi-numerical models. We first thought that the dispersion might come from not considering the escape fraction. We then developed an algorithm that could compute this escape fraction for each halo in post-processing using halo and particle catalogs. We discovered an interesting anti-correlation between the escape fraction and the luminosity for halos of the same dark matter mass. It might be due to a higher gas content in more luminous halos. However, the dispersion in the relation between ionized bubble volume and halos luminosity did not decrease when considering the escaped luminosity rather than the intrinsic one. We then computed the clumping factor based on the density field. The difference in clumping factor was the main cause of the dispersion between ionized bubbles of the same luminosity. The clumping factor, therefore, encapsulates enough spatial information to accurately include physical dispersion in the relation between bubble size and halo luminosity.

After studying the causes and consequences of the dispersion in the physical relations between ionized bubble volume and halos characteristics of the numerical simulation HIRRAH-21, we focused on the bubble size distribution (BSD). More precisely, we aimed at developing a method that would link this possibly observable distribution to astrophysical relations. We used the HIRRAH-21 simulation as a reference and first detailed our bubble identification process in chapter 7. In this same chapter, we also introduced the halo mass functions formalisms which were both the theoretical groundwork of our analytical model and one of the required quantities to compute it. We finally described the reference bubble size distribution model (Furlanetto et al., 2004).

After several months of research, we completed an analytical model that links the BSD to the physical relation M (V ) between the ionized bubbles volume and the collapsed mass they contain. We presented it in chapter 8. Its main asset is its adaptability because it can be implemented with any conditional mass function formalisms, it can consider a dispersion in the physical relation or a sample variance in the HMF and is parametrizable as any functional form for the physical relation can be used. To estimate the predictive power of our model, we considered two functional forms of M (V ), a simple power-law and a softplus function inspired by the effect of the mass resolution limit in HIRRAH-21. We then let the physical parameters of M (V ) vary and match the numerical BSD from the HIRRAH-21 simulation with our analytical one. For global ionization fractions of 1% and 3%, the physical relation M (V ) corresponding to our BSD best fit was very close to the M (V ) computed directly from the simulation. It showed the parameter inference ability of our model at low ionization fraction.

The efficiency of our analytical model, however, quickly degraded with increasing redshift. As shown in chapter 9, it is mostly due to the percolation process where bubbles overlap, an effect that has not been successfully included in theoretical models so far. After emphasizing the percolation nature of the reionization process, we presented a simple process aimed at including the percolation effect to already existing bubble size distributions produced without it. Though fairly simple, this algorithm is both fast and easy to add to existing algorithms. It also produced good results until the dominance of the infinitely large percolation cluster (x H II ∼ 0.25) at the cost of an additional parameter to tune that account for the spatial correlation of the bubbles. This tunable parameter is still a strong limit of the process as it seems to vary with the global ionization fraction.

There are multiple paths that can be explored to improve on this work. First, supervised learning methods is a field far too wide to pretend that we explored it thoroughly. It is especially the case for neural networks where deeper and more complex architectures may lead to better results than what we are currently using. For example, the 21-cm power spectrum presents correlations both in wavenumber and redshift and locally-connected or convolution layers might be able to extract the information from such features.

More importantly, we restricted our study to the maximum-likelihood values but there are multiple methods to output a whole confidence interval. We can cite, for example, the method presented in [START_REF] Nix | Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)[END_REF] where the mean value of the distribution and its variance are each output by one neuron and the error function is modified accordingly. Another method would be to use Dropout layers both for learning and predicting. During the prediction process, each data will be submitted multiple times to the network that will predict slightly different value each time due to the random dropout of some of its weights. We can then build an empirical distribution of the prediction that can even be multi-modal (Gal & Ghahramani, 2015). One last route would be to use Bayesian Neural Network where the values of the weights are drawn from normal distribution whose mean and variance are the value that will be adjusted.

Another topic to study would be the training with very few examples (tens or hundreds at best) of the reconstruction method based on neural networks. If manageable, such conditions would allow to use fully coupled numerical-hydrodynamic simulations like LICORICE to construct the data set. It would prevent biases in the prediction due to the approximations made by the semi-numerical simulation that produce the data set. To achieve this task, it will probably be necessary to develop specific data augmentation methods.

For the analytical model that links the BSD to the physical relation V (M ), the main upgrade that can be done is a more thorough study of the percolation effect over the BSD. The implementation of such an effect begins with a more refined percolation process that would, for example, better take into account the spatial correlation rather than encompassing this crucial information into one parameter. It implies an in-depth modification of the current model. However, it will probably be the only way to accurately model the BSD after the appearance of the percolated cluster. It is crucial, if one wants to do parameter inference starting from this BSD.

A deeper understanding of the cause of dispersion, especially in the physical relation V (M ) would also help to determine the best possible parametrization (with as few free parameters as possible) for this relation. It might also be of help for semi-numerical simulation that would be able to better include the dispersion non-negligible effects in their process.

Finally, with the appearance of observational juggernauts like the SKA which will observe with better sensitivity and at a faster pace than any current facilities, the need of numerical methods to reduce and analyze data will grow accordingly. If artificial intelligence is no magical solution to any problem, it is, nonetheless, a powerful tool that we need to understand and master. Having a better understanding of our Universe also requires to develop new ways to infer the underlying model parameters value that, combined with already existing ways, put us one step closer to the reality. For comparison, we show the binned observational measurements at z ≈ 6 (Finkelstein et al., 2015;Bowler et al., 2015;Bouwens et al., 2015aBouwens et al., , 2017)), at z ≈ 8 (Finkelstein et al., 2015;Bouwens et al., 2015a), and at z ≈ 10 (Bouwens et al., 2015a(Bouwens et al., , 2016;;Oesch et al., 2018). For z ≈ 6 and z ≈ 8 our results are in agreement with the observational values and are still consistent with the uncertain observations for z ≈ 10.

fiducial model in Trac et al. (2015). This limit corresponds to the minimum mass for a dark matter halo to host a galaxy assuming that galaxies are formed only in halos where the gas cools efficiently through atomic transitions.

The galaxy luminosity functions for z ≈ 6, 8, and 10 as obtained during the simulation are shown in Figure A.2. We truncate the curves before the star formation limit and also show the observational measurements at z ≈ 6 (Finkelstein et al., 2015;Bowler et al., 2015;Bouwens et al., 2015aBouwens et al., , 2017)), at z ≈ 8 (Finkelstein et al., 2015;Bouwens et al., 2015a), and at z ≈ 10 (Bouwens et al., 2015a(Bouwens et al., , 2016;;Oesch et al., 2018). The GLF appears to be consistent with the observational constraints at z ≈ 6 and z ≈ 8. In the case of z ≈ 10, our luminosity function is still consistent with the observational results but has a larger amplitude at low M UV than expected from the observational measurements. However the uncertainties on the measurements at z ≈ 10 are high and our result matches the theoretical expectation from SCORCH I (Trac et al., 2015).

Radiation Escape Fractions

The escape fraction of ionizing photons f esc may be computed using high resolution radiation hydrodynamic simulations (e.g., Wise & Cen, 2009;Kimm & Cen, 2014;Yajima et al., 2014;Ma et al., 2015;Kimm et al., 2017;Trebitsch et al., 2017), although significant uncertainty remains. However, in our simulation the resolution of 24.4 h -1 kpc is insufficient to properly resolve the interstellar medium (ISM) and the circumgalactic medium (CGM) needed to self-consistently model it. We then use a parametric approach to model the escape fraction f esc .

In Price et al. (2016) we found that the new estimations of τ (Ade et al., 2016;Adam et al., 2016;Aghanim et al., 2016) implies a generic redshift evolution in the radiation escape fraction f esc (z). Moreover, a simple parametric form can be used to fit that evolution. Following these results, we then chose a two-parameter single power-law

f esc (z) = f 8 1 + z 9 a 8 (A.3)
where f 8 is the value of the escape function at z = 8. In our study, we compare the cases where a 8 = 0, 1, and 2 mainly to have a better understanding on the effect of that escape fraction on the reionization history. We chose to use Equation A.3 for all the galaxies independently of their masses. Indeed, the evolution of the escape fraction as a function of the mass of the galaxy is still uncertain, the existence of a correlation even being recently questioned (Yajima et al., 2014;Ma et al., 2015).

Recent measurements of the Thomson optical depth yield τ = 0.058 ± 0.012 (Adam et al., 2016) and τ = 0.054 ± 0.007 (Planck Collaboration et al., 2018). These values imply that the universe must be half-ionized at z ≈ 8 and by assuming that the reionization ends before z ≈ 5.5, the exponent a 8 in Equation A.3 is likely to be 0 a 8 2. Figure A.3 shows the behaviour of f esc as a function of the redshift for three typical values of a 8 . The parameter f 8 have been selected for the models to eventually match the value τ ≈ 0.06 which we choose to respect the two measurements. For a 8 = 0, 1 and 2, we have respectively f 8 = 0.111, 0.130 and 0.150. We also compare our assumed evolutions to two recent results. The first result is from Faisst (2016), which obtain f esc (z = 6) = 0.057 +0.083 -0.033 and f esc (z = 9) = 0.104 +0.155 -0.063 for galaxies with log(M/M ) ∼ 9.0, based on an empirical prediction of f esc made by combining the relation between [O III /O II ] and f esc with the redshift evolution of [O III /O II ] as predicted from local high-z analogs. The second result is from Greig & Mesinger (2017a), which obtain the constraint f esc (z ∼ 7) = 0.14 +0.26 -0.09 derived from a Bayesian framework which includes model-dependent priors from high-z galaxy observations using recent observations of z ∼ 7 faint, lensed galaxies.

Our three models are consistent with the latest predictions from Faisst (2016) and are in good agreement with Greig & Mesinger (2017a). Moreover, the overall normalization of f esc is degenerate with other parameters such as the overall normalization of the GLF and galaxy spectral energy distributions (SED) which may lead to some differences between different studies. Our models then broadly respect the expected escape fraction profile.

Ionizing Photons

The photoionization rate ṅγ (z), or the cumulative ionizing photon number density n γ (> z), is computed using the fraction of photons which have escaped from their original galaxy. The photoionization rate is then related to f esc through : ṅγ (z) = f esc × ṅγ,total (z).

(A.4)

For our approach we use the following formula for the production rate of ionizing photon of Population II star (Trac et al., 2015):

Ṅγ ≈ 10 46.2-0.4M UV s -1 ≈ 10 25.5 s -1 L U V erg s -1 Hz -1 (A.5)

where the conversion between UV magnitude and luminosity has been made with the standard AB relation, M UV = -2.5 log L U V 4.345 × 10 20 erg s -1 Hz -1 (A.6)

It is worth noting that Equation A.5 can be different from the one used in other works (Trac et al., 2015) because of the normalization which is uncertain to a factor of approximatively 2. Again, it emphasizes that we can only carefully compare our functional form of f esc with some observational constraints as these constraints are derived from a different computation of ṅγ (z). However, we hereby confront our results for ṅγ (z) and its cumulative n γ (> z) to another work from Bouwens et al. (2015b) as they do not depend on an arbitrary choice in their definition. In Figure A.4, we show the evolution of the photoionization rate per hydrogen atoms and of the cumulative ionizing photon number density per hydrogen atoms as functions of the redshift z for our three models of escape fraction. We show that our results differ with the one from Bouwens et al. (2015b) where a constant clumping factor of 3 is considered. It may be due to the fact that, in our study the clumping factor is not fixed and varies with the redshift resulting in a clumping factor always greater than 3. Moreover, the photoionization rate from Bouwens et al. (2015b) does not come from a simulation but from an analytical computation, a calculation that we have also done with our varying clumping factor in Doussot et al. (in prep). (2016), which are respectively τ = 0.054 ± 0.007 and τ = 0.058 ± 0.012, are also shown for comparison. Our models were calibrated based on Adam et al. (2016) and prior to the latest results (Planck Collaboration et al., 2018). We obtain results near τ ≈ 0.06, as planned by construction, and are in agreement with the measurements. However, if all of our models are consistent with the observational results, their temporal evolutions of τ are not similar which means that the reionization history is different for each one of them.

Ionization History

In Figure A.6, we show the volume and mass weighted ionization history, respectively x H II V and x H II M , for the different studied forms of f esc . We also show some of the latest results obtained from Lyman-α measurements (Schroeder et al., 2013;McGreer et al., 2015;Tilvi et al., 2014;Konno et al., 2018;Ota et al., 2017;Mason et al., 2018) and from Planck observations with a constraint on the end of the reionization before z ≈ 6 (Adam et al., 2016). Our results are generally in agreement with most of the experimental results cited in Figure A.6 and no strong contradictions seem to appear. Figure A.7, which presents slice of the reionization-redshift field for our three models, gives a first understanding of the spatial behaviour of the reionization process. We show that the x H II V,a8=0

x H II V,a8=1

x H II V,a8=2

x H II M,a8=0

x H II M,a8=1

x H II M,a8=2

McGreer2014 Tilvi2014 Schroeder2013 Planck Collaboration Konno2017 Ota2017 Mason2017

Figure A.6: Volume weighted (continuous) and mass weighted (dash) ionization fraction as a function of redshift for the three presented models with z(x H II = 0.5) ≈ 8. We show the latest results inferred from Lyman-α measurements (Schroeder et al., 2013;McGreer et al., 2015;Tilvi et al., 2014;Konno et al., 2018;Ota et al., 2017;Mason et al., 2018) and from Planck observations with the constraint z(x H II = 0.99) > 6 (Adam et al., 2016). reionization has the same spatial behaviour for our three models. The region around the galaxies are ionized first by the sources, the ionization grows until the whole universe is reionized and ends in the underdense cold intergalactic medium. Figure A.7 also emphasizes a strong difference between the cases as the reionization in case a 8 = 0 seems to have occurred in a shorter timescale than for the two other cases whereas the reionization in the case a 8 = 2 appears to extend

Duration and Asymmetry

We here study how the evolution of the radiation escape fraction affects the duration of reionization. It is hard to measure from simulations the physical processes at the very beginning and the very end of the reionization. Thus, we define two durations ∆z 50 ≡ z 0.25 -z 0.75 (A.7) ∆z 90 ≡ z 0.05 -z 0.95 (A.8) where z j ≡ z(x H II = j), to express the duration of the reionization like in some previous work (Zahn et al., 2012;Battaglia et al., 2013a). To better characterize the reionization history, we also define the asymmetry as following:

Az 50 ≡ z 0.25 -z 0.5 z 0.5 -z 0.75 (A.9)

Az 90 ≡ z 0.05 -z 0.5 z 0.5 -z 0.95 (A.10) Table A.1 present, using the mass-weighted and volume-weighted ionization history, the characterizing values of each model which emphasize the differences between their reionization histories. We see that the duration of the reionization is longer when the exponent a 8 in the power-law fit of f esc is larger. Moreover, these values suggest a simple relation for ∆z(a 8 ) and Az(a 8 ) even if other simulations with 0 a 8 2 are needed to determine it. For completeness, the trend in the evolution of the parameters is represented in Appendix A.1.5. See Trac (2018) for how the redshift midpoint, duration and asymmetry can be used to parametrize the reionization history.

From Table A.1 we also observe that, in all three cases, there is a non-negligible asymmetry on the duration of the part of the reionization before and after x H II = 0.5. The asymmetry is even larger when we consider a larger interval of the reionization history (i.e. 0.05 ≤ x H II ≤ 0.95 instead of 0.25 ≤ x H II ≤ 0.75). For our three simulations we then have that the first-half of the reionization (x H II < 0.5) is longer than the latter-half (x H II > 0.5). It can be explained by the fact that the photoionization rate ṅγ (shown in Figure A.4) gradually increases for a decreasing redshift. Hence, at the beginning of the reionization, the photoionization rate is at its lowest values explaining why the ionization process is slow.

We also show that, while ∆z increases for an increasing a 8 , the asymmetry decreases for an increasing a 8 . It is consistent with the temporal evolution of f esc and of ṅγ shown in Figures A.3 and A.4 because the higher the escape fraction is at the beginning of the reionization, the greater the number of ionizing photons escaping the galaxy is and so the quicker the ionization process is. Moreover the photoionization in the model a 8 = 2 is higher than in the two other models at high redshift and lower at low redshift. It implies an acceleration of the reionization at the beginning and a slowdown of the process at the end.

The duration of the reionization is also inferred in the Planck Collaboration (Adam et al., 2016) based on a joint analysis using the South Pole Telescope (SPT) measurements of the patchy kinetic Sunyaev-Zel'dovich (KSZ;Sunyaev & Zeldovich, 1970;Ostriker & Vishniac, 1986) effect angular power spectrum at l=3000 (George et al., 2015) and only our theoretical models from Battaglia et al. (2013b). Assuming that z 0.99 > 6, ∆z CMB ≡ z 0.1 -z 0.99 < 2.8 (95% confidence) (A.11) However this result depends on the assumptions made in the analysis and modelling of the patchy KSZ angular power spectrum. Using the definition A.11 of the duration, we obtain the results presented in Table A.2.

Our model a 8 = 2 does not match the constraint z 0.99 > 6 as it can be seen in Table A.3 but the two other models do respect it. Accordingly, our theoretical predictions and the observational constraints from the Planck Collaboration are currently in tension. However, there are some limits to emphasize about both works. From an observational point of view, there are still large uncertainties in isolating the KSZ effect and the power spectrum from the other components like the cosmological microwave background (CMB), the thermal Sunyaev-Zel'dovich (TSZ) effect and the cosmic infrared background (CIB) using only 3 frequency bands by the South Pole Telescope. From a theoretical point of view, the patchy and homogeneous KSZ components are still imperfectly modeled. There is also discrepancies in predictions for the homogeneous KSZ effect and power spectrum (e.g. Trac et al., 2011;Shaw et al., 2012) implying that more work is needed to improve the simulations. Furthermore, in Adam et al. (2016), our theoretical models from Battaglia et al. (2013b) are used to consider the homogeneous and patchy KSZ contibutions. It is important to note that the patchy KSZ effect from Battaglia et al. (2013b) are based on semi-numerical models that have only minor asymmetry compared to our current RadHydro simulations. Their patchy KSZ power spectrum are fitted based only on the value of z 0.5 and the duration of the reionization, but the asymmetry parameter is required for more precise constraints. Finally, there are some recent works that tend to show that quasars contribution to the reionization on large scale can accelerate the end of the process (Madau & Haardt, 2015;Chardin et al., 2015;D'Aloisio et al., 2017). In this case the duration of the reionization is shortened while the escape fraction f esc for the galaxy population stays the same. Consequently, the two results can probably be reconciled by removing some of these previous limits.

Temperature

Like the photoionization, the photoheating is also impacted by the evolution of the radiation and its behaviour in our simulations needs to be shown for completeness. the fact that the ionization fraction is higher in this case at this redshift. On the contrary, at z 7.5, the temperature in the model a 8 = 0 is higher than in the other models because of the fact that the ionization is higher for this model at this redshift. We also show that the maximal temperature is higher in the model a 8 = 0. It is due to the shorter duration of the reionization ∆z which implies that the effect of the adiabatic cooling process during the reionization, due to the universe expansion, is smaller that it could have been on a longer period. The photo-heating history of the simulations are in agreement with their photo-ionization history highlighting the self-consistency of our results.

After the end of the ionization process, in all cases, the photoheating process cannot stabilize the temperature at its higher values leading to a cooling of the gas. That cooling is in a more advanced state in the model with a 8 = 0 than in the other because the reionization ended up early giving to the gas more time to cool.

Despite their differences, the underlying physical process in all models should be the same as expected from a theoretical point of view. In Figure A.9, we show the normalized volume weighted average temperature of the sampling box as a function of the volume weighted ionization fraction of hydrogen x H II V . Analyzing T V as a function of x H II V and normalizing it by its maximal value allow us to cancel the influence of the different durations of the reionization ∆z of the models. As the behaviour is strictly the same for all models, we have another endorsement of the photo-heating evolution of the simulations matching the expectations. We also show a nearly linear dependency between T V and x H II V with a coefficient of proportionality of the order of 1.

We show in Figure A.10 the temperature distribution of the same slice of our sampling box at the same redshift for our three different models to give a first insight of the spatial distribution of the temperature. By referring to these temperature fields, we confirm that, as expected, the temperature evolution in our simulations is strongly linked to the reionization history and so changes with the assumed functional form of the escape fraction. At z ≈ 9.5 the increase of the temperature, revealing the position of the ionization front, is located around the sources where the reionization has started. At z = 5.5 it is worth noting that, in all models, the gas in the vast underdense intergalactic medium is hotter than the high density gas closer to the sources.

As previous work has emphasized it (e.g. Trac et al., 2008), it is because the gas in the IGM has been ionized later than the gas around the sources that the former has less time to cool than the latter. A broader study of the spatial distribution of the temperature and its heterogeneity depending on the density of the gas will be done in future work. For example, see D 'Aloisio et al. (2019) for a more in depth analysis of the heating of the IGM by hydrogen reionization.

A.1.5 Conclusion

The new RadHydro simulations based on the works of Trac et al. (2015) and Price et al. (2016) allow us to have a better understanding of the Epoch of Reionization and of the global behaviour of the parameters that constraint that epoch. In this study, we have presented the first main results of that simulation for three different cases. These cases come from the form of the escape fraction f esc which has been fitted as a simple power-law form and which has been shaped in three different ways: to be constant (a 8 = 0) as assumed in most previous works, to vary linearly (a 8 = 1), and to vary quadratically (a 8 = 2).

Each of these models matches the observational values of the optical depth τ from Adam et al. (2016) andPlanck Collaboration et al. (2018) and are then consistent with the observations. Based on these cases, we can isolate the f esc dependency of the reionization history. We concluded that the duration of the reionization ∆z increases with the increase of the exponent of the escape fraction's power-law. On the contrary the asymmetry Az between the beginning and the end of the reionization decreases with the increase of that exponent. However, our duration of the reionization conflicts with the result from Adam et al. (2016) which highlights that more studies as well as a better observation and modelization of the KSZ effect are needed.

In term of the photoheating, we pointed out that the increase in temperature happens during the ionization process, and then that there is a correlation between the temperature and the reionization history. We have also shown that the maximum value of the temperature is related to the duration of the reionization ∆z a shorter duration leading to lesser time for the adiabatic cooling process to act and consequently to a harsher heating. However, by normalizing the temperatures by their maximum values and showing them as a function of the ionization fraction x H II , it is relevant to think that the underlying photoheating process of the reionization stays the same whatever the model of escape fraction. Finally, after the ionization process, the gas starts to cool which spatially results in the fact that, at the end of the reionization, the underdense gas regions are hotter than the overdense gas regions which were ionized earlier and had more time to cool.

Appendix A : Convergence test

To study the f esc dependency of the duration of the reionization ∆z and its asymmetry Az, we hereby present our analysis of the convergence of these values by increasing the resolution. Figures A.11 and A.12 shows the duration of the reionization ∆z and the asymmetry of the process Az depending on the resolution using respectively the mass-weighted and the volumeweighted ionization history. As we presented it above, ∆z seems to increase and Az to decrease when a 8 increases. Here, we show that this conclusion does not depend on the resolution of the simulation. Moreover, especially in the case of the duration ∆z, the values appear to converge to a certain limit value when the resolution increases. That fact allow us to conclude that the previously deduced dependencies are likely true and are not just a result of our limited resolution. However more studies for other values of z 0.5 and then a 8 are required to really endorse that conclusion. In the Simulations and Constructions of the Reionization of Cosmic Hydrogen (SCORCH) project, we compare analytical models of the hydrogen ionization fraction with radiation-hydrodynamic simulations. We derive analytical models of the mass-weighted hydrogen ionization fraction from the local ionization balance equations as a more accurate alternative to the widely adopted model based on the volume filling factor. In particular, our model has a recombination term quadratic in the ionization fraction, which is consistent with the two-body interaction nature of recombination. Then, we use the radiation-hydrodynamic simulations to study the clumping factors needed to solve the analytical equations, and provide accurate fitting functions. We find that the ionized hydrogen clumping factors from our radiative transfer simulations are significantly different than those from other simulations that use a uniform photoionization background. In addition to redshift dependence, we also see the dependence of ionized hydrogen clumping factor on ionization fraction, and we incorporate this into our fits. We calculate the reionization histories using our analytical models and clumping factors and compare with widely adopted models, and all of our models achieve < 7% difference from simulation results while the other models have > 20% deviations. The Thomson optical depths from reionization calculated from our analytical models result in < 5% deviation from simulations, while the previous analytical models have > 20% difference in and could result in biased conclusions of the IGM reionization.

B

  Remerciements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Résumé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Preamble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Figure 1 . 1 :

 11 Figure 1.1: Schematic representation of the type of sources during the EoR along with the range where various cooling and star-forming mechanisms are effective.

Figure 1

 1 Figure 1.2: Representation of the ionized regions produced by the code LICORICE (see section 3.2).

Figure 1 . 4 :

 14 Figure 1.4: The 21cm transition as depicted on the Pioneer plaque that were placed on board the 1972 Pioneer 10 and 1973 Pioneer 11 spacecrafts.

Figure 1 . 8 :

 18 Figure 1.8: The 21-cm global signal as a function of redshift, for 193 different astrophysical models. The color indicates the ratio between the Lyman-α intensity (in units of erg s -1 cm -2Hz -1 sr -1 ) and the X-ray heating rate (in units of eV s -1 baryons -1 ) at the minimum point. grey curves indicates cases with τ > 0.09, and a non-excluded f X =0 case in black; these cases are all excluded from the colour bar range. Taken fromCohen et al. (2018).

Figure 2 . 1 :

 21 Figure 2.1: Representation of the major observing facilities of this century and their respective typical observation range on the electromagnetic spectrum.

Figure 2

 2 Figure 2.2: Representation of a short dipole antenna

Figure 2 . 3 :

 23 Figure 2.3: Array configurations for respectively : MWA Phase 1 (upper left panel), the central and inner remote stations of LOFAR (red) with a subplot showing the central stations only (blue)( upper right panel), GMRT (lower left) and PAPER (lower right panel). Based on Trott & Pober (2019).

Figure 2

 2 Figure 2.4: Current layout of NenuFAR in Nancay, France with the total number of available stations (i.e. mini-arrays (MA)) at each year from 2018 to 2020.

Figure 2 . 5 :

 25 Figure 2.5: Current upper limits on the power spectrum for modes k ∼ 0.1 -0.4 hcMpc -1 of the 21-cm signal. Signals predicted from 21cmFAST (Mesinger et al., 2011) (dotted black) and 21cmSSD (Semelin et al., 2017) (plain black) are shown for comparison.

Figure 2

 2 Figure 2.6: Schematic evolution of the 21-cm global signal. Reproduced from (Pritchard & Loeb, 2012).

Figure 2 . 7 :

 27 Figure 2.7: Current and future global 21-cm signal detectors.

Figure 2 . 10 :

 210 Figure 2.10: SKA1 main characteristics.

Figure 2 . 11 :

 211 Figure 2.11: Radio Interferometer comparison of sensitivity (left panel) and survey speed(right panel) as a function of the observed frequency. The expected performance of SKA1 (blue), SKA2 (orange), LOFAR (dark gray), uGMRT (gray), JVLA (light gray) and NenuFAR (red) are compared.

Figure 4

 4 Figure 4.1: Application possibilities of common Machine Learning algorithm. This list is by no means exhaustive as IA field is constantly evolving.

Figure 4 . 2 :

 42 Figure 4.2: Representation of a part of a neural network with the relevant notations.

Figure 4

 4 Figure 4.3: Example of an over-trained method that started to learn the specificities of its learning data set (red curve) rather than focusing on the general relation (blue curve).

Figure 4

 4 Figure 4.4: Example of data augmentation for image recognition, a problem with multiple symmetries.

Figure 4 . 5 :

 45 Figure 4.5: Comparison between RMSprop algorithm and a fiducial Stochastic Gradient Descent for a toy model.

  CHAPTER 4. ARTIFICIAL INTELLIGENCE most adapted to its architecture and activation function.

Figure 5

 5 Figure5.1: A toy model with a single parameter θ predicting a 2-valued power spectrum. The left panel shows how, when adding noise, two different parameter values can result in the same noised signal, neither of which would be the highest-likelihood parameter value (θ in this case). The right panel shows how the ambiguity disappears when considering noise realizations that are perpendicular to the model-manifold.

  Figure 5.2: Histogram of the number of cases in the test set composed of noised signal, that find a particular (σ, λ) duplet of hyper-parameters as being optimal for the prediction by the local ridge kernel regression method.

Figure 5 . 3 :

 53 Figure 5.3: Schematized architecture of the neural network used in this study.

Figure 5 . 4 :

 54 Figure5.4: Predicted values of R mfp as a function of the real ones for the theoretical perfect prediction (black line) and the actual predictions of our local ridge kernel regression method (yellow dots). For reference the 1σ uncertainty from SKA thermal noise as estimated from Bayesian inference[START_REF] Greig | [END_REF] is also plotted (purple cross).

Figure 5

 5 Figure 5.5: Normalized distribution of the prediction relative error δ = y pred y true -1 as a function of the true parameter y true for noise-free signals of the test set, for our three astrophysical parameters ζ ion , R mfp and T vir . The rms value of the error distribution is computed in each bin (blue line), and the 1σ uncertainty from SKA thermal noise as estimated from Bayesian inference (Greig & Mesinger, 2015) is plotted for comparison (green point). All these are plotted for different supervised learning methods, from top to bottom : the neural network, the global ridge kernel regression and the local ridge kernel regression.

Figure 5

 5 Figure 5.6: Normalized distribution of the prediction relative error δ = y pred y true -1 as a function of the true parameter y true for signals of the test set affected by perpendicularized noise as described in section 5.4.3, for our three astrophysical parameters ζ ion , R mfp and T vir . The rms value of the distribution is computed in different bins (blue line), and the 1σ uncertainty from SKA thermal noise as estimated from Bayesian inference (Greig & Mesinger, 2015) is plotted for comparison (green point). All these are plotted for different supervised learning methods, from top to bottom : the neural network, the global ridge kernel regression and the local ridge kernel regression.

4.WW

  Compute a local basis at the considered signal P ζ x ion ,R y mfp ,T z vir by making a weighted sum of the eight basis : (α, β, γ)e 2,α,β,γ(5.16)V 3,x,y,z = (α, β, γ)e 3,α,β,γ (5.17)

Figure 6 . 1 :

 61 Figure 6.1: Halos luminosity (in erg.s -1 ) as a function of their dark matter mass (in M ). Every point represents a halo of the HIRRAH-21 simulation and is colored according to its star mass content. The mean relation (plain black line) and the standard deviation (black dash lines) are also shown for comparison as well as the relative dispersion V ar(Y ) Y

Figure 6 . 2 :

 62 Figure 6.2: Ionized bubbles volume as a function of their inner halos luminosity for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. The halos of the simulation (red dots), the mean relation (plain black line) and the standard deviation (black dash lines) are also shown for comparison as well as the relative dispersion V ar(Y ) Y

4.

  Figure 6.3: Slices of the HIRRAH-21 simulation centered around two halos. The colormap represents the ionization field, the black circle the computed halo radius R halo , the white dot the halo center of mass x halo and the green dot its star particles.

  Figure 6.4: Escaped fraction as a function of the halo luminosity for halos of same dark matter mass (respectively 5 × 10 9 , 1 × 10 10 and 2 × 10 10 M ) in the HIRRAH-21 simulation at a global ionzation fraction of x H II ∼ 0.01.

Figure 6 .

 6 Figure 6.5 represents ionized bubbles volume as a function of their inner halos escaped luminosity for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. We see that this

Figure 6

 6 Figure 6.5: Ionized bubbles volume as a function of their inner halos escaped luminosity for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. The halos of the simulation (red dots), the mean relation (plain black line) and the standard deviation (black dash lines) are also shown for comparison as well as the relative dispersion V ar(Y ) Y in the bottom panel.

Figure 6 . 6 :

 66 Figure 6.6: Ionized bubbles volume as a function of their inner halos luminosity given a range of escape fraction for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. For each range of escape fraction we show the mean relation (plain lines) and the standard deviation (dash lines). The relative dispersions V ar(Y ) Y are also shown in the bottom panel for comparison.

Figure 6

 6 Figure 6.7: Ionized bubbles volume as a function of their inner halos luminosity given a range of clumping factor for the HIRRAH-21 simulation at a global ionization fraction of x H II ∼ 0.01. For each interval of clumping factor we show the mean relation (plain lines) and the standard deviation (dash lines). The relative dispersions V ar(Y ) Y are also shown in the bottom panel for comparison.

  Figure 7.1: Bubble size distribution at three different global ionization fractions (x H II = 0.01, 0.03 and 0.1) for three values of x lim H II : 0.1 (blue lines), 0.5 (red lines) and 0.7 (yellow lines).

Figure 7 . 2 :

 72 Figure 7.2: Schematic representation of the evolution of trajectories following a random walk. The right panel shows the distribution of trajectories as a function of overdensity for the last computed value of σ in the left panel. The black line shows the distribution of all trajectories whereas the red line shows the distribution of trajectories that crossed the barrier (i.e. lead to a collapse).

  Figure 7.3: Size distribution in R dp dR for numerical simulations (plain lines) and analytical computation (dashed lines).The left panel is adapted fromMesinger & Furlanetto (2007) and shows the size distribution at (z, x H II ) = (10, 0.11) (red), (9.25, 0.21) (yellow), (8.50, 0.39) (green), (8.00, 0.55) (blue), (7.50, 0.73) (cyan) and (7.00, 0.90) (pink). The right panel is taken fromZahn et al. (2007) and shows the size distribution at (z, x H II ) =(8.16, 0.11), (7.68, 0.20), (7.26, 0.33), (6.89, 0.52) and (6.56, 0.77).

  Figure 8.1: Volume-weighted ionized (purple) and neutral (orange) fraction as function of the redshift from the HIRRAH-21 simulation.

Figure 8 . 2 :

 82 Figure 8.2: Bubble size distribution from the HIRRAH-21 simulation at global ionization fraction x H II ∼ 1%, 3%, 10% and 20%.

Figure 8 . 3 :

 83 Figure 8.3: Collapsed mass distribution computed at redshift z ∼ 9.8 (corresponding to x H II ∼ 1%) considering the sample variance (continuous) or not (dash) using the conditional mass function of Equation8.3 (blue) or of Equation8.2 (yellow). The results are shown for ionized regions of radius of : 0.4 cMpc, 3 cMpc, 10 cMpc and 20 cMpc.

Figure 8 . 4 :

 84 Figure 8.4: Various fit of the reference numerical Bubble Size Distributions (gray histogram) using power-law parametrization from section 8.4.1 (purple lines) or logarithmic Softplus parametrization from section 8.4.1 (orange lines) considering the sample variance (dash lines) or not (plain lines). A numerical fit of the reference BSD using Equation8.1 fromFurlanetto et al. (2004) inferring the efficiency parameter ζ is shown as a comparison (yellow dash line). The fit is done at x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel).

Figure 8 . 5 :

 85 Figure 8.5: Collapsed mass inside an ionized region as a function of its volume. The gray colormap represents the distribution of halos the HIRRAH-21 simulation and its numerical mean and dispersion are shown as green lines in the corresponding panels. The inferred relations with (dash lines) and without (plain lines) considering the sample variance are shown for both our power-law (purple lines) and our logarithmic Softplus (orange lines) parametrizations. The fit is done at x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel).

Figure 8 . 6 :

 86 Figure 8.6: Various fit of the reference numerical Bubble Size Distributions (gray histogram) using power-law parametrization from section 8.4.1 (purple lines) or logarithmic Softplus parametrization from section 8.4.1 (orange lines) considering the dispersion (dashdotted lines) or not (plain lines). A numerical fit of the reference BSD using Equation8.1 fromFurlanetto et al. (2004) inferring the efficiency parameter ζ is shown as a comparison (yellow dash line). The fit is done at x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel).

Figure 8 . 7 :

 87 Figure 8.7: Collapsed mass inside an ionized region as a function of its volume. The gray colormap represents the distribution of halos the HIRRAH-21 simulation and its numerical mean and dispersion are shown as green lines in the corresponding panels. The inferred relations with (dashdotted lines) and without (plain lines) considering the dispersion are shown for both our power-law (purple lines) and our logarithmic Softplus (orange lines) parametrizations. The fit is done at x H II ∼ 1% (left panel) and x H II ∼ 3% (right panel).

Figure 8 . 8 :

 88 Figure 8.8: Bubble Size distribution at x H II ∼ 1% using the same parameters (inferred with the model using Sheth & Tormen (1999)) in Equation 8.11 using the CMF of Equation 8.2 based on Sheth & Tormen (1999) (purple line) and the CMF of Equation 8.3 based on Press & Schechter (1974) (green line). The Numerical BSD from the HIRRAH-21 simulation and a numerical fit of the reference BSD using Equation 8.1 from Furlanetto et al. (2004) inferring the efficiency parameter ζ (yellow line) are shown for comparison.

Figure 9 . 1 :

 91 Figure 9.1: Representation of the simulation box of HIRRAH-21 at three different values of the global ionization fraction (x H II ∼ 0.1, 0.5, 0.9). the largest ionized bubble (red) and the other ionized bubbles (purple) are shown.

Figure 9 . 2 :

 92 Figure 9.2: Slice of the ionization field of the HIRRAH-21 simulations (see section 8.3.1) at three different values of the global ionization fraction : x H II ∼ 0.01, 0.03, 0.1 and 0.3.

25 Figure 9

 259 Figure 9.3: Bubble Size Distributions a function of the comoving volume. The numerical BSD (gray histogram) is compared to BSD produced from our analytical model detailed in 8 when using the numerical relation between the bubble volumes and the collapsed mass. The results are shown either without the percolation routine (black line) or with it for three differents fudge factor values. A numerical fit of the reference numerical BSD using Equation 8.1 from Furlanetto et al. (2004) and inferring the efficiency parameter ζ is shown as a comparison (yellow dash line). The results are shown for global ionizations fractions of x H II ∼ 0.1 (left panel) and x H II ∼ 0.25 (right panel).

Figure 9 . 4 :

 94 Figure 9.4: Fit of the reference numerical Bubble Size Distributions (gray histogram) using softplus parametrization, the percolation algorithm and considering physical dispersion (orange dash line). A numerical fit of the reference BSD using the analytical model fromFurlanetto et al. (2004) and inferring the efficiency parameter ζ is shown as a comparison (yellow dash line). The fit is done at x H II ∼ 0.1 (left panel) and x H II ∼ 0.25 (right panel).

Figure 9 . 5 :

 95 Figure 9.5: Collapsed mass inside an ionized region as a function of its volume. The gray colormap represents the distribution of halos in the HIRRAH-21 simulation and its numerical mean and dispersion are shown as green lines in the corresponding panels. The inferred relation considering the dispersion is shown for the logarithmic Softplus parametrization (orange dash lines). The fit is done at x H II ∼ 0.1 (left panel) and x H II ∼ 0.25 (right panel).

Figure A. 2 :

 2 Figure A.2: Galaxy luminosity function (binned) as a function of the UV magnitude at z ≈6 (red), z ≈ 8 (green), and z ≈ 10 (blue) as obtained for the SCORCH simulations. For comparison, we show the binned observational measurements at z ≈ 6(Finkelstein et al., 2015;Bowler et al., 2015; Bouwens et al., 2015aBouwens et al., , 2017)), at z ≈ 8(Finkelstein et al., 2015; Bouwens et al., 2015a), and at z ≈ 10 (Bouwens et al., 2015a, 2016;Oesch et al., 2018). For z ≈ 6 and z ≈ 8 our results are in agreement with the observational values and are still consistent with the uncertain observations for z ≈ 10.

Figure

  Figure A.3: Escape fraction as a function of redshift following the Equation A.3 for our three models. The observationally-based predictions of Faisst (2016) and Greig & Mesinger (2017a) are shown for comparison.

Figure A. 4 :

 4 Figure A.4: Photoionization rate per hydrogen atoms (left) and cumulative ionizing photon number density per hydrogen atoms (right) as functions of the redshift z for our three functional forms of f esc with a 8 = 0 (red), a 8 = 1 (blue), and a 8 = 2 (green). We show for comparison the observational results, based on HST observations, from Bouwens et al. (2015b) (shaded grey) which considers a constant clumping factor.

Figure A. 5 :

 5 Figure A.5: Optical depth as a function of the redshift for our three forms of f esc . The optical depth of τ = 0.054 ± 0.007 from Planck Collaboration et al. (2018) (light blue) and τ = 0.058 ± 0.012 from Adam et al. (2016) (light red) are shown for comparison. The optical depths of our three models are in agreement with both measurements.

Figure

  Figure A.5 shows the obtained optical depth τ for our three models as a function of the redshift. The observational measurements of Planck Collaboration et al. (2018) and Adam et al.(2016), which are respectively τ = 0.054 ± 0.007 and τ = 0.058 ± 0.012, are also shown for comparison. Our models were calibrated based onAdam et al. (2016) and prior to the latest results(Planck Collaboration et al., 2018). We obtain results near τ ≈ 0.06, as planned by construction, and are in agreement with the measurements. However, if all of our models are consistent with the observational results, their temporal evolutions of τ are not similar which means that the reionization history is different for each one of them.

  Figure A.7: Image of a slice of size (50 h -1 Mpc) 2 of the reionization-redshift field (the value of one cell is the redshift when it has been ionized) for both of our three models at the end of the reionization.

Figure A. 8 :

 8 Figure A.8: Volume weighted average temperature T V as a function of redshift for the three models.

Figure A. 9 :

 9 Figure A.9: Normalized volume weighted average temperature of the sampling box as a function of the volume weighted ionization fraction of hydrogen x H II for the three models.

Figure A. 10 :

 10 Figure A.10: Image of a slice of size (50 h -1 Mpc) 2 of the temperature field for our three models at z ≈ 9.5 (top) and z = 5.5 (bottom).

Figure A. 11 :

 11 Figure A.11: Characterizing values of the reionization as a function of a 8 and using the massweighted ionization history x H II M while varying the resolution of the simulation with: at left z 0.5 the redshift at which x H II M = 0.5, in the middle the duration of the reionization ∆z 50 (continuous) and ∆z 90 (dash) and at right the asymmetry Az 50 (continuous) and Az 90 (dash).

  Figure A.12: Characterizing values of the reionization as a function of a 8 and using the volumeweighted ionization history x H II V while varying the resolution of the simulation with: at left z 0.5 the redshift at which x H II V = 0.5, in the middle the duration of the reionization ∆z 50 (continuous) and ∆z 90 (dash) and at right the asymmetry Az 50 (continuous) and Az 90 (dash).

Table 2 .

 2 1: Basic features of the instruments that are currently undertaking EoR observations. Based on[START_REF] Trott | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF].

  Local Ridge Kernel Regression LS 2.69 ×10 -2 3.79 ×10 -2 0.48 ×10 -2 Global Ridge Kernel Regression LS 1.36 ×10 -2 3.78 ×10 -2 0.30 ×10 -2Neural Network 1.37 ×10 -2 2.53 ×10 -2 0.15 ×10 -2 Global Ridge Kernel Regression TS 0.68 ×10 -2 1.76 ×10 -2 0.09 ×10 -2 Local Ridge Kernel Regression TS 0.39 ×10 -2 0.19 ×10 -2 0.04 ×10 -2 Global Ridge Kernel Regression TS 2.88 ×10 -2 2.84 ×10 -2 0.34 ×10 -2 Local Ridge Kernel Regression TS 1.10 ×10 -2 0.60 ×10 -2 0.16 ×10 -2 Table 5.4: Rms relative errors χ of the predictions for respectively : signals without noise (top) and signals affected by perpendicularized noise (bottom). The χ have been computed individually for each parameter, for the global and local ridge kernel regression presented in section 5.4 and optimized using the learning set (LS) or the test set (TS). The signals of the learning set have been noised with a perpendicularized noise. Results for our neural network are shown for comparison.

	Without noise	χ ζ ion	χ R mfp	χ log(T vir )
	With noise	χ ζ ion	χ R mfp	χ log(T vir )
	Local Ridge Kernel Regression LS 10.3 ×10 -2 14.1 ×10 -2 4.23 ×10 -2
	Global Ridge Kernel Regression LS 2.89 ×10 -2 2.83 ×10 -2 0.34 ×10 -2
	Neural Network	3.70 ×10 -2 4.04 ×10 -2 0.41 ×10 -2

Table 8 .

 8 1: Mean square relative error of the BSD fitting process (χ 2 -BSD) and of the physical M coll (V ) relation reconstruction (χ 2 -BSD) for all the models discussed in sections 8.4.2 and 8.5.2 at two global ionization fraction x H II ∼ 1% and x H II ∼ 3%.

	)

Table A .

 A 1: Characterizing Values of each Model using the Mass-Weighted Ionization History x H II M and the Volume-Weighted Ionization History x H II V . x H II M x H II V Model z 0.5 ∆z 50 ∆z 90 Az 50 Az 90 z 0.5 ∆z 50 ∆z 90 Az 50 Az 90 a 8 = 0 7.96 1.87 4.68 1.63 2.89 7.63 1.43 3.79 1.64 3.09 a 8 = 1 7.91 2.27 5.45 1.59 2.69 7.51 1.77 4.53 1.64 2.99 a 8 = 2 7.83 2.89 6.54 1.49 2.33 7.30 2.33 5.61 1.59 2.71 x H II M x H II V Model t 0.5 ∆t 50 ∆t 90 At 50 At 90 t 0.5 ∆t 50 ∆t 90 At 50 At 90 a 8 = 0 6.88 2.06 4.34 1.26 1.61 7.27 1.74 3.90 1.34 1.88 a 8 = 1 6.93 2.53 5.12 1.17 1.36 7.43 2.21 4.72 1.27 1.65 a 8 = 2 7.71 3.31 6.50 1.00 1.02 7.71 3.09 6.21 1.13 1.28 itself in a larger time-scale.

Table A .

 A 2: Duration of the Reionization with the Definition A.11 using Respectively the Mass-Weighted Ionization History x H II M the Volume-Weighted Ionization History x H II V . CMB, x H II M 3.9 4.6 5.7 ∆z CMB, x H II V 3.1 3.8 4.8

	a 8	0	1	2
	∆z

Table A .

 A 3: Characteristic Quantities from SCORCH for a 8 = 0, 1, and 2.

	A.2. SCORCH. III. ANALYTICAL MODELS OF REIONIZATION WITH VARYING CLUMPING FACTORS
	A.2
			a 8 = 0			a 8 = 1			a 8 = 2
	z	x H II M	x H II V	nγ (>z) n H	x H II M	x H II V	nγ (>z) n H	x H II M	x H II V	nγ (>z) n H
	13.5	0.009	0.002	0.021	0.014	0.005	0.032	0.023	0.01	0.049
	13.25	0.011	0.003	0.026	0.018	0.006	0.039	0.028	0.012	0.058
	13.0	0.014	0.004	0.032	0.022	0.008	0.047	0.033	0.015	0.068
	12.75	0.017	0.005	0.039	0.026	0.01	0.056	0.04	0.018	0.081
	12.5	0.021	0.007	0.047	0.032	0.013	0.067	0.047	0.022	0.095
	12.25	0.026	0.009	0.057	0.038	0.016	0.08	0.055	0.027	0.112
	12.0	0.032	0.012	0.069	0.046	0.02	0.096	0.065	0.032	0.131
	11.75	0.039	0.015	0.084	0.055	0.024	0.113	0.076	0.039	0.152
	11.5	0.048	0.019	0.101	0.065	0.03	0.134	0.089	0.046	0.177
	11.25	0.058	0.024	0.121	0.078	0.037	0.158	0.103	0.055	0.206
	11.0	0.07	0.031	0.145	0.092	0.046	0.187	0.12	0.067	0.238
	10.75	0.085	0.04	0.174	0.109	0.057	0.219	0.138	0.079	0.275
	10.5	0.102	0.05	0.206	0.128	0.069	0.255	0.159	0.093	0.315
	10.25	0.121	0.062	0.243	0.148	0.083	0.296	0.18	0.109	0.358
	10.0	0.143	0.077	0.285	0.171	0.1	0.341	0.204	0.127	0.406
	9.75	0.168	0.095	0.333	0.197	0.118	0.391	0.229	0.146	0.458
	9.5	0.197	0.116	0.388	0.225	0.14	0.447	0.256	0.168	0.515
	9.25	0.23	0.141	0.45	0.257	0.165	0.51	0.285	0.191	0.576
	9.0	0.268	0.171	0.521	0.292	0.194	0.579	0.317	0.218	0.642
	8.75	0.312	0.209	0.602	0.331	0.228	0.655	0.351	0.248	0.714
	8.5	0.362	0.254	0.693	0.375	0.268	0.74	0.387	0.281	0.791
	8.25	0.42	0.309	0.797	0.424	0.314	0.834	0.426	0.318	0.875
	8.0	0.487	0.375	0.915	0.478	0.367	0.938	0.469	0.359	0.964
	7.75	0.563	0.456	1.049	0.539	0.43	1.052	0.514	0.404	1.061
	7.5	0.65	0.553	1.202	0.606	0.503	1.179	0.563	0.456	1.164
	7.25	0.747	0.667	1.374	0.679	0.586	1.318	0.615	0.512	1.275
	7.0	0.851	0.798	1.57	0.76	0.682	1.471	0.67	0.574	1.393
	6.75	0.951	0.931	1.792	0.845	0.789	1.64	0.729	0.643	1.519
	6.5	0.999	0.998	2.044	0.93	0.903	1.824	0.791	0.718	1.652
	6.25	1.0	1.0	2.328	0.989	0.985	2.026	0.854	0.799	1.793
	6.0	1.0	1.0	2.65	1.0	1.0	2.247	0.919	0.886	1.943
	5.75	1.0	1.0	3.011	1.0	1.0	2.485	0.972	0.96	2.098
	5.5	1.0	1.0	3.425	1.0	1.0	2.75	0.999	0.999	2.264

SCORCH. III. Analytical Models of Reionization with Vary- ing Clumping Factors 1

Table B .

 B Light speed G 6.6743 × 10 -11 m 3 .kg -1 .s -2 1: Astrophysical constants and parameter with the associated values that we use throughout this work.

				Constant Table
	Symbol Value	Unit	Name
	k b	1.38	J.K -1	Boltzmann constant
	c	3 × 10 8	m.s -1
				Gravitational constant
	H 0	67.8	km.s -1 .Mpc -1 Hubble constant
	Ω m	0.3175	∅	Matter fraction at z = 0
	Ω b	0.049	∅	Baryon fraction at z = 0
	Ω Λ	0.6825	∅	Dark energy fraction at z = 0
	σ 8	0.8149	∅	Mass variance at R = 8h -1 Mpc at z = 0
	n spec	0.968	∅	scalar spectral index
	Y He	0.251	∅	Helium fraction
	τ	0.054	∅	Thompson optical depth
	E 10	5, 9 × 10 -6	eV	Energy gap between the two hyperfine ground states of HI
	α B	2.6 × 10 -13	cm 3 .s -1	Case B recombination coefficient value at T∼ 10 4 K
	δ c	1.686	∅	Critical density threshold at z = 0 before collapse

The emitted photon is not necessarily an ionizing photon as hydrogen can recombine to an excited state and then emit other photons through the de-excitation cascade process.

See Cen (1992) for analytical expressions of the various ionization rates

For more details about these effects look atFurlanetto et al. (2006);Pritchard & Loeb (2012);[START_REF] Furlanetto | The Cosmic 21-cm Revolution: Charting the first billion years of our Universe[END_REF]

He + rate coefficient for spin exchange is believed to be comparable to the rate coefficient of the free proton

Due to Earth rotation, τg also changes with time but this is slow compared to the oscillations of the observations at frequencies ν 10MHz that are orders of magnitude faster.

Though LOFAR is believed to be able to perform a low-resolution attempt(Zaroubi et al., 2012).

A process far from trivial experimentally because the sky brightness distribution I is sampled at a finite number of (u,v) points, because of the finite number of antennas. One has to therefore take into account this discrete number by looking at the interferometric array response when observing a point source, called the point spread function and then deconvoluting the raw distribution by it[START_REF] Pen | [END_REF]Harker et al., 2010;Beardsley et al., 2016;Patil et al., 2016).

The proportional constant only serving the purpose of transforming visibility units into power units(Parsons et al., 2012a)

τ is a delay almost equivalent to the geometrical delay defined in Section 2.1.2(Parsons et al., 2012b)

We hereby focus on instrument that have published results but other projects also exist like LEDA or 21CMA

http://eor.berkeley.edu/

http://reionization.org/

https://www.gmrt.org/

The announced results was first of (70 mK) 2(Paciga et al., 2013) but was then corrected after accounting for the signal loss during the foreground fitting. This event strongly emphasizes, if needed, the sheer difficulty of foreground subtraction and the huge impact of the slightest model imperfection during the procedure.

https://www.mwatelescope.org/

http://www.lofar.org

Most point sources below 200MHz are unpolarized, but if the sky model is expressed using a polarization basis and its calibration imperfect, some of the polarized emission into the total intensity can happen.

Even if a south pole catalog called GLEAM has been produced by MWA observations(Hurley-Walker et al., 2017) and its north pole counterpart LoTTs(Shimwell et al., 2017(Shimwell et al., , 2019) ) is currently being built using LOFAR.

This is valid as long as θ is greater than the diffraction limit of Equation2.1

y is the distance equivalent of the observed bandwith. In other words, here only a slice of width y of the signal is considered.

More details can be found inMorales (2005);McQuinn et al. (2006); Furlanetto et al. (2006)

SKA partners to this date are Australia, Canada, China, France, Germany, India, Italia, Japan, Netherlands, Portugal, South Africe, South Corea, Spain, Sweden, Switzerland and the United Kingdom. The african partners countries of this project are Botswana, Ghana, Kenya, Madagascar, Mauricius, Mozambique, Namibia and Zambie.

It is worthnoting that the virial temperature can be linked to and thus computed through the halo mass following an analytical formula(Barkana & Loeb, 2001;Mesinger et al., 2016).

Each sub-box can be computed using an arbitrary number of threads

Here the term particle is used in its numerical definition, not in its physical one. Typically with our usual resolution, a numerical particle has the weight of a small-size halo by itself. For example, our baryonic numerical particles are usually composed of ∼ 10 8-9 M and can have both gas and stellar content.

Note that empty cubes are not stored

It is worth noting that there is still a minimal smoothing length to avoid values below the gravitational softening in strongly overdense regions which would trigger unwanted diffusion effects.

Note that the kernel properties have been chosen so that the surface term is canceled

The number of particles that limit the tree construction is in fact tunable but has been set to 32 in the fiducial code.

Typically the simulation box size

i.e. mathematical model of learning and computer implementation. In a broader definition, research to create an entity which simulates human/animal behavior seems to be as old as history itself with report and myths of automatons capable of performing given tasks found in any ancient culture.

the backpropagation algorithm, still widely used today

Though for neural network specifically, an interesting listing attempt called "Neural Network Zoo" has been made : https://www.asimovinstitute.org/neural-network-zoo/

The bias parameter roughly follows the same equation than the weights for modification, we will thus implicitly include the bias when talking about weights in the following.

Like for the perceptron, the MLP neurons of the same layer are mutually independents.

The effective number of weights for a learning step is different from the total number of weights in the network when using dropout, see section 4.6.2.

Two perpendicular hyperplanes from two neighbouring points of the model-manifold will intersect on the concave side of a curved manifold. Conversely an observed noised signal (on the concave side) will belong to a single perpendicular hyperplane only if its distance to the model-manifold is smaller than the radius of curvature of the manifold at the intersection point (speaking in terms of the two-dimensional case).

https://keras.io

https://www.tensorflow.org/

Note that the ionized bubbles identification algorithm will be detailed in the following chapter 7, where understanding of the ionized bubbles distribution is the main focus.

Note that each field is a simple precision data cube of size 1024

, i.e. a

Go file. The HIRRAH-21 simulation being composed of 4096 domains, the fields cannot be all stored, hence the computation domain-wise allowing us the erase the fields of one domain when it has been considered.3 Such a case can happen as the halo finder associate a particle to a halo using a friend-of-friend approach whereas the halo radius is simply computed from its total mass.

In numerical simulation no cluster can obviously be infinite. However, when periodical boundaries are set for the simulation box the "infinitely" large cluster become easy to define. It is a cluster that stretches from one edge of the box to the other along at least one axis.

The exponent value for a random percolation process in three dimension is thus universal and is 0.41(Stauffer et al., 1994).

For computational purposes for the parameters inference, the original percolation routine was written in Fortran. However, we translate it in a Python class mainly because it will be simpler to re-use this way.

This section contains the abstract of the articleChen et al. (2020). See this article for more information.

Remerciements

We thank Anson D'Aloisio, Francois Lanusse, and Michelle Ntampaka for helpful discussions. AD acknowledges the McWilliams Center for Cosmology for hosting his internship. HT acknowledges support from NASA grant ATP-NNX14AB57G and STSCI grant HST-AR-15013.002-A. RC acknowledges support from NASA grant 80NSSC18K1101. Simulations were run at the NASA Advanced Supercomputing (NAS) Center.

Appendices

A

Studies in collaboration with Professor Hy Trac

During the free time of the first year of my PhD, I have continued a collaboration with Professor Hy Trac of the Carnegie Mellon University of Pittsburgh. This collaboration have started during a six months internship that I did for the second semester of my first year of master. After the internship, and for the two following years, we still worked remotely on the completion of two projects started there. These studies were not directly part of my PhD but were closely linked as it also concern the Epoch of Reionization and make use of numerical simulations. The two following sections display the abstracts of the two articles that summarize the results of our studies.

A.1 SCORCH. II. Radiation-Hydrodynamic simulations of reionization with varying radiation escape fractions

This section strictly and uniquely contains the article Doussot et al. (2018) whose authors are : Aristide Doussot, Hy Trac and Renyue Cen.

A.1.1 Abstract

In the Simulations and Constructions of the Reionization of Cosmic Hydrogen (SCORCH) project, we present new radiation-hydrodynamic simulations with updated high-redshift galaxy populations and varying radiation escape fractions. The simulations are designed to have fixed Thomson optical depth τ ≈ 0.06, consistent with recent Planck observations, and similar midpoints of reionization 7.5 z 8.0, but with different ionization histories. The galaxy luminosity functions and ionizing photon production rates in our model are in good agreement with recent HST observations. Adopting a power-law form for the radiation escape fraction f esc (z) = f 8 [(1 + z)/9] a 8 , we simulate the cases for a 8 = 0, 1, and 2 and find a 8 2 in order to end reionization in the range 5.5 z 6.5 to be consistent with Lyman alpha forest observations. At fixed τ and as the power-law slope a 8 increases, the reionization process starts earlier but ends later with a longer duration ∆z and the decreased redshift asymmetry Az. We find a range of durations 3.9 ∆z 4.6 that is currently in tension with the upper limit ∆z < 2.8 inferred from a recent joint analysis of Planck and South Pole Telescope observations.

A.1.2 Introduction

Cosmic reionization is a major frontier topic in modern cosmology with intense ongoing theoretical and observational work. The Epoch of Reionization (EoR) starts with the formation of the first luminous sources in the first few hundred million years and ends with the reionization of hydrogen when the Universe is about a billion years old. In the process ionizing radiation from high-redshift stars, galaxies, and quasars convert the cold and neutral gas into a warm and highly ionized medium. Many fundamental questions regarding the radiation sources, reionization process, and the timing of the EoR remain. See [START_REF] Mesinger | Understanding the Epoch of Cosmic Reionization[END_REF] for an excellent review.

Recent observations from the Hubble Space Telescope (HST) and Planck have significantly improved observational constraints on the EoR. One constraint is provided by the integral optical depth, measured to be τ = 0.058 ± 0.012 (Adam et al., 2016). Analysis suggests that, using this measurement values, the duration of the reionization is ∆z CMB < 2.8 (Adam et al., 2016), assuming that the reionization is completed at redshift z ≈ 6. Many studies (e.g. Bouwens et al., 2015a;Finkelstein et al., 2015;Livermore et al., 2017) show that it is likely that the main contributors to the reionization are the dwarf galaxies.

The project SCORCH (Simulations and Constructions of the Reionization of Cosmic Hydrogen) is motivated and designed to gain a deeper understanding of the EoR by providing theoretical tools to improve the comparison between observations and theory. To make significant progress, a systematic framework to investigate the effects of distribution and properties of radiation sources and sinks on the reionization process is needed.

Among the parameters driving the reionization, the galaxy luminosity function (GLF) is one of the less constrained at high redshift. In SCORCH I (Trac et al., 2015), we propose a way to extrapolate the known luminosity function to higher redshift and fainter magnitude. The obtained galaxy luminosity function is consistent with the measurements from Bouwens et al. (2015a) and Finkelstein et al. (2015) at redshift 6 z 10 and also in good agreement with cosmological simulations (e.g. Gnedin, 2016;Feng et al., 2016;Liu et al., 2016;[START_REF] Ocvirk | [END_REF] and semi-analytical models (e.g. Mashian et al., 2016;Mason et al., 2015).

The escape fraction of ionizing photons f esc , the fraction of ionizing photons which escapes from the galaxies where they have been created into the intergalactic medium, is a very important parameter but nearly impossible to observe directly at EoR. Overall, f esc is governed by all the internal processes of emission and absorption of ionizing photons in galaxies and relies on high resolution hydrodynamic simulations to place our understanding on a solid physical basis. While theoretical progress on this front is onging, the computed f esc can be sensitively dependent on simulation methods, resolution and treatments of physical processes (such as A.1. SCORCH. II. RADIATION-HYDRODYNAMIC SIMULATIONS OF REIONIZATION WITH VARYING supernova feedback), comparison among different works can be difficult currently. However, empirically, there are some constraints by assuming a given GLF with a low-luminosity limiting magnitude M SF ∼ -10.0 and a galaxy-driven reionization model, f esc 0.10 -0.20 at z 6 (Bolton & Haehnelt, 2007). Curiously, at low-redshift the values obtained from observational measurements are f esc 0.05 -0.1 (Chen et al., 2007;Iwata et al., 2009;Smith et al., 2018) which suggests or requires that f esc varies with redshift [START_REF] Alvarez | [END_REF]Sun & Furlanetto, 2016;Price et al., 2016). In Price et al. (2016) both parametric and non-parametric functional forms have been tested to obtain an expression of f esc as a function of redshift that is conformal with this expected trend. It is found that f esc can be well-fitted by a simple power-law form whose exponent depends on the value of the optical depth τ that is taken as a reference.

Here, in Paper II of the SCORCH project, we produce and analyze new reionization simulations, combining previous work on the abundance of high-redshift galaxies (Trac et al., 2015) and the evolution of the radiation escape fraction (Price et al., 2016). Section A.1.3 describes the methods, including the radiation-hydrodynamic simulations, galaxy population models, and radiation escape fraction models. Section A.1.4 presents initial results on the photoionization and photoheating. More detailed results will be presented in upcoming papers. Section A.1.5 summarizes our work and the Appendix includes additional tests and results. We adopt the concordance cosmological parameters: Ω m = 0.30, Ω Λ = 0.70, Ω b = 0.045, h = 0.7, n s = 0.96, and σ 8 = 0.8

A.1.3 Methods

Radiation-Hydrodynamic Simulations

We run three new radiation-hydrodynamic simulations that are consistent with the latest observations. The simulations are designed to have fixed Thomson optical depth τ ≈ 0.06, consistent with recent Planck observations (Ade et al., 2016;Adam et al., 2016;Aghanim et al., 2016). They start with the same initial conditions, but have different reionization histories. They have the same modeled galaxy populations, but use different radiation escape fraction models. In the three simulations, f esc (z) is either constant or varies linearly or quadratically with respect to 1 + z.

To run our simulations, we use the Radhydro code which has already been used to model both hydrogen and helium reionization (Trac et al., 2008;Battaglia et al., 2013a;La Plante et al., 2017). In order to simultaneously solve collisional gas dynamics, collisionless dark mark dynamics, and radiative transfer of ionizing photos, the Radhydro code combines hydrodynamic and N-body algorithms (Trac & Pen, 2004) with an adaptive ray-tracing algorithm (Trac & Cen, 2007). As the ray-tracing algorithm has adaptive splitting and merging, it improves the resolution and the scaling.

The three Radhydro simulations, all starting with the same initial conditions at redshift z = 300 and having 2048 3 dark matter particles, 2048 3 gas cells, and up to 12 billion adaptive rays. We use a fixed grid and a comoving box of side length 50 h -1 Mpc, focusing on atomic cooling halos. Consequently, we have a resolution of 24.4 h -1 kpc. For each ray we track five frequencies (15.7, 21.0, 29.6, 42.9, 74.1 eV) above the hydrogen ionizing threshold of 13.6 eV. The two first frequencies are chosen to be below the first helium ionizing threshold, the two following frequencies are below the second helium ionizing threshold and the last frequencies is above all threshold. The nonequilibrium solvers for the ionization and energy equations use the photoionization and photoheating rates computed from the incident radiation flux. The three simulations are run down to redshift z = 5.5.

The generation of the halo and galaxy catalogs is done by a particle-particle-particle-mesh (P 3 M; Trac et al., 2015) N-body simulation with 3072 3 dark matter particles using a highresolution version of the same initial conditions as the Radhydro simulations. Every 20 million cosmic years, a hybrid halo finder is run on the fly to locate dark matter halos and build merger trees. The particle mass resolution of 3.59 × 10 5 h -1 M allows the measurement of halo quantities such as mass and accretion rate down to the atomic cooling limit (T ∼ 10 4 K, M ∼ 10 8 h -1 M ). The halo mass accretion rate is calculated as

where M 2 is the mass of a given progenitor at a given time t 2 and M 1 is the mass of its descendant at a previous time t 1 . The radiation sources are modeled using an updated subgrid approach allowing us to populate dark matter halos with galaxies, by matching the observed galaxy luminosity functions, while being able to compute accurately the spatial distribution of ionizing sources. Starting from the halo mass accretion rate, we infer the luminosity-accretion rate relation L UV ( Ṁ , z) from the abundance matching performed by equating the number density of galaxies to the number density of halos:

Using the halo mass accretion rate instead of the halo mass M allows us to account for the scatter in mass-to-light ratio and the episodic nature of star formation. More details of the abundance matching technique can be found in SCORCH I (Trac et al., 2015) and a review on reionization simulations has been done in Trac & Gnedin (2011).

Galaxy Luminosity Functions

The reionization history depends strongly on the abundance of escaped ionizing photons and its evolution. The current observable galaxies with M UV -17 and at z 10 are only a part of the ionizing sources that are responsible for the ionization history. To compute the GLF, it is therefore required to extrapolate the known luminosity function to fainter magnitude and higher redshift. We then use the fiducial model that has been created and detailed in SCORCH I (Trac et al., 2015).

Appendix B : Simulation Results

We summarize in Table A.3 the values of the main quantities obtained from SCORCH RadHydro simulation for a representative redshift set.