Harold Erbin
email: erbin@mit.edu

Riccardo Finotello
email: riccardo.finotello@cea.fr

Robin Schneider
email: robin.schneider@physics.uu.se

Mohamed Tamaazousti
email: mohamed.tamaazousti@cea.fr

Deep multi-task mining

published or not. The documents may come

Introduction

There is a growing body of research that applies modern techniques from data science to problems in string theory [START_REF] Ruehle | Data science applications to string theory[END_REF]. The reasons for that are twofold. On the one hand, standard computations in string theory are hard, in particular they can be NP-hard or even undecidable [START_REF] Ruehle | Data science applications to string theory[END_REF][START_REF] Denef | Computational Complexity of the Landscape I[END_REF][START_REF] Halverson | Computational Complexity of Vacua and Near-Vacua in Field and String Theory[END_REF]. Due to double exponential scaling laws in terms of computational resources with respect to the input parameters, string theory calculations often fail to finish in a reasonable amount of time even on modern machines. On the other hand, there are too many configurations to consider. The largest estimates put a bound of O(10 272 000 ) when considering F-theory compactified on a Calabi-Yau four-fold [START_REF] Taylor | The F-theory geometry with most flux vacua[END_REF]. Parsing that many configurations is impossible, thus computational smart ways are needed to select potentially interesting vacuum configurations [START_REF] Halverson | Branes with Brains: Exploring String Vacua with Deep Reinforcement Learning[END_REF][START_REF] Larfors | Explore and Exploit with Heterotic Line Bundle Models[END_REF].

An important key component in realistic string theory compactifications are Calabi-Yau manifolds. These manifolds have been studied extensively in the past, and thus they comprise some of the best datasets within the string theory community [START_REF] He | Calabi-Yau Spaces in the String Landscape[END_REF]:

1. The first widely used dataset are the 7890 complete intersection Calabi-Yau, in short CICY, manifolds in three complex dimensions by Candelas et al. [START_REF] Candelas | Complete Intersection Calabi-Yau Manifolds[END_REF][START_REF] Green | All Hodge Numbers of All Complete Intersection Calabi-Yau Manifolds[END_REF][START_REF] Anderson | Fibrations in CICY Threefolds[END_REF].

2. The largest dataset, the Kreuzer-Skarke list, contains 473 million reflexive polytopes in four dimensions. These encode a toric ambient space, from which one obtains Calabi-Yau three-folds by considering the hypersurface defined by the canonical bundle [START_REF] Kreuzer | Complete classification of reflexive polyhedra in four-dimensions[END_REF].

3. CICY four-folds have also been classified and amount to 921 497 distinct configuration matrices [START_REF] Gray | All Complete Intersection Calabi-Yau Four-Folds[END_REF][START_REF] Gray | Topological Invariants and Fibration Structure of Complete Intersection Calabi-Yau Four-Folds[END_REF].

The incredible progress in data science, in particular image recognition, over the past decade can in part be attributed to large and clean datasets [START_REF] Russakovsky | ImageNet Large Scale Visual Recognition Challenge[END_REF]. They allowed researchers to benchmark their algorithms and let the best ones compete against each other, which in turn resulted in rapid development and ever improving neural network architectures [START_REF] Krizhevsky | ImageNet Classification with Deep Convolutional Neural Networks[END_REF][START_REF] Szegedy | Going Deeper with Convolutions[END_REF][START_REF] Szegedy | Rethinking the Inception Architecture for Computer Vision[END_REF][START_REF] Szegedy | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[END_REF][START_REF] He | Deep residual learning for image recognition[END_REF]. We will proceed in a similar vein in this paper. The number of independent Kähler moduli of CICY three-folds has been successfully analyzed using neural networks in the past. These benchmarks were initiated by He, who proposed to treat their configuration matrices as a simple two-dimensional image [START_REF] He | Deep-Learning the Landscape[END_REF].

In previous works [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF][START_REF] Erbin | Machine learning for complete intersection Calabi-Yau manifolds: a methodological study[END_REF], two of the authors have shown that learning h (1,1) is possible to great accuracy, but the limited training data is not sufficient to generalize the learning to the number of complex structure moduli h (2,1) . Computing Hodge numbers of Calabi-Yau manifolds is of great importance, since cohomology computations are an integral part of string theory compactifications. They for example determine the number of massless fermion generations in string theory compactifications. Thus, the goal is to identify performant algorithms in these wellstudied datasets of tangent bundle cohomologies, which can then generalize to more complicated vector bundles.

In the rest of this paper, we will present two different approaches for learning Hodge numbers of CICY four-folds. First, we will treat the problem as a standard image classification task where the Hodge numbers are the image labels. For this purpose, we employ an Inception module based architecture [16-18, 21, 22] and show that a single set of hyperparameters generalizes well to all four Hodge numbers, yielding a mean accuracy over all Hodge numbers of 85 %. This suggests that we could scale the approach to a multi-task learning problem.

Subsequently, we show that all Hodge numbers can be learned simultaneously by utilizing a branched network with hard parameter sharing [START_REF] Caruana | Multitask Learning: A Knowledge-Based Source of Inductive Bias[END_REF][START_REF] Standley | Which Tasks Should Be Learned Together in Multi-task Learning[END_REF] between the task specific substructures, which ultimately are responsible for learning the distributions of the Hodge numbers. The multi-task approach has several advantages, with respect to single-task architectures. From a technical side, multi-task learning has been shown to improve the overall performance of the models [START_REF] Caruana | Multitask Learning: A Knowledge-Based Source of Inductive Bias[END_REF]. From a physics and algebraic geometry perspective, a single model hints towards the definition of a unified framework from which it may be possible to extract meaningful theoretical information, such as closed form formulas. The model we developed is capable of learning at the same time, and without rescaling, the four dimensions of the tangent space cohomologies of CICYs, accounting for the heavy class imbalance present in the dataset. This multi-task Ansatz leads to perfect performance on two of the four Hodge numbers and accuracy of 96 % and 83 % for h (3,1) and h (2,2) respectively, with a training ratio of 80 %.

The outline of this paper is as follows. In Section 2, we discuss related works of learning cohomologies and earlier results on Calabi-Yau threefolds. Section 3 explores the dataset of CICY four-folds and presents the results of our classification experiments. This is followed by our main results in Section 4 in which we introduce our deep learning model CICYMiner, a multitask regression model based on chained Inception modules that predicts all four Hodge numbers at once. We conclude in Section 5 with some outlooks. Python codes for this paper can be found at:

• https://github.com/robin-schneider/ cicy-fourfolds

• https://github.com/thesfinox/ ml-cicy-4folds

The list of packages used throughout the development comprises pandas [START_REF]pandas development team. pandasdev/pandas: Pandas[END_REF][START_REF] Mckinney | Data Structures for Statistical Computing in Python[END_REF] and numpy [START_REF] Harris | Array programming with NumPy[END_REF] for data operations, matplotlib [START_REF] Hunter | Matplotlib: A 2D graphics environment[END_REF] and seaborn [START_REF] Waskom | seaborn: statistical data visualization[END_REF] for visualisation, and tensorflow [START_REF] Abadi | TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems[END_REF] for the deep learning algorithms.

Related works

The first paper utilizing machine learning algorithm to predict various different cohomology dimensions was written by He [START_REF] He | Deep-Learning the Landscape[END_REF]. The author tackled the problem of predicting Hodge numbers of CICY three-and four-folds, but also line bundles over these manifolds [START_REF] He | Deep-Learning the Landscape[END_REF]. These studies have later been extended to systematically investigate CICY three-folds with linear regression, support vector machines, and dense neural networks achieving accuracies ranging from 37 % to 85 % [START_REF] Bull | Machine Learning CICY Threefolds[END_REF][START_REF] Bull | Getting CICY High[END_REF] when using 70 % training data. The benchmarks have subsequently been improved by using an Inception-based architecture to accurately predict 97 % of the test data using only 30 % training data, essentially solving the problem of predicting h (1,1) [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF][START_REF] Erbin | Machine learning for complete intersection Calabi-Yau manifolds: a methodological study[END_REF]. This work was supplemented by more methodological studies in which the dataset was augmented with various other (topological) quantities. Other works on CICY three-folds include [START_REF] Krippendorf | Detecting symmetries with neural networks[END_REF][START_REF] He | Distinguishing Elliptic Fibrations with AI[END_REF]. An initial exploration of CICY four-folds has been started by He and Lukas [START_REF] He | Deep-Learning the Landscape[END_REF][START_REF] He | Machine Learning Calabi-Yau Four-folds[END_REF]. The authors used a simple dense neural network and were able to predict h (1,1) with an accuracy of 96 %. This promising early result showed that the increased size of the dataset improves the performance significantly. However, in line with previous studies of h (2,1) on CICY three-folds, the authors were unable to accurately predict the value of the other Hodge numbers, reaching an accuracy of only 27 % for h (3,1) . They were successful in improving this accuracy for a subset of the dataset by considering all configuration matrices of shape (4, 4) and using feature enhancement. This feat was achieved by supplementing the training samples with all up to degree four monomials of the defining polynomials and pushed the accuracy to 95 %.

The Kreuzer-Skarke list has also been the target of deep learning algorithms. In order to identify equivalent Calabi-Yau manifolds coming from different triangulations, Demirtas et al. trained residual neural networks to learn the triple intersection numbers [START_REF] Demirtas | Bounding the Kreuzer-Skarke Landscape[END_REF]. They reached an almost perfect performance, which allowed them to cut down the computation time from seconds to microseconds. This in turn made it possible to derive an upper bound on the number of distinct Calabi-Yau manifolds arising from the polytope with the most triangulations, setting it to 10 428 .

There are several ongoing projects in learning Hodge numbers of line bundle cohomologies. These can be separated into two different approaches. First, learning the cohomology dimensions directly, for example on del Pezzo surfaces [START_REF] Bies | Machine Learning and Algebraic Approaches towards Complete Matter Spectra in 4d F-theory[END_REF] and on CICY three-folds [START_REF] Larfors | Explore and Exploit with Heterotic Line Bundle Models[END_REF][START_REF] He | Deep-Learning the Landscape[END_REF][START_REF] Ruehle | Evolving neural networks with genetic algorithms to study the String Landscape[END_REF][START_REF] Larfors | Line bundle cohomologies on CICYs with Picard number two[END_REF]. Second, neural networks have been used to classify cones in the cohomology-dimension landscape [START_REF] Klaewer | Machine Learning Line Bundle Cohomologies of Hypersurfaces in Toric Varieties[END_REF][START_REF] Brodie | Machine Learning Line Bundle Cohomology[END_REF][START_REF] Brodie | Index Formulae for Line Bundle Cohomology on Complex Surfaces[END_REF]. The Hodge numbers belonging to these cones can all be described by the same analytic equations [START_REF] Constantin | Formulae for Line Bundle Cohomology on Calabi-Yau Threefolds[END_REF].

Exploring the dataset

In this section, we will introduce and explore complete intersection Calabi-Yau four-folds. We then proceed to learn the four non-trivial Hodge numbers independently using neural networks with an Inception inspired architecture [START_REF] Szegedy | Going Deeper with Convolutions[END_REF][START_REF] Szegedy | Rethinking the Inception Architecture for Computer Vision[END_REF][START_REF] Szegedy | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[END_REF].

CICY four-folds

A complete intersection Calabi-Yau manifold is fully defined by its configuration matrix. This matrix encodes the polynomial degrees and ambient space factors in the following way:

M =    n 0 p 0 1 • • • p 0 K . . . . . . . . . . . . n r p r 1 • • • p r K    χ . ( 3.1)
Each p i j ∈ N is the degree of the j-th polynomial in the homogeneous coordinates of the i-th complex projective space with dimension n i . The Calabi-Yau condition is translated in the configuration matrix by requiring that

n i + 1 = K j=1 p i j . (3.2)
The Euler number χ is given in the subscript and can be directly computed by integrating the fourth Chern class or from the four non-trivial Hodge numbers as χ = 4 + 2h (1,1) -4h (2,1) + 2h (3,1) + h (2,2) . (3.3) In the first row we have on the left the distribution of h (1,1) , to the right of h (2,1) . In the bottom row h (3,1) is presented to the left and h (2,2) to the right.

A second linear relationship between the Hodge numbers can be derived by combining the indices χ q = χ(M, ∧ q T M * ) [START_REF] Gray | Topological Invariants and Fibration Structure of Complete Intersection Calabi-Yau Four-Folds[END_REF] leading to 44 = -4h (1,1) + 2h (2,1) -4h (3,1) + h (2,2) . (3.4)

The configuration matrices have been generated from an initial set of matrices and subsequently applying the splitting procedure [START_REF] Candelas | Complete Intersection Calabi-Yau Manifolds[END_REF][START_REF] Gray | All Complete Intersection Calabi-Yau Four-Folds[END_REF], finding new manifolds and discarding equivalent descriptions. In this way, a total of 921 497 topological distinct types of CICY manifolds were found, with 905 684 of them not being direct products of lower dimensional manifolds.

The Hodge number distributions are presented in Figure 1. The mean, maximum and minimum values are h (1,1) = 10.1 24 1 , h (2,1) = 0.817 33 0 , h (3,1) = 39.6 426 20 , h (2,2) = 241 1752 204 . (3.5)

Notice that the distributions of the Hodge numbers are, in general, imbalanced: for instance, h (2,1) vanishes for 70 % of the configuration matrices in the dataset. We find that 54.5 % are favourable (i.e. h (1,1) is equal to the number of projective spaces), less than the 61.9 % for CICY three-folds. 1 Hence, for slightly more than half of the cases we have h (1,1) = r, the number of projective ambient space factors. This number is important as it should be the baseline to compare any algorithm against.

Classifying Hodge numbers

Problems in image recognition are usually formulated as classification tasks. Take the ImageNet dataset which consists of 14 × 10 6 data points with over 21 000 classes. That is about one order of magnitude larger, both in samples and classes, than predicting h (2,2) . In this section, we will train one neural network to classify each of the four non-trivial Hodge numbers independently. We will use an architecture based on Inception modules [START_REF] Szegedy | Going Deeper with Convolutions[END_REF][START_REF] Szegedy | Rethinking the Inception Architecture for Computer Vision[END_REF][START_REF] Szegedy | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[END_REF] as was done for the best performing predictors of the CICY three-fold Hodge numbers [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF][START_REF] Erbin | Machine learning for complete intersection Calabi-Yau manifolds: a methodological study[END_REF]. This specific architecture has been shown to lead to the best performance on the input feature

H W F kernel W x 1 x F kernel 1 x H x F W W H H F' F' C H W 2 F' H W 2 F' BN output feature

Figure 2: An Inception module can be decomposed into the different convolutional kernels scanning over the width (W) and height (H) with filters (F). They are subsequently concatenated (C) and followed by a batch normalization (BN) layer. The Inception module is the main building block of both the CICYMiner and the classification architectures.

configuration matrices when using 1d kernels of maximal size. This partially reflects the fact that scanning coordinates in each projective space and a single variable over all projective spaces helps in better learning the connections between the different hypersurfaces of the CICYs (see Figure 2). The choice of maximal 1d kernel is, in fact, motivated by the mathematical machinery required to compute Hodge numbers. There, one has to compute the dimension of ambient space cohomology group representations, which are stacked for each projective space. These ambient space representations arise after splitting up the Koszul resolution

0 → ∧ K N * → ... → N * → O A → O A | M → 0 . (3.6)
which contains the antisymmetric products ∧ s N * of the defining hypersurfaces (N denotes the normal bundle, which contains the information about the polynomial degrees p i j ). Moreover, using an Inception based architecture lead to a performance increase of misclassification rate on the Im-ageNet dataset from 15.3 % for AlexNet [START_REF] Krizhevsky | ImageNet Classification with Deep Convolutional Neural Networks[END_REF] using a standard convolutional architecture to 6.7 % for the first version of GoogLeNet [START_REF] Szegedy | Going Deeper with Convolutions[END_REF].

We proceed as in earlier studies [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF][START_REF] Erbin | Machine learning for complete intersection Calabi-Yau manifolds: a methodological study[END_REF] by considering different train:val:test splits with respectively 10 %, 30 %, 50 % and 80 % training and 10 % validation data. The architecture hyperparameters have been optimized using Bayesian Optimization Hyperband [START_REF] Falkner | BOHB: Robust and Efficient Hyperparameter Optimization at Scale[END_REF][START_REF] Li | Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization[END_REF] on the problem of predicting h (3,1) . The same hyperparameters have then been used to also classify the other three Hodge numbers.

We opted to present the results of neural networks with a comparable number parameters 840 000 ± 10 000 to the number of configuration matrices. This architecture comprises four Inception modules, with respectively 3 × 64 and 16 filters, utilizing batch normalization for better gradient propagation into the earlier layers [START_REF] Szegedy | Going Deeper with Convolutions[END_REF][START_REF] Szegedy | Rethinking the Inception Architecture for Computer Vision[END_REF][START_REF] Szegedy | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[END_REF][START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF]. Figure 2 decomposes an Inception module into its different ingredients. The convolutional kernels scan over the configuration matrix dimensions, i.e. the maximal number of possible projective ambient spaces [START_REF] Szegedy | Going Deeper with Convolutions[END_REF] and the maximal number of polynomial constraints [START_REF] He | Deep-Learning the Landscape[END_REF]. The Inception modules are followed by three dense layers with 16 units, ReLU activation function and dropout layers with a 0.2 rate to contrast overfitting. Furthermore, we employ 1 (10 -5 ) and 2 (10 -6 ) regularization for all weights in the network. The last layer contains a softmax activation function with {h (i,j) min , . . . , h (i,j) max } classes. The network is trained with Adam optimizer and an initial learning rate of 4 × 10 -4 on a 32 minibatch size. This architecture is still trainable in a reasonable amount of time on a desktop computer with access to a GPU. In comparison to earlier studies [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF][START_REF] Erbin | Machine learning for complete intersection Calabi-Yau manifolds: a methodological study[END_REF], we found that leaving the outliers inside the training data does not negatively impact the results.

Figure 3 shows in the top row the training loss and accuracy, and in the bottom row validation accuracy tracked over the training process and test accuracy for the best-performing model. The best-performing model is the one with the highest validation accuracy, which one would get when employing early stopping on that metric. It is important to track the best performing models as sometimes the loss starts increasing again as visible from the h (2,1) curve. The error bars are computed from the different training ratios and the budget on the x-axis is given by budget = number of epochs

h (2, 1) h (3, 1) h (2,
h (2, 1) h (3, 1) h (2, 2)
× percentage of training data 80 . (3.7)
We observe that h (1,1) is predicted with almost perfect accuracy for any training ratio, while the accuracies of the other three Hodge numbers improve with more training data. However, when the training data contains more than 30 % of the samples one has diminishing returns for the accuracy. This is in line with previous observations for the CICY three-folds [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF][START_REF] Erbin | Machine learning for complete intersection Calabi-Yau manifolds: a methodological study[END_REF]. Even though the hyperparameters have been optimized to learn h (3,1) , it is the worst performing value. This is interesting as Figure 1 shows that the distribu- tion of h (2,2) spans a longer range, contains more outliers and has a thicker tail. The plots show that we avoid overfitting to the training data. Table 1 collects the accuracy at different training ratios and the mean-value for the four different training ratios of the best performing model. 2In the training process, we employed learning rate decay with a factor of 0.4, when the validation ac- curacy did not improve for epochs equivalent to 0.15 × budget. This is clearly visible from the loss and accuracy plots in the top row and accounts for the down-and up-stairs steps. Summarizing the results, we find that the hyperparameters found for predicting the worst performing Hodge number h (3,1) also generalize well to the other three Hodge numbers. This is a first indication that the prediction of Hodge numbers could benefit from multi-task learning.

h (1,1) h (2,1) h (3,1) h (2,2)

CICYMiner

In the previous section, we showed that a classification task based on Inception modules is effective in learning the Hodge numbers. As the optimization was conducted for h (3,1) , rather than an ad hoc structure for each output, the good results motivate further study on learning several Hodge numbers at the same time. In this section, we focus on a regression model for two main reasons. First, in general computations of vector bundle cohomologies, the predictions may not be bounded, thus an inference model has to be able to adapt by learning an approximation function, rather than classification probabilities. Second, previous studies showed that regression models on a similar task were more efficient than classification [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF].

Figure 4 shows the schematic of the architecture used in this section. The architecture enables multi-task learning by hard parameter sharing over an initial structure capable of learning a shared representation of the input. This, in general, has proven efficient at increasing the learning power of a single network, rather than differentiating and optimizing several, and to reduce the risk of overfitting [START_REF] Caruana | Multitask Learning: A Knowledge-Based Source of Inductive Bias[END_REF][START_REF] Baxter | A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling[END_REF]. The median layers of the network replicate a similar multitasked structure on the same learning objective: in fact, one branch of the sub-structures learning the Hodge numbers is an auxiliary architecture used to reinforce the stability of the representation. No additional regularization was added to the model, apart from a 0.2 dropout rate before the fully connected networks in the auxiliary branches. Such an architecture is thus capable of "mining" richer and more diverse features from a shared representation of the input by using different layer combinations. The model is partly inspired by a recently proposed DeepMiner [START_REF] Benzine | Deep Miner: A Deep and Multi-branch Network which Mines Rich and Diverse Features for Person Re-identification[END_REF] model, used for people re-identification tasks, capable of learning more information by using different branched structures and layers. As such, we refer to our model as CICYMiner: we leverage the DeepMiner architecture with the advantages of multi-task learning in order to learn a family of related tasks, which however present complicated and strongly diverse distribution functions (see Figure 1). The role of the auxiliary branches in CICYMiner (see Figure 4) is mainly related to feature mining, that is the ability to extract as much information as possible from intermediate representations, in order to guide the learning of the weights during learning. The auxiliary branches have, in fact, slightly different architectures with respect to the main branches, in order to perform different transformation on the inputs. An added value of the auxiliary branches is the duplication of the outputs, which in this multitask context can improve overall performance, with regard to outlier and overfit control.

Preprocessing and Evaluation Strategy

We use the same dataset presented for the classification objective in the previous section. Given the strong class imbalance, we select the training set by using a stratified approach on h (2,1) in order to preserve the distribution of the samples. The validation set is then chosen totally at random, using 10 % of the samples. The remaining samples form the test set. We preprocess the input data by simply rescaling the entries of the configuration matrices in the training set to the interval [0, 1]. Matrices in the validation and test sets are rescaled accordingly, using the statistics obtained from the training set.

The outputs of CICYMiner are, in fact, floating point numbers h(i, j) ∈ R + , as it is typical in regression tasks. They ultimately need to be rounded to integers to be directly compared with the true values and to compute the accuracy. The distributions of the Hodge numbers have not been rescaled as training led to lower accuracy when this strategy was adopted. The specialised branches of the network are, in fact, deep enough to apply the proper scaling starting from a shared representation and correctly learn the output distribution of the Hodge numbers.

In order to test the robustness and versatility of the network, we choose to keep the outliers in the training set. In multi-task learning architectures, they may strongly affect the behavior of the network and may need robust loss functions during training [START_REF] Zhang | Robust Multi-Task Learning With Flexible Manifold Constraint[END_REF]: this problem is directly addressed in what follows. On the other hand, what repre-sents an outlier for a certain task, can be valuable information for another [START_REF] Zhang | A Survey on Multi-Task Learning[END_REF], hence the choice of keeping the outliers in the training set. Empirically, we also experienced a decrease in accuracy when trying to find a good outlier exclusion strategy.

Training

In this case, training occurred over a fixed amount of 300 epochs, due to time restrictions on the cluster computing infrastructure. Training takes approximately 5 days on a single NVIDIA V100 GPU. We use the Adam [START_REF] Kingma | A method for stochastic optimization[END_REF] stochastic gradient descent with an initial learning rate of 10 -3 and a mini-batch size of 64 configuration matrices. Due to the long training time, the optimization was done using a grid search over a reasonable amount of choices of hyperparameters. The network is ultimately made of 10 7 trainable parameters, accounting for both the shared representation and the eight sub-networks learning Hodge numbers and their auxiliary outputs. In terms of typical computer vision multi-task learning, we still deal with a small network: for instance, the original Inception network by Google has 0.7 × 10 7 parameters for a single classification task [START_REF] Szegedy | Going Deeper with Convolutions[END_REF][START_REF] Szegedy | Rethinking the Inception Architecture for Computer Vision[END_REF][START_REF] Szegedy | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[END_REF].

We already motivated the choice of keeping the outliers in the training set. We address the arising issues by employing a Huber loss function [START_REF] Huber | Robust Estimation of a Location Parameter[END_REF]:

H {k} δ (x) =            1 2 k n=1 N k i=1 ω n x (i) 2 , x (i) ≤ δ δ k n=1 N k i=1 ω n x (i) - δ 2 , x (i) > δ (4.1)
where ω n for n = 1, 2, . . . , k are the loss weights of the different branches of the CICYMiner, δ is a hyperparameter of the model and x (i) is the residual error of the i-th sample. The choice of the loss turns out to be extremely useful in this regression task, as it behaves as a 2 loss for small residuals, and it is linear for larger errors. Robustness is thus implemented as a continuous interpolation between the quadratic and linear behaviour of the loss function. This is a solution usually adopted for classification [START_REF] Zhang | A Survey on Multi-Task Learning[END_REF] where combinations of 1 , 2 and Frobenius norm are used for robustness.

In our best implementation, we used δ = 1.5, and loss weights 0.05, 0.3, 0.25 and 0.35 for h (1,1) , h (2,1) , h (3,1) and h (2,2) , respectively (the auxiliary branches use the same values as the principal ones). The learning rate is set to reduce by a factor of 0.3 after 75 epochs without improvements in the total loss of the validation set (as a reference, at 80 % training ratio, this hard reduction mechanism triggered only once between epochs 270 and 300).

Results

The final results are presented in Figure 5 and in the last row of Table 2. As shown in the learning curve, h (1,1) reaches perfect accuracy with just 10 % of the training data, in alignment with previous attempts [START_REF] He | Machine Learning Calabi-Yau Four-folds[END_REF] and the classification results of the previous section. h (2,2) is in general the most difficult label to train and it is strongly dependent on the training ratio. The network appears to be underfitting the distributions of the Hodge numbers, and validation loss is still decaying after 300 epochs: it would be interesting to run training for longer time, in order to study the behaviour of the network. At a training ratio of 30 % the network reaches perfect accuracy on h (1,1) , while h (2,1) gets to 97 %. h (3,1) remains at 81 %, while h (2,2) reaches barely 49 %. Increasing the number of training samples is, in general, beneficial for all Hodge numbers: h (1,1) and h (2,1) reach 100 %, while the accuracy of h (3,1) and h (2,2) rises to 96 % and 83 %, respectively, when the training ratio reaches 80 %. For the first three outputs in Table 2, the regression metrics, Mean Squared Error (MSE) and Mean Absolute Error (MAE), show the ability to effectively learn the discreteness of the Hodge numbers: both metrics show, in fact, values which can be confidently rounded to well defined integer results (i.e. MAE 0.50 and MSE 0.25). The good performance of the first three Hodge numbers suggests the possibility to use relations such as the Euler characteristic (3.3), which can be computed from combinatorics, and the linear constraint (3.4). Using the latter to compute h (2,2) leads to an accuracy of 96 % on the test set, using the best results at 80 % training ratio. Using (3.4) and (3.3) together, h (3,1) and h (2,2) can reach perfect accuracy at 80 % training ratio. Using CICYMiner it is therefore possible to compute all four Hodge numbers with 100 % of accuracy.

Ablation study

CICYMiner introduces new elements, with respect to previous attempts at predicting Hodge numbers of CICYs [START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF][START_REF] He | Machine Learning Calabi-Yau Four-folds[END_REF], namely:

1. Huber loss for robustness; 2. auxiliary branches.

In this section, we separately analyse each new aspect, together with other variations of the architecture. Specifically, we analyse the impact of the batch normalization used in the Inception modules. We also address the use of attention mechanisms [START_REF] Bahdanau | Neural Machine Translation by Jointly Learning to Align and Translate[END_REF], used in the DeepMiner model, which in our case did not lead to an improvement in accuracy, but rather to a faster training. We proceed by modifying the backbone structure of CICYMiner. We first introduce the attention mechanism used in [START_REF] Benzine | Deep Miner: A Deep and Multi-branch Network which Mines Rich and Diverse Features for Person Re-identification[END_REF] for comparison. The Spatial Attention Module (SAM) and CHannel Attention Module (CHAM) are presented in Figure 6: the full attention mechanism is the composition CHAM • SAM used between each Inception module in the main branch of the task-specific architecture in Figure 4. We also analyse the performance of the model by simply removing the auxiliary branches in the top layers of the network. Then, as opposed to the Huber loss, we test the predictions using the usual MSE used in most regression tasks. We finally change the size and type of the normalization strategy used in the architecture: we first train a network with a mini-batch size of 256 samples, and we then compare the results with a Layer Normalization [START_REF] Ba | Layer Normalization[END_REF] strategy. Results are summarised in Figure 7 and numerically reported in Table 2. CICYMiner leads to the best overall performance for all four Hodge numbers. The distributions of the residuals, x (i) appearing in the Huber Loss (4.1), show in Figure 8 a homoscedastic behaviour (no correlations between predictions and absolute value of the residuals), which ultimately supports the completeness of the model and its ability to properly predict the four Hodge numbers correctly.

The use of a different loss function, which is not robust against outliers, led to largest drop in accuracy, overall: the difference starts to be consistent even for h (2,1) and h (3,1) , which do not present many outliers in Figure 1. The accuracy plummets when considering h (2,2) , as expected. The presence of outliers is also evident when increasing the mini-batch size: h (2,2) suffers the largest decrease in accuracy due to such normalisation strategy. At the same time, the introduction of a batch-size independent Layer Normalization strategy, which normalizes each sample over the channel direction rather than the batch dimension, leads to a similar decrease. The presence of outliers seems, therefore, a delicate issue for which the size of the mini-batches plays a relevant role.

A related aspect is represented by the ablation study on the auxiliary branches. As their role is to mine a richer variety of features to stabilise the shared representation, and learn better approximations of the output, the accuracy drops significantly in the case of highly imbalanced distributions. The largest drop impacts h (2,1) which suffers from predictions shifting towards zero. This shows that we indeed need a mechanism to get as much training information as possible through the addition of transformations and auxiliary branches, as in the CICYMiner. Finally, we analyse the impact of the attention modules: we insert such additional layers to improve the predictions of h (3,1) and h (2,2) only, as other Hodge numbers do not need additional transformations. The results do not strongly differ from the case without the attention modules, though h (2,2) drops by 2 % in accuracy. It therefore seems that the attention modules do not help the predictions in this case, supported by the naive intuition that the configuration matrices do not suggest the development of a sequence model, such as in Natural Language Processing (NLP) or deep learning for video sequences. However, the accuracy reached by the model occurs at around 100 training epochs, rather than 300 as in other cases. The loss function then presents h (1,1) h (2,1) h (3,1) a slight increase after that. The use of attention modules, together with an early stopping strategy, may therefore significantly cut the training time in this context.

Conclusion

In this paper, we were able to show that Inception-based neural networks achieve good accuracy in predicting h (3,1) and h (2,2) and can reach perfect accuracy for the Hodge numbers h (1,1) , h (2,1) . Earlier studies using dense architectures were only able to work accurately with h (1,1) [START_REF] He | Machine Learning Calabi-Yau Four-folds[END_REF]. Moreover, we showed that only a fraction of the training data is needed to already obtain promising results. This stands in contrast to earlier studies on CICY three-folds for which it was not possible to accurately predict h (2,1) (the only remaining non-trivial Hodge number in that case). The significant increase in dataset size is responsible for a good part of the increase in performance: the risk of overfitting is strongly reduced and generalization over all configuration matrices is more robust. This is also reflected in the observation that removing the tails of the Hodge number distribution is no longer needed in order to obtain good results. Our main results show that, given the two constraints (3.3) and (3.4) derived from tangent bundle indices, we are able to solve the problem of predicting all Hodge numbers with perfect accuracy.

Our results demonstrate that it is possible to obtain very accurate predictions for the dimension of cohomology groups with only partial training data. We emphasize that the computations of more generic vector bundle cohomologies also satisfy several linear relations and constraints derived from the index, Serre duality or vanishing theorems such as Kodairas. Thus, it is often sufficient to predict a single Hodge number with great accuracy to gain knowledge of all the others. In our experiments, training and validation error align, and we do not observe any significant high variance issues. The high validation and test accuracy suggests that the algorithm produces reliable results, even if it is only trained on partial data, say 30 %. This should open up venues for further investigation into other vector bundle computations.

It is then important to find configurations which yield high accuracy on the validation set. In earlier studies, researchers have used feature enhancement to improve accuracy [START_REF] Erbin | Machine learning for complete intersection Calabi-Yau manifolds: a methodological study[END_REF][START_REF] He | Machine Learning Calabi-Yau Four-folds[END_REF]. Unfortunately, it is not always possible to manipulate the input data via feature engineering in such a way. Adding a relevant monomial basis changes the dimension of the input space in nontrivial ways, such that one has to restrict oneself to a subset of the configuration matrices.

We opted to follow a model-centered approach, common in contemporary machine learning literature, by building a proper architecture with the right amount of parameters. We balance the increased risk of overfitting, due to a larger number of trainable variables, with the natural regularization of multi-task architectures, thus increasing the final accuracy. With its 10 7 parameters, CICYMiner is, given its underlying geometric nature, still a small network with respect to many state-of-art models for computer vision or NLP. Good examples are represented by Inception-Resnet-v2 [START_REF] Szegedy | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[END_REF], state-of-the-art in (a) h (1,1) residuals.

(b) h (2,1) residuals.

(c) h (3,1) residuals.

(d) h (2,2) residuals. single-task image classification with 5.6 × 10 7 parameters and a long training time on 20 NVidia Kepler GPUs, and GPT-3 [START_REF] Brown | Language Models are Few-Shot Learners[END_REF], state-of-the-art NLP model, with more than 175 × 10 9 model parameters. In fact, recent research suggests that neural networks often admit power-scaling laws with dataset size and model parameters [START_REF] Bahri | Explaining Neural Scaling Laws[END_REF]. It can also be noted that increasing the capacity of the model may be beneficial to the overall performance [START_REF] Belkin | Reconciling modern machine-learning practice and the classical bias-variance trade-off[END_REF]. However, the geometric and physical interpretability might then become quite complicated and involved, hence the suggestion to constrain the complexity of the CICYMiner architecture. It would be interesting to observe how far one can improve the accuracy of h (3,1) and h (2,2) by using larger networks or adding more data samples to the dataset, or even by just prolonging the training time on multiple GPUs. Additional data samples can in principle be easily generated via (in-)effective splits of the already existing configuration matrices. These redundant matrices had been discarded when compiling the initial dataset [START_REF] Gray | All Complete Intersection Calabi-Yau Four-Folds[END_REF]. As a conclusion, our paper builds further the case for using deep learning in algebraic geometry by demonstrating that an appropriate neural network architecture can predict accurately Hodge numbers of CICY. Moreover, since algebraic geometry uses datasets which are not of the type usually encountered in usual machine learning applications, our results extend their range of applications.

Figure 1 :

 1 Figure 1: The plots show the histograms with logarithmic y-axis of the four non-trivial Hodge numbers.In the first row we have on the left the distribution of h(1,1) , to the right of h(2,1) . In the bottom row h(3,1) is presented to the left and h(2,2) to the right.

Figure 3 :

 3 Figure 3: The first row shows training loss and accuracy plotted against the computation budget. The error bars represent the upper and lower bounds for the four different training ratios. In the second row, we plot on the left the validation accuracy and on the right the test accuracy of the best performing models for the different training ratios.

Figure 4 :

 4 Figure 4: The basic building block of CICYMiner are Inception modules. The architecture is built to enable the hard parameter sharing in the bottom layer, in order to construct a common representation of the input. The task specific sub-structures then replicate the behaviour through an auxiliary branch, which further uses dense layers, batch normalization (BN) and dropout (D) to control overfitting. The final model predicts all Hodge numbers at once. The composition of the Inception modules are shown in Figure 2.

Figure 6 :

 6 Figure 6: Substructures of the attention mechanism used in the ablation study. Here, × indicates a matrix product along appropriate axes, while • is the Hadamard product (element-wise). Reshape operations (R) are also indicated.

Figure 7 :

 7 Figure 7: Summary of the ablation study.

Figure 8 :

 8 Figure 8: Residual plots at 80 % training ratio.

.00 0.95 0.70 0.75

	10 % 0.99	0.87	0.59	0.62
	30 % 1.00 0.91	0.67	0.73
	50 % 1.00 0.94	0.68 0.75
	80 % 1mean 1.00	0.92	0.66	0.71

Table 1 :

 1 Comparison of the test accuracy for different training ratios.

  On the left, we show the final test accuracy of CICYMiner for the four different training ratios. On the right, we present the loss function at 80 % training ratio, smoothed with a running average over 20 epochs.

		0.9 1.0													training validation
	accuracy	0.3 0.4 0.5 0.6 0.7 0.8						h (2, 2) h (3, 1) h (2, 1) h (1, 1)	loss (mean squared error)	10 0 10 1
		0.1	0.2	0.3	0.4 training ratio 0.5	0.6	0.7	0.8		0	50	100	150 epochs	200	250	300
	Figure 5:

Table 2 :

 2 h(2,2) Comparison of the accuracy obtained by similar models at 80 % training ratio. Regression metrics are also specified for CICYMiner at the same ratio.

							1.0
	+att	1.00 0.99 0.96 0.81		0.8
	MSE loss	1.00 0.97	0.92	0.50
	no aux bs-256 layer norm	1.00 0.84 1.00 0.99 1.00 0.99	0.92 0.94 0.92	0.72 0.65 0.66	accuracy	0.4 0.6
	CICYMiner 1.00 1.00 0.96 0.83 MSE (10 -4 ) 1.3 98 560 6800		0.2
	MAE (10 -3 )	7.8	19	130	360		0.0	h (1, 1)	h (2, 1)	h (3, 1)	h (2, 2)
								CICYMiner +att		MSE loss no aux	bs-256 layer norm

There exists another dataset of CICY three-folds in which 99.1 % are favourable[START_REF] Anderson | Fibrations in CICY Threefolds[END_REF], but no such feature enhanced data is available for the four-folds. However, the results from[START_REF] Erbin | Inception neural network for complete intersection Calabi-Yau 3-folds[END_REF]

, 22] show that using favourable matrices helps mostly in computing h[START_REF] Ruehle | Data science applications to string theory[END_REF][START_REF] Ruehle | Data science applications to string theory[END_REF] .

Using a five-fold increase in network weights (4 × 10 6 ) one is able to improve the accuracy of h[START_REF] Halverson | Computational Complexity of Vacua and Near-Vacua in Field and String Theory[END_REF][START_REF] Ruehle | Data science applications to string theory[END_REF] and h[START_REF] Denef | Computational Complexity of the Landscape I[END_REF][START_REF] Denef | Computational Complexity of the Landscape I[END_REF] to over 80 %. However, this comes at the cost of significant more training time and we then enter the regime where there are more weights than samples in the dataset.

Acknowledgements

RS is funded in part by the Swedish Research Council (VR) under grant numbers 2016-03873, 2016-03503, and 2020-03230. RS is grateful for 12 financial support from the Liljewalch scholarship. HE is funded by the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No 891169. HE is also supported by the National Science Foundation under Cooperative Agreement PHY-2019786 (The NSF AI Institute for Artificial Intelligence and Fundamental Interactions, http://iaifi.org/). The work of RF is supported by a joint programme (PTC) between the Direction des énergies and the Direction de la recherche tecnologique of the CEA Paris-Saclay. Computations were in part enabled by resources provided by the Swedish National Infrastructure for Computing (SNIC) at the HPC cluster Tetralith, partially funded by the Swedish Research Council through grant agreement no. 2018-05973, and the FactoryIA supercomputer, financially supported by the Ile-de-France Regional Council.


D Passiatore
email: donatella.passiatore@poliba.it

L Sciacovelli
email: luca.sciacovelli@ensam.eu

P Cinnella
email: paola.cinnella@sorbonne-universite.fr

G Pascazio

DIRECT NUMERICAL SIMULATION OF A HYPERSONIC BOUNDARY LAYER IN CHEMICAL NON-EQUILIBRIUM

The influence of high-enthalpy effects in hypersonic, spatially developing boundary layers is investigated by means of direct numerical simulations. The flow of a reacting mixture of nitrogen and oxygen over a flat plate at Mach 10, previously investigated in the literature using linear stability theory (LST), is simulated using a computational domain encompassing the laminar, transitional, and turbulent regimes. Transition is triggered by forcing Mack's second mode through suction and blowing at the wall. In the laminar region, the solution matches reasonably well the locally self-similar profiles, computed under chemical non-equilibrium assumptions. Strong dissociation phenomena are observed, due to the high temperatures reached close to the (uncooled) plate surface. The transitional regime is investigated by means of modal analysis. Despite the significant chemical activity, the results confirm the classical transition scenario for high-Mach number boundary layers, for which the secondmode resonance is the main mechanism responsible for turbulent breakdown. In the turbulent region, first-and second-order statistics reveal that chemical reactions do not modify significantly dynamic quantities such as velocity and Reynolds stress profiles, but greatly affect thermal properties, due to their endothermic nature. For the configuration at hand, chemical dissociation is slower than the characteristic time-scale of the flow, and the peak of chemical activity is located in the viscous sublayer, leading to mild modifications of the turbulent field compared to a frozen-chemistry model.

INTRODUCTION

The investigation of turbulent hypersonic flows is a major subject when dealing with planetary atmosphere reen-tries or vehicles capable of flying at hypersonic speed in the atmosphere. Recently, new vehicle concepts involving hypersonic flight are arousing interest not only in the defense and military fields, but also in the areas of spatial tourism and trans-atmospheric flights. The accurate prediction of boundary layer phenomena, such as laminarto-turbulent transition, near-wall turbulence, wall friction and heat flux is of utmost importance in order to improve the design of hypersonic vehicles. Moreover, the amount of energy present in these specific configurations can cause vibrational excitation of the gas molecules and chemical reactions (with characteristic times similar to those of the fluid motion), giving rise to nonequilibrium thermochemical states. Such high-enthalpy effects have a strong impact on aerodynamic performances and heat transfer rates, but may also affect transition and turbulence dynamics [START_REF] Candler | Rate effects in hypersonic flows[END_REF]. Despite the enormous progress realized so far, high-speed aerodynamics and aerothermodynamics constitute some of the areas where improvements need to be made. Carrying out physical experiments in the working conditions of interest is generally a costly (sometimes unfeasible) task, given the difficulty in reproducing such critical configurations [START_REF] Bertin | Critical hypersonic aerothermodynamic phenomena[END_REF]. To palliate the lack of experimental data, it is possible to carry out scale-resolving numerical simulations, such as Large-Eddy Simulations (LES) or Direct Numerical Simulations (DNS), at least in a range of moderate Reynolds numbers. In particular, DNS ensures the proper resolution of the whole spectrum of active temporal and spatial scales, without any hypothesis nor modeling assumption, allowing to avoid the introduction of turbulence models not tailored (and thus not enough reliable) for hypersonic flows [START_REF] Roy | Review and assessment of turbulence models for hypersonic flows[END_REF]. Despite the high computational cost of DNS, the generation of high-fidelity databases for out-of-equilibrium, high-speed flows is therefore of great importance for understanding the underlying physical mechanisms and improving the existing turbulence models. There exist limited DNS studies on turbulent, hypersonic boundary layers. For instance, Duan et al. [START_REF] Duan | Direct numerical simulation of hypersonic turbulent boundary layers with varying freestream Mach number[END_REF][START_REF] Duan | Direct numerical simulation of hypersonic turbulent bound-ary layers. Part 2. Effect of wall temperature[END_REF] performed DNS of temporally-evolving, zero-pressuregradient turbulent boundary layers in the high-Mach regime and varying wall temperatures; whereas Mach numbers up to 20 were considered from Lagha et al. [START_REF] Lagha | A numerical study of compressible turbulent boundary layers[END_REF]. These configurations, however, do not exhibit hightemperature effects due to the cryogenic free-stream conditions. A comparative study between low-and highenthalpy, reactive boundary layers was performed subsequently [START_REF] Duan | Direct numerical simulation of hypersonic turbulent boundary layers. Part 4. Effect of high enthalpy[END_REF], although moderate temperature values were imposed at the wall (≈ 2400). Recently, more attention has been paid to hypersonic, cold-wall boundary layers in spatially-evolving configurations [START_REF] Zhang | Direct numerical simulation database for supersonic and hypersonic turbulent boundary layers[END_REF][START_REF] Huang | Simulation and modeling of cold-wall hypersonic turbulent boundary layers on flat plate[END_REF]. The objective of the current paper is to assess the influence of strong chemical activity, caused by hypersonic conditions, on wall-bounded turbulent flows. Specifically, we aim at generating a new DNS database of highenthalpy, spatially-developing turbulent boundary layers. The current analysis encompasses the laminar, transitional and fully turbulent regime. Free-stream conditions were chosen such that chemical reactions are triggered by the high near-wall temperature values. A nominal Mach number equal to M ∞ =10 and a wall temperature computed under adiabatic condition are considered. These conditions have been widely used in the past for stability studies [START_REF] Malik | Real gas effects on hypersonic boundary-layer stability[END_REF][START_REF] Marxen | A high-order numerical method to study hypersonic boundary-layer instability including hightemperature gas effects[END_REF][START_REF] Marxen | Direct numerical simulations of hypersonic boundarylayer transition with finite-rate chemistry[END_REF]. The present DNS study aims at providing further insights on the complete transition scenario, broadening the existing knowledge on the purely transitional regime to the fully turbulent one. The paper is structured as follows. The governing equations, the numerical method and the simulation set-up are described in Section 2. Numerical results are presented in Section 3. After a brief investigation on the laminar flow regime, breakdown to turbulence is carefully scrutinized by means of spectral and modal analyses; afterwards, an overview of the main statistical turbulent quantities is provided. Conclusions and future works are drawn in Section 4.

METHODOLOGY

Governing equations

Flows under investigation are governed by the compressible Navier-Stokes equations for multicomponent, chemically-reacting gases, which read in differential form as:

∂ ρ ∂t + ∂ ρu j ∂ x j = 0 (1)
∂ ρu i ∂t + ∂ (ρu i u j + pδ i j ) ∂ x j = ∂ τ i j ∂ x j (2)
∂ ρE ∂t + ∂ [(ρE + p) u j ] ∂ x j = ∂ (u i τ i j -q j ) ∂ x j - ∂ ∂ x j N ∑ n=1 ρ n u D n j h n (3)
∂ ρ n ∂t + ∂ ρ n u j ∂ x j = - ∂ ρ n u D n j ∂ x j + ωn (4)
being ρ = ∑ N n=1 ρ n the mixture density, N the total number of species, u i the velocity vector components, p the pressure, δ i j the Kronecker symbol and τ i j the viscous stress tensor, defined as

τ i j = µ ∂ u i ∂ x j + ∂ u j ∂ x i - 2 3 µ ∂ u k ∂ x k δ i j , (5)
with µ the mixture dynamic viscosity. Moreover, E = e + ∑ N n=1 h f n Y n + 1 2 u i u i denotes the total chemical energy, with e the specific internal energy, h f n and Y n = ρ n /ρ the n-th species enthalpy of formation and mass fraction, respectively; q j represents the heat flux modelled by means of the Fourier law, q j = -λ ∂ T ∂ x j , λ being the mixture thermal conductivity and T the temperature. Lastly, u D n j , h n , and ωn are the n-th species diffusion velocity (defined hereafter), specific enthalpy and rate of production, respectively. The code solves the mixture density and N -1 species conservation equations, the N-th species being computed as ρ N = ρ -∑ N-1 n=1 ρ n . In these simulations, the N-th species is Nitrogen (see eq. ( 8)) since it keeps the largest mass fraction throughout the computational domain. Each species is assumed to behave as a perfect gas; Dalton's pressure mixing law leads then to the mixture pressure equation equation of state:

p = N ∑ n=1 p n = RρT N ∑ n=1 Y n W n = T N ∑ n=1 ρ n R n (6)
being R n and W n the gas constant and molecular weight of the n-th species, respectively, and R = 8.314 J/mol K the universal constant of gases. The thermodynamic properties of high-T air species are computed considering the contributions of translational, rotational and vibrational modes for each species [START_REF] Gnoffo | Conservation equations and physical models for hypersonic air flows in thermal and chemical nonequilibrium[END_REF]. An iterative Newton's method is used to compute T given the mixture internal energy and the species partial densities. With regard to the transport coefficients, pure species' viscosity and thermal conductivity are computed using Blottner's model and Eucken's formula, respectively [START_REF] Blottner | Chemically reacting viscous flow program for multi-component gas mixtures[END_REF]; the corresponding mixture properties are evaluated by means of Wilke's mixing rule. The diffusion velocities are given by Fick's law

u D n j ρ n = -ρD n ∂Y n ∂ x j + ρ n N ∑ n=1 D n ∂Y n ∂ x j , (7)
where D n is an equivalent diffusion coefficient of species n into the mixture, computed as in [START_REF] Gupta | A review of reaction rates and thermodynamic and transport properties for an 11-species air model for chemical and thermal nonequilibrium calculations to 30000 K[END_REF]. The second term in the right-hand-side of eq. ( 7) represents a correction needed to recover total mass conservation (see Ref. [START_REF] Poinsot | Theoretical and numerical combustion[END_REF]).

Air is modeled as a five-species mixture of N 2 , O 2 , NO, O and N. The chemical source terms ωn are computed following Park's model [START_REF] Park | Assessment of two-temperature kinetic model for ionizing air[END_REF], for which the species interact with each other through a 17-reaction mechanism:

N 2 + M ⇐⇒ 2N + M O 2 + M ⇐⇒ 2O + M NO + M ⇐⇒ N + O + M (8) N 2 + O ⇐⇒ NO + N NO + O ⇐⇒ N + O 2
where M represents any of the five species considered. Lastly, the reaction rates are modeled by means of Arrhenius' law.

Numerical methods

The governing equations are approximated using highorder finite-difference schemes. The convective fluxes are discretized by means of tenth-order schemes on 11points stencils, whereas standard fourth-order schemes are used for viscous fluxes. The discretization method is supplemented with a higher-order extension of Jameson's adaptive artificial dissipation [START_REF] Jameson | Numerical solutions of the Euler equations by finite volume methods using Runge-Kutta time-stepping schemes[END_REF]. A highly-selective shock sensor, built on a combination of the original Ducros' extension [START_REF] Ducros | Large-eddy simulation of the shock/turbulence interaction[END_REF] of Jameson's pressure-based sensor with the Bhagatwala & Lele modification [START_REF] Bhagatwala | A modified artificial viscosity approach for compressible turbulence simulations[END_REF] is used to trigger the shock-capturing term. Time integration is carried out by means of a third-order TVD Runge-Kutta scheme [START_REF] Gottlieb | Total variation diminishing runge-kutta schemes[END_REF].

Simulation set-up

A boundary-layer over a semi-infinite flat plate is considered. The thermodynamic conditions are similar to those adopted in several stability studies (see Refs. [START_REF] Malik | Real gas effects on hypersonic boundary-layer stability[END_REF][START_REF] Mary L Hudson | Linear stability of hypersonic flow in thermochemical nonequilibrium[END_REF][START_REF] Franko | Effects of chemistry modeling on hypersonic boundary layer linear stability prediction[END_REF][START_REF] Marxen | Direct numerical simulations of hypersonic boundarylayer transition with finite-rate chemistry[END_REF][START_REF] Miró | Diffusion and chemical nonequilibrium effects on hypersonic boundary-layer stability[END_REF]); specifically, the imposed free-stream values are M ∞ =10, T ∞ =350 K and p ∞ =3596 Pa. Air in equilibrium at its free-stream conditions is prescribed (namely, Y N 2 = 0.767082 and Y O 2 = 0.232918, similar to Marxen et al. [START_REF] Marxen | Direct numerical simulations of hypersonic boundarylayer transition with finite-rate chemistry[END_REF]). Adiabatic, non-catalytic boundary conditions are imposed at the wall. The domain is rectangular with even spacing in the streamwise (x) and spanwise (z) directions; in the wall-normal (y) direction, the following grid stretching is applied: where L y and n y denote the domain length and the number of grid points along y-direction, respectively; j ∈ [1, n y ] and α=0.13. The extent of the computational domain is

y( j) L y = (1 -α) j -1 n y -1 3 + α j -1 n y -1 (9)
L x × L y × L z = 8000δ * in × 320δ * in × 100πδ * in (the inlet dis- placement thickness δ *
in of the boundary layer being used as length scale) discretized with N x × N y × N z = 5520 × 256 × 240 points. Locally self-similar profiles, computed under finite-rate chemistry assumptions, are prescribed at the inlet at a distance corresponding to Re δ * =6375 (Re δ * denoting the Reynolds number based on the displacement thickness). The derivation of the self-similar equations and their numerical solution are shown in the next section. A blowing and suction method, similar to the one used by Franko & Lele [START_REF] Franko | Breakdown mechanisms and heat transfer overshoot in hypersonic zero pressure gradient boundary layers[END_REF], is adopted to trigger transition to turbulence. Specifically, the suction-andblowing function is imposed along a forcing strip located at Re δ * = 13880 and reads:

v w u ∞ = f (x)g(z) N mode ∑ m=1 A m sin(ω m t + β m z), (10)
where f (x) and g(z) are two perturbation-modulation functions defined as in Franko & Lele [START_REF] Franko | Breakdown mechanisms and heat transfer overshoot in hypersonic zero pressure gradient boundary layers[END_REF] and A m ,

ω m = ω m δ * in /c ∞ and β m = β m δ *
in represent the amplitude, non-dimensional pulsation and spanwise wave number, respectively. Here, the superscript (•) denotes the dimensional values and c ∞ the freestream speed of sound. The collection of forcing parameters for each mode are listed in table 1. Lastly, a sponge layer is applied near the inlet up to 35δ * in , to prevent abrupt distortions caused by the high Mach number. A sketch of the computational domain, starting downstream of the leading edge, is provided in fig. 1.

Locally self-similar equations for reacting boundary layers

The imposition of a similarity solution at a given distance from the virtual origin of the plate allows one to avoid the numerical simulation of the plate leading edge, bypassing its singularity and reducing computational cost. Global self-similar solutions for high-T laminar boundary layers only exist in restricted cases, such as in presence of frozen or equilibrium chemistry assumptions [START_REF] Anderson | Hypersonic and high temeprature gas dynamics[END_REF].

Taking into account finite-rate chemistry, global similarity breaks down due to the the presence of source terms in the species equations. Nevertheless, it is possible to compute a locally (i.e., x-dependent) self-similar solution, allowing the introduction of inlet profiles suitable for the configuration under investigation. Starting from the boundary-layer equations for steady, compressible, multicomponent, reacting, two-dimensional and zeropressure-gradient flows, the following variable transformations are introduced (see Ref.

ψ ∂ ξ = 1 √ 2ξ f (η), ∂ ψ ∂ η = 2ξ f (η)
, with f = u/U e , allow to manipulate the boundary-layer equations obtaining the following formulation in the self-similar coordinate system:

C f + f f = 0 (11) ρ 2 D n ρ e µ e Y n -Y n NS ∑ i=1 D i D n Y i + fY n + 2ξ ωn /ρ ρ e µ e u 2 e = 0 (12)
T e h 0,e Cλ µ θ + f g + u 2 e h 0,e f f f + u 2 e h 0,e C f f + NS ∑ n=1 h n h 0,e ρ 2 D n ρ e µ e Y n -Y n NS ∑ i=1 D i D n Y i = 0 (13)
Here, g = h/h 0,e is the self-similar parameter for the enthalpy, h 0,e being the edge stagnation enthalpy, θ = T /T e and C = ρ µ/ρ e µ e denotes the Chapman-Rubesin parameter. In case of frozen chemistry or chemical equilibrium, the ξ -dependent term in eq. ( 12) vanishes and the resulting equations become globally self-similar. The system (11)-( 12) constitutes a two-point boundary value problem, subjected to the following boundary conditions:

f = f = 0, Y n = 0, g = 0 for η = 0 f = 1, Y n = Y n,e , g = 1 - 1 2 u 2 e h 0,e for η → ∞,
which is easily integrated numerically by means of a globally-convergent Newton method.

RESULTS

After having reached a statistically steady state, timeand spanwise-averaged statistics are collected for approximately two turnover times with a sampling time interval of ∆t s c ∞ /δ * in = 3.45 × 10 -2 . To carry out the spectral analysis, a selection of subvolumes of the computational domain is extracted at a sampling interval of ∆t s c ∞ /δ * in = 2.30 × 10 -1 , corresponding exactly to sixteen snapshots per each period of the fundamental forcing harmonic. Figure 2 displays an instantaneous visualization of the boundary layer spatial evolution. The blowing and suction forcing, located near the inlet, triggers the turbulence breakdown and provides large transitional and turbulent zones to analyse. In the following, streamwise profiles are shown as a function of the nondimensional quantity (xx 0 )/δ * in , x 0 being the distance from the leading edge. The inspection of the skin friction coefficient

C f = 2τ w ρ ∞ u 2 ∞
(where (•) denotes the Reynolds average), shown in fig. 3(a), reveals that the breakdown appears at ≈ 3000δ * in , whereas the fully turbulent region starts from ≈ 5000δ * in , i.e. from the streamwise location at which the downstream decrease of C f begins.

Laminar regime

The laminar regime is analysed first. Figure 3 ρ ∞ µ ∞ , and the locally self-similar solution at selected stations. It is worth noting that after the forcing strip (not visible in the figure), the evolution of C f matches the laminar correlation, meaning that no mean-flow distortion has occurred yet. On the other hand, the similarity theory predicts slightly higher values of C f ; this gap is likely due to the presence of an external pressure gradient in the full Navier-Stokes equations, which is neglected in the self-similar ones. Similar slight discrepancies are present in the streamwise evolution of species mass fractions at the wall (fig. 4(a)). Overall, the numerical solution and the similarity theory are found to be in fair agreement. As the Reynolds number increases, the natural evolution of the boundary layer gives rise to

Transitional regime

The nonlinear breakdown to turbulence is now investigated. The transition scenario is described by means of a modal analysis, which consists in evaluating the amplitude of excited modes through a spatio-temporal Fourier transform. The new coordinate system is then based on the angular frequency and spanwise wavenumber (ω, β ).

For the sake of simplicity, the angular frequency is normalized by 0.855 and the spanwise wave number by 0.2, corresponding to the couple (ω 9 , β 9 ) in table 1. In previous works [START_REF] Franko | Breakdown mechanisms and heat transfer overshoot in hypersonic zero pressure gradient boundary layers[END_REF][START_REF] Gloerfelt | Silent inflow condition for turbulent boundary layers[END_REF], the amplitude of each mode is evaluated employing the modal energy, which takes into account both the kinetic and internal energy contributions.

On the other hand, other references make use of the maximum amplitude of the streamwise velocity fluctuation [START_REF] Marxen | Direct numerical simulations of hypersonic boundarylayer transition with finite-rate chemistry[END_REF]. The latter criterion is retained in this work for the modal growth analysis. To this aim, instantaneous velocity values are extracted along a cut of the computational domain in the y-direction, which follows the streamwise evolution of the wall-normal location at which the ufluctuation peaks. A total number of 800 samples are considered. The maximum disturbance amplitude is then evaluated as |u | 2 max = | u u * |/u 2 ∞ (where (•) stands for the Fourier component and (•) * denotes the complex conjugate) and results in a function of (ω/ω 9 , β /β 9 , x). Figure 5 displays the normalized streamwise velocity in a xz plane, cut at the end of the transitional zone. The qualitative evolution of the disturbances shows that the footprint of three-dimensional modes starts to be visible at (xx 0 )/δ * in ≈ 2500, albeit they do not emerge significantly. This behavior underlines that the predominant mode in the first part of the domain is the two-dimensional Mack mode, excited with the strongest amplitude through suction and blowing (see tab. 1). Nonlinear effects lead to the growth of oblique and stationary modes and cause the development of streaks which, in this specific configuration, rapidly destabilize the flow and promote the breakdown to turbulence. This scenario occurs in the region 3000 < (xx 0 )/δ * in < 4000, corresponding to the slight kink in the C f of figure 3(a), which indicates that a mean-flow distortion has occurred. To better understand this mechanism, frequency-wavenumber spectra are dis- accordance with the modes excited by the forcing function, the couple (2,0) being the strongest in magnitude.

(x -x 0 )/δ * in = 36 ω/ω 9 (x -x 0 )/δ * in = 2000 ω/ω 9 (x -x 0 )/δ * in = 3280 ω/ω 9 (x -x 0 )/δ * in = 4000 ω/ω 9 β /β 9
Moving towards the successive streamwise location, the amplitude of sub-harmonics (1,1) and (-1,1) starts to increase; these modes are explicitly introduced in the forcing function but with an extremely low amplitude with respect to the primary harmonic, and are amplified by the flow as they move downstream. Finally, at the last two selected stations, exchange between various waves has led to the growth of the stationary waves (0,2) and (0,1), respectively. In order to follow the entire streamwise development, figure 7 shows the maximum velocity disturbance amplitude of selected modes. Initially, the mode (2,0) predominates over all the excited modes, but it rapidly decays and then saturates. At the same time, the sub-harmonic oblique wave (1,1) drops of several orders of magnitude first, and then becomes the most energetic mode at (xx 0 )/δ * in ≈ 2000. Downstream of this station, the streaks modes (0,1) and (0,2) start to emerge. The higher streak mode (0,2) initially dominates, but at later transition stages the principal stationary mode (0,1) overwhelms the former, leading to the breakdown and to the final saturation stage (not shown in the figure). The transition event is qualitatively similar to the second-mode breakdown of [START_REF] Franko | Breakdown mechanisms and heat transfer overshoot in hypersonic zero pressure gradient boundary layers[END_REF] for a M ∞ = 6 non-reacting boundarylayer. Late transition is typical of this scenario, since the stationary waves start to gain energy once the Mack mode saturates. We also observe that the present transition mechanism differs from wall-cooled boundary layers, in which early and persistent streaks are visible from the beginning of the transitional zone; here, the extremely high temperature results in late formation of the streaks which, by the way, remain the principal responsible for transition to turbulence.

u + V D = 1 u τ u 0 ρ ρ w du ( 14
)
The logarithmic law is well described by u + = 1 κ ln(y + ) + C, with κ=0.41 and C=5.8. Different values for the constant C have been used in literature, ranging from 5.2 to 6.2 [START_REF] Bradshaw | Compressible turbulent shear layers[END_REF][START_REF] Maeder | Numerical investigation of supersonic turbulent boundary layers[END_REF][START_REF] Duan | Direct numerical simulation of hypersonic turbulent boundary layers. Part 4. Effect of high enthalpy[END_REF]; the value 5.8 seems to be suitable in this case. The data are found to better match the logarithmic law as the Reynolds number increases; the higher values in the outer layer encountered when moving further downstream are due to the decreasing trend of the friction velocity u τ . Figure 9 reports the profiles of the Damköhler number Da n = ωn /ρ µ w /τ w , which is a measure of the ratio between the flow residence times and the chemical relaxation times. Small values of Da n imply that chemical dissociation is slower than the characteristic time scale of the flow and the monotonic trend highlights that the maximum chemical activity is close to the wall. Furthermore, the profiles at lower Re θ exhibit slightly higher values in magnitude. Globally, the Reynolds stresses (shown in fig. 10(a) in inner scaling) confirm previous observations [START_REF] Duan | Direct numerical simulation of hypersonic turbulent boundary layers. Part 4. Effect of high enthalpy[END_REF] stating that the influence of chemical reactions on the kinematic field is small. Of note, the semi-local transformation introduced in Trettel & Larsson's work [START_REF] Trettel | Mean velocity scaling for compressible wall turbulence with heat transfer[END_REF] chemistry, since species mass fractions do not directly follow the temperature trend.

CONCLUSIONS

High-enthalpy effects on compressible wall-bounded turbulent flows have been analyzed by means of direct numerical simulations of a hypersonic, chemically out-ofequilibrium turbulent boundary layer. The novelty of this work lies in a complete analysis of one of the few existing spatially-developing boundary layers with finite-rate chemistry, encompassing laminar-to-turbulence breakdown as well. The free-stream thermodynamic conditions, albeit not representative of any physical re-entry trajectory or trans-atmospheric flight, are chosen to match those of a configuration largely analyzed in previous stability studies; specifically, a nominal Mach number of 10 and a free-stream temperature of 350 K. The temperatures reached in the boundary layer are such that gas dissociation phenomena occur, giving rise to a non- Wall quantities such as the skin friction coefficient, mean temperature and species mass fractions are analysed and compared to the local self-similar solution in the first portion of the domain. The profiles follow the same trend of the similarity results, even though discrepancies are present due to the hypothesis of null pressure gradient in the boundary layer equations set. As the reacting mixture evolves along the plate, oxygen is partially dissociated, but the equilibrium composition is never reached. A modal analysis based on the maximum streamwise velocity fluctuations points out that the mechanism responsible for the breakdown is the second mode fundamental resonance. This scenario leads to a late transition and to the growth of streaks modes once that the twodimensional mode saturates; transition is observed at a very short distance downstream of the streak appearance. Mean and fluctuating statistics reveal that chemical activity is mostly located between the plate wall and the buffer layer region, although the chemical time scales are much smaller than flow residence time scale throughout the flow. Fluctuations of species mass fractions exhibit Since most of the flow features are driven by the wall condition, further investigations are in progress for elucidating the influence of wall cooling. In that case, it is expected that the region of maximum chemical activity moves away from the wall, approaching that of maximum turbulence production and therefore enhancing the turbulence-chemistry interactions. Thermal nonequilibrium conditions will be considered as well, with the aim of quantifying the influence of thermal relaxation phenomena on wall turbulence dynamics.

Figure 1 :

 1 Figure 1: Sketch of computational domain.

  [START_REF] Lees | Laminar heat transfer over blunt-nosed bodies at hypersonic flight speeds[END_REF] for more details):ξ = ρ e µ e U e x = ξ (x), η = U e2ξ y 0 ρdy = η(x, y), where the subscript (•) e denotes variables computed at the boundary layer edge. The definition of the streamfunction ∂ ψ ∂ y = ρu, ∂ ψ ∂ x = -ρv and the equivalent expression in terms of transformed variables ∂

  (b) displays a close-up of C f in the first part of the domain, along with the compressible extension of Blasius' laminar correlation, C lam f = 0.664 √ Re x ρ w µ w

Figure 2 :Figure 3 :Figure 4 :

 234 Figure 2: Instantaneous visualization of the computational domain: isosurface of Q-criterion coloured with the distance from the wall. On the back, Schlieren-like density contours showing the propagation of waves outside the boundary layer.

Figure 5 :

 5 Figure 5: Instantaneous visualization of normalized velocity in a xz plane, extracted at y/δ * in = 13. Four streamwise positions are marked, at (xx 0 )/δ * in = 36, 2000, 3280 and 4000, the first one being the forcing strip.

Figure 6 :Figure 7 :

 67 Figure 6: Frequency-wave number spectra at different streamwise positions.

Figure 8 :Figure 9 :

 89 Figure 8: Wall-normal profiles of the van Driesttransformed streamwise velocity; ( ) Re θ = 5200, ( ) Re θ = 8000, ( ) Re θ = 10500.

Figure 10 :

 10 Figure 10: Wall-normal profiles velocity fluctuations (a) and temperature fluctuations (b), in inner scaling; ( ) Re θ = 5200, ( ) Re θ = 8000, ( ) Re θ = 10500.

Figure 11 :

 11 Figure 11: Relative fluctuating quantities of species mass fractions (Y rms /Y ) n ; ( ) Re θ = 5200, ( ) Re θ = 8000, ( ) Re θ = 10500. (Y rms /Y ) N is not shown due to its very low values.

Table 1 :

 1 Parameters of the modes excited by the forcing function in equation (10): non-dimensional frequency ω m , amplitude A m and spanwise wave number β m for the m-th mode.

	N mode	ω m	A m	β m
	1	1.71	0.05	0
	2	0.855 2.50 × 10 -3	0
	3	0	2.50 × 10 -3 +0.2
	4	0	2.50 × 10 -3 +0.4
	5	0	2.50 × 10 -3 +0.6
	6	1.71 2.50 × 10 -3 -0.2
	7	1.71 2.50 × 10 -3 +0.2
	8	0.855 2.50 × 10 -3 -0.2
	9	0.855 2.50 × 10 -3 +0.2
	10	1.71 2.50 × 10 -3 -0.4
	11	1.71 2.50 × 10 -3 +0.4
	12	1.71 2.50 × 10 -3 -0.6
	13	1.71 2.50 × 10 -3 +0.6

Table 2 :

 2 Numerical parameters of the selected locations for turbulence analysis.(xx 0 )/δ *

	in	Re θ	∆x + ∆y + w	∆y + δ	∆z +
	5077	5200 4.74 0.53 2.48 4.29
	6443	8000 4.65 0.52 3.36 4.20
	7575	10500 4.54 0.51 4.08 4.10
	3.3 Turbulent regime
	Moving further downstream, the flow enters in the fully
	turbulent region. Statistical quantities are investigated at
	three locations reported in table 2. A preliminary assess-
	ment of the velocity profiles scaling is shown in fig. 8,
	displaying the classical Van Driest transformation for the
	streamwise velocity:

ACKNOWLEDGMENTS

This work was granted access to the HPC resources of IDRIS and TGCC under the allocation A0072B10947 made by GENCI (Grand Equipement National de Calcul Intensif). We also acknowledge CINECA for awarding access to the Galileo supercomputer under the allocation HP10CLMXP0.