Davy Monticolo
email: davy.monticolo@univ-lorraine.fr

Schema First! Learn Versatile Knowledge Graph Embeddings by Capturing Semantics with MASCHInE

Keywords: CCS CONCEPTS, Computing methodologies → Artificial intelligence, Knowledge representation and reasoning, • Information systems → World Wide Web Knowledge Graph Embeddings, Schema-based Representation Learning, Link Prediction, Entity Clustering, Node Classification 1 Resource Description Framework. See

Knowledge graph embedding models (KGEMs) have gained considerable traction in recent years. These models learn a vector representation of knowledge graph entities and relations, a.k.a. knowledge graph embeddings (KGEs). Learning versatile KGEs is desirable as it makes them useful for a broad range of tasks. However, KGEMs are usually trained for a specific task, which makes their embeddings task-dependent. In parallel, the widespread assumption that KGEMs actually create a semantic representation of the underlying entities and relations (e.g., project similar entities closer than dissimilar ones) has been challenged. In this work, we design heuristics for generating protographs -small, modified versions of a KG that leverage RDF/S information. The learnt protograph-based embeddings are meant to encapsulate the semantics of a KG, and can be leveraged in learning KGEs that, in turn, also better capture semantics. Extensive experiments on various evaluation benchmarks demonstrate the soundness of this approach, which we call Modular and Agnostic SCHema-based Integration of protograph Embeddings (MASCHInE). In particular, MASCHInE helps produce more versatile KGEs that yield substantially better performance for entity clustering and node classification tasks. For link prediction, using MASCHinE substantially increases the number of semantically valid predictions with equivalent rank-based performance.

INTRODUCTION

Knowledge Graphs (KGs) such as DBpedia and YAGO represent facts as triples (ℎ, 𝑟, 𝑡) formed by a head ℎ and a tail 𝑡 linked by a semantic relation 𝑟 which qualifies the nature of their relationship. Typical learning problems with KGs are Link Prediction (LP), Entity Clustering (EC), and Node Classification (NC).

In most cases, these problems are addressed using Knowledge graph embedding models (KGEMs), which generate vector representations for entities and relations of a KG, a.k.a. Knowledge Graph Embeddings (KGEs). While generating dense and numerical vectors for entities and relations, KGEMs are expected to produce KGEs that retain the underlying semantics of the KG [START_REF] Wang | A Survey on Knowledge Graph Embeddings for Link Prediction[END_REF]. This widespread assumption that KGEMs create a semantic representation of the underlying entities and relations (i.e., project similar entities closer than dissimilar ones) has been challenged recently [START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF]. Their semantic capabilities presumably inherited from the rich information contained in the KG is neither fully satisfying nor consistent for tasks such as concept clustering [START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF]. In this work, we claim that this is because schemas (e.g. RDF/S and OWL) are often overlooked as an interesting source of information for improving embeddings.

Injecting schema-based information into the training phase could help enhancing the semantic awareness of KGEMs [START_REF] Hubert | Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction[END_REF]. This way, better performance can be reached using available ontological information, without introducing major overhead in models' complexity. A few works incorporate such schemas when training KGEM, whether it is in the loss function [START_REF] Claudia D'amato | Injecting Background Knowledge into Embedding Models for Predictive Tasks on Knowledge Graphs[END_REF][START_REF] Hubert | Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction[END_REF], in the negative sampling procedure [START_REF] Jain | Improving Knowledge Graph Embeddings with Ontological Reasoning[END_REF][START_REF] Krompaß | Type-Constrained Representation Learning in Knowledge Graphs[END_REF], or in the model representations [START_REF] Cui | Type-augmented Relation Prediction in Knowledge Graphs[END_REF][START_REF] Xie | Representation Learning of Knowledge Graphs with Hierarchical Types[END_REF]. All of those methods are usually bound to just one KGEM and therefore not universally applicable.

Another line of research that is less explored focuses on preprocessing graphs prior to encoding their components [START_REF] Chen | HARP: Hierarchical Representation Learning for Networks[END_REF][START_REF] Pietrasik | Probabilistic Coarsening for Knowledge Graph Embeddings[END_REF], rather than focusing on the representation capability of the KGEM itself. This idea of summarizing [START_REF] Cebiric | Summarizing semantic graphs: a survey[END_REF][START_REF] Goasdoué | RDF graph summarization for first-sight structure discovery[END_REF] or coarsening [START_REF] Pietrasik | Probabilistic Coarsening for Knowledge Graph Embeddings[END_REF] RDF 1 graphs proves useful in many circumstances. The intended goals and expected benefits are multiple: scaling up training [START_REF] Huang | Scaling Up Graph Neural Networks Via Graph Coarsening[END_REF], initializing embeddings with more robust vector representations learnt on the coarsened graph [START_REF] Pietrasik | Probabilistic Coarsening for Knowledge Graph Embeddings[END_REF], and improving predictive performance in KG-based downstream tasks [START_REF] Wang | Exploiting Global Semantic Similarities in Knowledge Graphs by Relational Prototype Entities[END_REF]. It is worth a reminder that several heuristics exist for coarsening or summarizing graphs [START_REF] Cebiric | Summarizing semantic graphs: a survey[END_REF]. All these approaches, however, use statistics over the graphs, instead of considering schema information. In contrast, our work focuses on using RDF/S information to generate an abstraction of the KG that encapsulates general knowledge about how entities and relations interact based on entity types, and relation domains and ranges. We call such an abstraction a protograph (see left part of Figure 2).

If considering schema-based information is usually done with the goal of improving results w.r.t. a given task such as LP [START_REF] Claudia D'amato | Injecting Background Knowledge into Embedding Models for Predictive Tasks on Knowledge Graphs[END_REF][START_REF] Hubert | Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction[END_REF], it would be worth investigating whether the learnt embeddings actually have stronger semantic capabilities [START_REF] Hubert | Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction[END_REF][START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF]. If so, they are expected to provide better performance with regards to other tasks such as entity classification and clustering. In this work, we propose an approach named MASCHInE that leverages RDF/S information in a model-agnostic way, making it applicable to arbitrary KGEMs. Specifically, we create and embed protographs from a KG schema. The protograph embeddings are then used to initialize the actual KGEs, and training starts on these pre-trained KGEs. Finally, we assess the versatility of these KGEs by comparing results with a vanilla, traditional learning approach.

The main contributions of our work are summarized as follows:

• We devise two heuristics for building protographs that aim at encapsulating concise and meaningful information derived from the schema that underpins a KG. • To the best of our knowledge, we propose the first work that studies the potential of pre-training embeddings with a protograph-assisted approach based on a schema, and subsequently use these embeddings for multiple tasks, e.g. link prediction, entity clustering and node classification. • Through extensive experiments on several benchmarks, we demonstrate the value of using schema-based protographs as a means to generate versatile, multi-purpose embeddings.

The remainder of the paper is structured as follows. Related work is presented in Section 2. In Section 3, we detail our approach for building protographs and how they fit into our approach MAS-CHInE. In Section 4, the potential usefulness of embeddings learnt with MASCHInE is discussed, and experiments w.r.t. several benchmarks are carried out. Lastly, Section 5 sums up the main findings and outlines promising future research directions.

RELATED WORK

Knowledge graph embeddings. KGEMs have gained significant attention in recent years due to their ability to represent structured knowledge in a continuous vector space. The seminal translational model TransE [START_REF] Bordes | Translating Embeddings for Modeling Multi-relational Data[END_REF] represents entities and relations as lowdimensional vectors and defines the relationship between a head, a relation, and a tail using a translation operation in the embedding space. Most of the KGEMs subsequently proposed aim at addressing its limitations and improve the representation expressiveness of KGEs [START_REF] Wang | A Survey on Knowledge Graph Embeddings for Link Prediction[END_REF], such as DistMult [START_REF] Yang | Embedding Entities and Relations for Learning and Inference in Knowledge Bases[END_REF], ComplEx [START_REF] Trouillon | Complex Embeddings for Simple Link Prediction[END_REF], ConvE [START_REF] Dettmers | Convolutional 2D Knowledge Graph Embeddings[END_REF], and TuckER [START_REF] Balazevic | TuckER: Tensor Factorization for Knowledge Graph Completion[END_REF]. Embeddings learnt by these KGEMs demonstrated potential applicability in tasks such as LP [START_REF] Wang | A Survey on Knowledge Graph Embeddings for Link Prediction[END_REF].

Schema-based representation learning. Many knowledge graphs are backed by a more or less expressive schema defining which classes of entities exist and by which relations they are allowed to interact, among others. Recent approaches in KG-based learning leverage the schema as a supplementary source of information to learn higher-quality embeddings for the task at hand. A common approach is to alter the negative sampling procedure in order to produce more realistic negative triples. For instance, type-constrained negative sampling (TCNS) [START_REF] Krompaß | Type-Constrained Representation Learning in Knowledge Graphs[END_REF] replaces the head or the tail with a random entity belonging to the same type as the ground-truth entity. The model itself can be adapted to take schema-based information into account. In TaRP [START_REF] Cui | Type-augmented Relation Prediction in Knowledge Graphs[END_REF], type and entity-level information are simultaneously considered and encoded as prior probabilities and likelihoods of relations, respectively. Altering the loss function with schema-based information is another line of research. In [START_REF] Claudia D'amato | Injecting Background Knowledge into Embedding Models for Predictive Tasks on Knowledge Graphs[END_REF], background ontological information is injected in the pairwise hinge loss function as constraints. Hubert et al. [START_REF] Hubert | Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction[END_REF] leverage domains and ranges of relations, and propose semantic-driven loss functions that extend the most frequently used loss functions in LP.

PROPOSED APPROACH

In this section, we detail the two heuristics we propose for building protographs using schema-based information. Next, we provide some perspective on the overall proposed approach MASCHInE.

Protographs

Most KGs are underpinned by a schema, e.g., DBpedia, Wikidata, and YAGO. Generating a protograph using schema-based information to improve KGEM performance with respect to a given task is an under-explored avenue.

The protographs we build leverage RDF/S information and contain an entity 𝑝 𝐶 for each class 𝐶, and the same set of relations as the KG. The first design principle consists in adding an edge for each relation with domain and range restrictions, i.e., given the two axioms 𝑑𝑜𝑚𝑎𝑖𝑛(𝑟, 𝐶 𝑖 ), 𝑟𝑎𝑛𝑔𝑒 (𝑟, 𝐶 𝑗 ), we add a triple (𝑝 𝐶 𝑖 , 𝑟, 𝑝 𝐶 𝑗 ) to the protograph2 . We refer to this strategy as P1.

The protograph P1 contains as many triples as there are relations with both a domain and range3 in the KG. This may lead to relatively small-sized protographs from which there is little data for the KGEM to learn from. In addition, relation domains and ranges can refer to generic classes. If entities are typed in a fine-grained way -i.e. with more specific classes -such classes will be absent from the protograph. As such, we propose a second design principle. Assuming we observe the following axioms 𝑑𝑜𝑚𝑎𝑖𝑛(𝑟, 𝐶 𝑖 ), 𝑟𝑎𝑛𝑔𝑒 (𝑟, 𝐶 𝑗 ), 𝑠𝑢𝑏𝑐𝑙𝑎𝑠𝑠𝑂 𝑓 (𝐶 ′ 𝑖 , 𝐶 𝑖 ), 𝑠𝑢𝑏𝑐𝑙𝑎𝑠𝑠𝑂 𝑓 (𝐶 ′ 𝑗 , 𝐶 𝑗 ), we add the following triples to the protograph: (𝑝 𝐶 𝑖 , 𝑟, 𝑝 𝐶 𝑗 ), (𝑝 𝐶 ′ 𝑖 , 𝑟, 𝑝 𝐶 𝑗 ), (𝑝 𝐶 𝑖 , 𝑟, 𝑝 𝐶 ′ 𝑗 ). In other words: for each subclass of the class appearing in the domain or range of a relation, an additional triple is created.

We refer to this protograph as P2. Figure 1 illustrates this process for a triple extracted from DBpedia.

P2 usually contains more triples than P1, at the possible expense of containing dubious ones. To illustrate, the DBpedia ontology contains the triples: 𝑑𝑜𝑚𝑎𝑖𝑛(𝑐𝑢𝑟𝑟𝑒𝑛𝑡𝑊 𝑜𝑟𝑙𝑑𝐶ℎ𝑎𝑚𝑝𝑖𝑜𝑛, 𝑆𝑝𝑜𝑟𝑡), 𝑟𝑎𝑛𝑔𝑒 (𝑐𝑢𝑟𝑟𝑒𝑛𝑡𝑊 𝑜𝑟𝑙𝑑𝐶ℎ𝑎𝑚𝑝𝑖𝑜𝑛, 𝐴𝑔𝑒𝑛𝑡), and 𝑠𝑢𝑏𝑐𝑙𝑎𝑠𝑠𝑂 𝑓 (𝐹𝑎𝑚𝑖𝑙𝑦, 𝐴𝑔𝑒𝑛𝑡). They lead to the triple (𝑃 𝑆𝑝𝑜𝑟𝑡 , 𝑐𝑢𝑟𝑟𝑒𝑛𝑡𝑊 𝑜𝑟𝑙𝑑𝐶ℎ𝑎𝑚𝑝𝑖𝑜𝑛, 𝑃 𝐹𝑎𝑚𝑖𝑙 𝑦 ) for which the relevancy and correctness are questionable. Conceptually, with P2, we accept some noise to be introduced in the protograph for the benefit of more triples being generated. However, the generation of dubious triples is limited by two factors: firstly, when adding triples that derive from an original triple (i.e. containing the domain and range of the relation), we only consider direct subclasses -in contrast to transitive subclasses. Secondly, we fix one side of the triple and generate as many new triples as there are direct subclasses of the class of the other side, i.e., we avoid generating triples (𝑝 𝐶 ′ 𝑖 , 𝑟, 𝑝 𝐶 ′ 𝑗 ).

MASCHInE

The proposed MASCHInE approach allows for generating pretrained embeddings based on a protograph, to be further fine-tuned on the initial KG. MASCHInE is intended to produce more versatile KGEs that can be successfully used in various tasks. Conceptually, MASCHInE is made of four different modules that are depicted in Figure 2 and further detailed below.

(a) Protograph building is the preliminary step that generates a second source of data to learn from. In particular, schema-based information is leveraged to create a protograph that encapsulates more general knowledge about how entities and relations interact based on ontological constraints such as entity types, relation domains and ranges, etc. A mapping dictionary is created to pair entities between the KG and the protograph (the set of relations in the KG and the protograph are identical). In this work, each KG entity maps to its (potentially multiple) most specific classes -represented as entities in the protograph. In Section 3.1, we presented two heuristics for building protographs with such constraints. Other possibilities exist and we leave them for future work.

(b) Protograph KGE learning is the task of learning KGEs on the protograph itself. This can be achieved regardless of the KGEM at hand and is therefore model-agnostic 4 .

(c) Instance KGE initialization consists in transferring protograph embeddings to the corresponding entity and relation embeddings of the KG. To do so, we assign each entity in the KG the embedding vector of its corresponding protograph entity using the dictionary created in step (a) (i.e., its most specific class). When an entity maps to multiple protograph entities (i.e. when there exist mutiple most specific classes for a given entity), the protograph embeddings of its most specific classes are averaged and the resulting embedding is transferred to the entity.

(d) Fine-tuning is performed on the entity KGEs. At this stage, embeddings learnt on the protograph could still intervene in the fine-tuning procedure. In this work, however, we stick to plain entity KGE initialization, i.e. protograph embeddings are frozen after being transferred, and do not interact with entity KGEs anymore.

EXPERIMENTS

In our experiments, we apply the MASCHInE approach presented in Section 3. We experiment with five popular KGEMs. We first train on protographs, and then use the protograph embeddings to initialize entity and relation embeddings in the KG. Training starts on the KG and the fine-tuned KGEs are ultimately assessed in three tasks, namely LP, EC, and NC. For each task, we compare the vanilla setting (V), i.e., directly training the respective KGEM on the KG, with the MASCHInE approach based on P1 (resp. P2). Datasets, optimal hyperparameters, scripts, and pre-trained embeddings are publicly released 5 . In the following, protographs, KG entities and relations are encoded using five mainstream KGEMs: TransE [START_REF] Bordes | Translating Embeddings for Modeling Multi-relational Data[END_REF], DistMult [START_REF] Yang | Embedding Entities and Relations for Learning and Inference in Knowledge Bases[END_REF], ComplEx [START_REF] Trouillon | Complex Embeddings for Simple Link Prediction[END_REF], ConvE [START_REF] Dettmers | Convolutional 2D Knowledge Graph Embeddings[END_REF], and TuckER [START_REF] Balazevic | TuckER: Tensor Factorization for Knowledge Graph Completion[END_REF]. Datasets. Due to the need for schema-defined KGs, the following three benchmark datasets are considered: YAGO14k, FB15k187, and DBpedia77k [START_REF] Hubert | Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction[END_REF]. In particular, all entities are typed and relations have both a defined domain and range. Table 2: Link prediction results on YAGO14k, FB15k187, and DBpedia77k using KGEMs trained without protograph (V), with P1, and P2. For a given dataset and KGEM, comparisons are made between the three settings, and best results are in bold fonts. Hits@K and Sem@K are abbreviated as H@K and S@K, respectively.

Link Prediction

YAGO14k FB15k187 DBpedia77k

MRR H@3 H@10 S@3 S@10 MRR H@3 H@10 S@3 S@10 MRR H@3 H@10 S@3 S@10 Evaluation metrics. This section is motivated by evaluating the fine-tuned KGEs for LP. Performance is assessed w.r.t. Mean Reciprocal Rank (MRR), Hits@𝐾, and Sem@𝐾 [START_REF] Hubert | Knowledge Graph Embeddings for Link Prediction: Beware of Semantics![END_REF][START_REF] Hubert | Sem@𝐾 : Is my knowledge graph embedding model semantic-aware?[END_REF]. The latter accounts for the proportion of triples whose predicted head (resp. tail) belongs to the domain (resp. range) of the relation.

Implementation details. KGEMs were implemented in PyTorch.

In accordance with [START_REF] Hubert | Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction[END_REF], KGEMs are trained during 400 epochs. For TransE, DistMult, and ComplEx, uniform random negative sampling [START_REF] Bordes | Translating Embeddings for Modeling Multi-relational Data[END_REF] was used to pair each train triple with one negative counterpart. ConvE and TuckER are trained using 1-N scoring [START_REF] Dettmers | Convolutional 2D Knowledge Graph Embeddings[END_REF].

In 1) and entities share many classes together. In this situation, the protograph may not contain discriminative enough information for pre-training embeddings, which is reflected in the similar results achieved regardless of the training setting. In addition, we observe that some KGEMs seem to benefit more from the MASCHInE approach. In particular, ConvE and TuckER tend to perform better w.r.t. both rank-based and semantic-oriented metrics. Overall, we notice that including protographs in the training phase helps making semantically valid predictions, as evidenced by the better Sem@𝐾 under P1 and P2. Protographs may help organize the embedding space in such a way that geometric distances between embeddings better reflect their semantic similarities. Regarding rank-based metrics, we notice equivalent performance, which could be explained since LP relies less on semantics and more on relational patterns [START_REF] Bhardwaj | Poisoning Knowledge Graph Embeddings via Relation Inference Patterns[END_REF] compared to other tasks such as entity clustering (see below).

Entity Clustering

Jain et al. [START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF] demonstrate that embeddings do not consistently capture the semantics of the KG (in particular, they only considered schema-agnostic approaches). In this section, we investigate whether embeddings learnt for the LP task under P1 and P2 can actually prove useful in EC.

4.2.1

Experiments on the original datasets. We investigate whether the embeddings learnt for the LP task (see Section 4.1) under P1 and P2 allow a better class separability for the task of EC.

Datasets. We reuse YAGO14k, FB15k187, and DBpedia77k as presented in Section 4.1. The ground-truth labels are the most-generic classes an entity belongs to. It should be noted that entities belonging to several most-generic classes are filtered out as they would be assigned multiple ground-truth labels.

Implementation details. Entity embeddings are retrieved at the best epoch on the validation sets of YAGO14k, FB15k187, and DB-pedia77k, for all the KGEMs and under the three settings V, P1, and P2. Then, following the evaluation protocol in Jain et al. [START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF], the k-means clustering algorithm is run using default parameters of scikit-learn 6 .

Evaluation metrics. Clustering results are evaluated using the Adjusted Rand Index (ARI) -which measures the similarity between the cluster assignments by making pairwise comparisons -and Normalized Mutual Information (NMI) -which measures the agreement between the cluster assignments. It should be noted that these metrics are applicable as we have the ground-truth labels for each entity. In particular, we compare clusters output by k-means with ground-truth labels that are the classes entities belong to. ARI and NMI results are reported in Figure 3. Lowest and highest scores are 0 (-1 for ARI) and 1, respectively.

Experimental results. Except for TuckER on FB15k187, both settings P1 and P2 provide substantial improvements. It seems that the entity embeddings learnt by TuckER are not able to cluster entities based on their classes, which diminishes the relevancy of using this model for drawing comparisons. That being said, the benefit of P1 and P2 is consistent across datasets. This is even more striking for YAGO14k, as evidenced in Figure 4: entities sharing the same ground-truth class tend to cluster more under P2. Jain et al. [START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF] pointed out the limitations of using KGEs for semantic representation of the underlying entities and relations. The relatively low performance of embeddings for EC under V (Figure 3 and Figure 4) indeed suggests that the semantic similarity between entities is not reflected through geometric similarities in their embeddings. However, P1 and P2 generate entity latent representations whose geometric similarities are more representative of their classes.

Experiments on GEval clustering problems.

In this follow-up entity clustering experiment, the GEval evaluation framework [START_REF] Maria | GEval: A Modular and Extensible Evaluation Framework for Graph Embedding Techniques[END_REF] is used. GEval provides gold standard datasets based on DBpedia and suggest clustering models, configurations and evaluation metrics 7 .

Datasets. GEval proposes 4 datasets based on DBpedia. The first dataset contain entities of classes dbo:AmericanFootballTeam and dbo:BasketballTeam. We do not use this dataset as DBpedia77k does not contain any of these entities. Instead, we use the other three datasets for which DBpedia77k has a biggest coverage. For clarity, we name these datasets as CC, CCB, and CAMAP. The first two ones contain different numbers of DBpedia entities of classes dbo:Cities and dbo:Countries, while the third dataset contains five types of entities. Therefore, there are two ground-truth clusters in the first two datasets and five clusters in the third one.

Implementation details. We rely on GEval guidelines and perform the preliminary filtering step described as follows. First, we select entity embeddings learnt on DBpedia77k for the LP task (see Section 4.1). Embeddings of entities that do not appear in the GEval benchmark datasets are filtered out. Clustering with all the algorithms suggested in the GEval framework is performed on the remaining entities and evaluated w.r.t. several metrics.

Evaluation metrics. As in the previous experiment, we use ARI.

Following the GEval guidelines, we additionally use Adjusted Mutual Information Score (AMI), V-Measure (VM), Fowlkes-Mallow Score (FM), Homogeneity (H), and Completeness (C). V-Measure is the harmonic mean of homogeneity and completeness measure, while the Fowlkes-Mallow Score is the geometric mean of pairwise precision and recall. All these metrics measure the correctness of the cluster assignments based on ground-truth labels. For each of these datasets, and under each setting, results achieved with the best performing clustering method are shown in Table 3. Lowest and highest scores are 0 (-1 for ARI) and 1, respectively.

Experimental results. First, it is worth noting that results on CAMAP are significantly higher than on the other two datasets. Recall that CAMAP includes entities belonging to five different classes, whereas CC and CCB only contain entities from dbo:Cities and dbo:Countries. In this experiment, the embeddings learnt by the KGEMs are most useful for performing EC with five ground-truth clusters than with two, which might seem counterintuitive. However, in [START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF], it is demonstrated that in a KG, only a small subset of its entities actually have an embedding with a meaningful semantic representation. This is the reason why for the clustering task, results are not consistent across subsets of entities [START_REF] Jain | Do Embeddings Actually Capture Knowledge Graph Semantics?[END_REF]. This is in line with our experimental findings that lead to the same conclusion. [START_REF] Zouaq | What is the schema of your knowledge graph?: leveraging knowledge graph embeddings and clustering for expressive taxonomy learning[END_REF] is also concerned with the ability of KGEMs to embed entities from the same class to the same region of the embedding space. Their results suggest that entities belonging to rare classes have embeddings that are less separable w.r.t. to entity embeddings of other classes compared with entities belonging to frequent classes. This assumption needs to be qualified in light of our experimental findings: lots of entities in DBpedia77k are dbo:City and dbo:Country. Following [START_REF] Zouaq | What is the schema of your knowledge graph?: leveraging knowledge graph embeddings and clustering for expressive taxonomy learning[END_REF], we should expect better results for CC and CCB in Table 3. However, it should be noted that both dbo:City and dbo:Country are subclasses of dbo:Place (equivalent to dbo:Location), which is over-represented in DBpedia77k 8 . It is arguably harder to separate entity embeddings for CC and CCB, as all of them actually belong to the same parent class. In contrast, CAMAP contain albums, cities, companies, movies, and universities. In this case, dbo:City is a child class of dbo:Place, dbo:Company and dbo:University are child classes of dbo:Agent while dbo:Album and dbo:Movie are child classes of dbo:Work. With three different generic classes, the semantic similarities between entities in CAMAP is more easily reflected into the geometric similarities of their embeddings. Interestingly, the discrepancies in results, i.e., better results on CAMAP compared with CC and CCB, are setting-dependent (V, P1, P2). Indeed, they are alleviated when using either P1 or P2. Under V, TransE, ComplEx, and ConvE provide substantially better results on CAMAP compared to CC. Once using P2, we cannot see any significant difference in results on CAMAP compared to CC for these models. This means that training with a protograph-assisted approach led the KGEMs to generate higher-quality embeddings in terms of class separability, especially for entities in CC and CCB that are harder to distinguish. With the sole exception of TuckER whose performance may be due to non-optimal hyperparameter tuning, strong improvements are achieved w.r.t. the EC task on all datasets. Interestingly, not only V is outperformed by either P1 or P2 in each scenario, but actually by both of them at the same time. As a result, our schema-based, protograph-assisted learning approach MASCHInE proves effective for EC, regardless of the protograph design principle.

Node Classification

Following on from the previous clustering experiment that clearly showed the benefit of incorporating pre-trained protograph embeddings, in this section we are interested in performing node classification with such embeddings. In this experiment, entities are labelled on the basis of whether they comply with some description logic (DL) constructors.

Datasets. In this experiment, we use the synthetic DLCC node classification benchmark [START_REF] Portisch | The DLCC Node Classification Benchmark for Analyzing Knowledge Graph Embeddings[END_REF]. The DLCC benchmark helps analyze the quality of KGEs by evaluating them w.r.t. which kinds of DL constructors they can represent. We use the synthetic gold standard datasets 9 . These are 12 synthetic datasets featuring labelled entities to classify and underpinned by a schema defining entity types, relation domains and ranges. The synthetic part of DLCC has been chosen because the authors argue that those pose classification problems which are not influenced by any side effects and correlations in the knowledge graph.

Implementation details. For these 12 test cases, their respective protographs are built. Then, the KGEMs are trained w.r.t. the LP task under the three different settings V, P1, and P2. In line with [START_REF] Portisch | The DLCC Node Classification Benchmark for Analyzing Knowledge Graph Embeddings[END_REF], multiple classifiers were trained for each test case: decision trees (DT), naive bayes (NB), k-nearest neighbors (k-NN), support vector machine (SVM), and multilayer perceptron (MLP). Based on the best entity embeddings learnt during LP, these models are used to classify using the default scikit-learn parameters.

Evaluation metrics. NC performance is measured using F-score, which is the harmonic mean of precision and recall. Lowest and highest scores are 0 and 1, respectively. For each model and test case, results achieved with the best classifier are reported in Table 4.

Experimental results. First, not all of the gold standard test cases are equally hard for the task of NC. For instance, entities in tc01, tc05, and tc06 are obviously easier to label than in the other datasets. Particularly, the DLCC synthetic gold standard datasets are built on the basis of different DL constructors in order to analyze which KG constructs are more easily recognized by KGEMs.

For the less trivial cases, the gains that can be achieved by using protographs are more significant. Especially in cases which are fairly badly solved by vanilla embeddings (e.g., tc07, tc12), there are gains of more than 10 percentage points in F1. It is further remarkable that on this problem, compared to link prediction and entity clustering, TuckER can also benefit from the protographs.

When looking at which protograph construction strategy is more suitable for which kind of problem, we see that for those test cases which use a concrete class in their definition (particularly tc07, tc08, tc11, and tc12), P2 is clearly superior to P1. This may be due to the fact that P1 does not create class embeddings for all classes used in the respective constructors. Therefore, the resulting entity embeddings are less discriminating.

CONCLUSION AND FUTURE WORK

In this paper, we present MASCHInE -a novel model-agnostic and protograph-assisted approach for learning KGEs. MASCHInE first trains embeddings on protographs and then transfers these embeddings to learn the final KGEs. We have shown that this approach produces more versatile KGEs, which can yield significant improvements in entity clustering and node classification. This is particularly noticeable for entity clustering, as protographs encode information about classes, on which entity clustering heavily relies on. We also demonstrated the advantage of using MASCHInE for link prediction: the use of protographs in the training procedure leads KGEMs to make more semantically valid predictions, which clearly highlights that geometric distance in the embedding space can be influenced by the semantics of the underlying KG.

In future work, we will extend MASCHInE by adding new protograph design principles. While in the current form, the transfer from the RDFS-based protograph embeddings to the KG entity embeddings is done once, we will study more expressive ontological constructs (e.g., OWL) and alternatives for KG and protograph iterative co-training. Additionally, it would be worth exploring whether pre-training over protographs provides benefits compared to passing the relevant RDFS triples to the KGEM as part of the whole graph. Comparing our approach with KGEMs that take advantage of additional information in the KG -e.g. attributes -also deserves further exploration.

Figure 1 :

 1 Figure 1: Excerpt from DBpedia class hierarchy (left) and the generated prototriples (right) according to P2. The full URI names have been shortened due to space limitations. The prototriple highlighted in green (right) comes from the schema, whereas the following triples are generated by fixing the domain (resp. range) of dbo:headquarter and replacing the class in the range (resp. domain) with each of its direct subclasses.

Figure 2 :

 2 Figure 2: Overview of MASCHInE. The overall training approach is based on several modules that each features different possibilities. First, the protograph is built following predefined design principles (a). Protograph embeddings are learnt (b) before being transferred to the corresponding entities and relations of the KG (c). Next, training starts on the KG (d). During this step, embeddings learnt on the protograph can still interact in the fine-tuning procedure (purple dotted arrow). Finally, the fine-tuned KGEs can be used in several tasks.

Figure 3 :

 3 Figure 3: Entity clustering results on YAGO14k, FB15k187, and DBpedia77k.

Figure 4 :

 4 Figure 4: PCA visualizations of YAGO14k entity embeddings using the first two principal components. Each point represents an entity, whose color denotes its ground-truth class.

Table 1 :

 1 KG and protograph characteristics. Column headers from left to right: number of entities, relations, and triples.

	Dataset		| E |	| R |	| T |
		KG 14, 178	37	19, 183
	YAGO14k	P1	22	37	37
		P2	590	37	4, 959
		KG 14, 305 187 278, 436
	FB15k187	P1	138 187	187
		P2	138 187	187
		KG 76, 651 150 190, 028
	DBpedia77k	P1	55 150	150
		P2	186 150	3, 210

Table 1

 1

provides statistics for

  this work, we stick with the loss function originally used for each KGEM. Evaluation is performed every 10 epochs and the best model w.r.t. MRR on the validation set is saved for performance assessment on the test set. The aforementioned procedure prevails for the vanilla training mode. For P1 and P2, we use the same procedure as for training on the KG, except that 200 epochs of training are previously performed on the protograph, for a total of 600 training epochs.

	Experimental results. LP results are reported in Table 2. On
	YAGO14k and DBpedia77k, learning embeddings with P1 or P2 pro-
	vides better Sem@𝐾 values compared to V. No significant change is
	noticed on FB15k187. This is likely to be an artefact of the Freebase
	class hierarchy and how entities are typed in FB15k187: the class hi-
	erarchy only has 2 levels, all relation domains and ranges are level-2
	classes (therefore P1 and P2 are the same, see Table

Table 3 :

 3 Entity clustering results on the GEval datasets. Bold font indicates which setting performs the best for a given model and dataset, while underlined results correspond to the second best setting.

				CC					CCB					CAMAP
			ARI	AMI VM	FM	H	C	ARI	AMI VM	FM	H	C	ARI	AMI VM	FM	H	C
		V	0.179 0.140 0.141 0.596 0.143 0.139	0.183 0.147 0.147 0.649 0.164 0.133	0.564 0.783 0.785 0.736 0.951 0.668
	TransE	P1	0.621 0.514 0.515 0.823 0.505 0.526	0.288 0.311 0.311 0.692 0.349 0.280	0.593 0.795 0.796 0.756 0.956 0.682
		P2	0.680 0.570 0.571 0.849 0.563 0.579	0.299 0.314 0.314 0.698 0.353 0.284	0.646 0.815 0.816 0.791 0.964 0.708
		V	0.018 0.024 0.027 0.709 0.015 0.095 -0.024 0.009 0.010 0.770 0.006 0.028 -0.054 0.099 0.106 0.478 0.087 0.136
	DistMult	P1	0.891 0.817 0.817 0.948 0.813 0.821	0.762 0.673 0.673 0.907 0.708 0.642	0.655 0.705 0.707 0.798 0.782 0.646
		P2	0.891 0.824 0.824 0.948 0.818 0.830	0.756 0.626 0.627 0.907 0.646 0.608	0.933 0.872 0.872 0.963 0.886 0.859
		V	-0.011 0.009 0.011 0.694 0.007 0.033 -0.024 0.009 0.010 0.771 0.006 0.028	0.139 0.345 0.350 0.520 0.341 0.360
	ComplEx	P1	0.897 0.827 0.828 0.951 0.823 0.832	0.619 0.560 0.560 0.845 0.606 0.521	0.969 0.916 0.917 0.983 0.939 0.896
		P2	0.319 0.264 0.265 0.665 0.269 0.262	0.493 0.460 0.460 0.788 0.507 0.422	0.961 0.914 0.914 0.978 0.938 0.892
		V	0.311 0.234 0.235 0.691 0.223 0.249	0.271 0.242 0.243 0.687 0.271 0.220	0.591 0.608 0.611 0.76 0.672 0.559
	ConvE	P1	0.878 0.805 0.805 0.942 0.799 0.811	0.719 0.628 0.628 0.889 0.665 0.596	0.934 0.880 0.881 0.964 0.928 0.839
		P2	0.916 0.848 0.848 0.960 0.847 0.849	0.790 0.697 0.697 0.919 0.727 0.669	0.954 0.915 0.915 0.975 0.968 0.868
		V	0.179 0.138 0.139 0.596 0.141 0.137	0.158 0.120 0.121 0.639 0.135 0.109	0.396 0.526 0.529 0.618 0.636 0.453
	TuckER	P1	0.069 0.051 0.052 0.543 0.053 0.051	0.137 0.094 0.094 0.631 0.105 0.086	0.376 0.635 0.637 0.599 0.795 0.531
		P2	0.047 0.042 0.043 0.533 0.043 0.042	0.033 0.051 0.052 0.578 0.058 0.046	0.276 0.487 0.490 0.520 0.621 0.405

Table 4 :

 4 Node classification results on the 12 DLCC test cases. Listed are the results of the best classifier for each combination of model and test case. Bold fonts indicate which setting performs the best for a given model and test case. Underlined results show which combination of model and setting performs the best for each test case.

	Test Case Setting TransE DistMult ComplEx ConvE TuckER Test Case Setting TransE DistMult ComplEx ConvE TuckER
		V	0.702	0.810	0.820	0.885	0.832		V	0.654	0.564	0.566	0.732	0.669
	tc01	P1	0.647	0.754	0.784	0.875	0.857	tc02	P1	0.566	0.554	0.554	0.764	0.637
		P2	0.637	0.794	0.769	0.852	0.792		P2	0.571	0.627	0.627	0.687	0.679
		V	0.584	0.571	0.554	0.629	0.669		V	0.638	0.540	0.512	0.818	0.570
	tc03	P1	0.589	0.541	0.519	0.699	0.664	tc04	P1	0.760	0.812	0.832	0.870	0.552
		P2	0.521	0.544	0.526	0.622	0.556		P2	0.805	0.802	0.818	0.818	0.680
		V	0.704	0.683	0.678	0.822	0.814		V	0.645	0.940	0.938	0.940	0.950
	tc05	P1	0.887	0.937	0.940	0.940	0.814	tc06	P1	0.718	0.875	0.898	0.935	0.950
		P2	0.937	0.940	0.940	0.940	0.902		P2	0.740	0.828	0.788	0.895	0.942
		V	0.578	0.515	0.552	0.560	0.522		V	0.552	0.552	0.520	0.598	0.575
	tc07	P1	0.505	0.540	0.560	0.560	0.552	tc08	P1	0.585	0.560	0.558	0.575	0.580
		P2	0.582	0.605	0.655	0.675	0.648		P2	0.595	0.588	0.588	0.578	0.672
		V	0.552	0.522	0.525	0.730	0.745		V	0.578	0.572	0.562	0.640	0.690
	tc09	P1	0.560	0.538	0.552	0.745	0.780	tc10	P1	0.598	0.550	0.538	0.625	0.692
		P2	0.550	0.615	0.598	0.672	0.742		P2	0.538	0.510	0.535	0.590	0.588
		V	0.632	0.518	0.535	0.688	0.700		V	0.650	0.538	0.552	0.702	0.708
	tc11	P1	0.628	0.542	0.522	0.618	0.685	tc12	P1	0.572	0.530	0.565	0.708	0.700
		P2	0.630	0.552	0.558	0.638	0.678		P2	0.625	0.802	0.738	0.802	0.752

Note that 𝐶 𝑖 and 𝐶 𝑗 might be the same, so there can be cycles in the protograph.

In the KGs used in this paper, all relations have an explicit domain and range.

In our experiments, the same KGEM is used for both pre-training over the protograph and fine-tuning embeddings on the KG.

https://github.com/nicolas-hbt/versatile-embeddings

https://scikit-learn.org/stable/index.html

https://github.com/mariaangelapellegrino/Evaluation-Framework

In DBpedia77k, 11, 586 entities belong to the dbo:Place class.

https://github.com/janothan/DL-TC-Generator


TOWARD LOW EARTH ORBIT (LEO) APPLICATIONS: THE SCIENTIFIC JOURNEY OF THE "SPACE PULSATING HEAT PIPE" EXPERIMENTS

Keywords: Pulsating Heat Pipe, Microgravity, Thermal Management, Low Earth Orbit, Two-phase flows, Infrared Analysis

This paper shortly summarises the experimental results obtained since 2011 by a large European academic consortium for the scientific conceptualisation, the definition of the technical requirements, the generation of experimental data, and the validation of a numerical code, for the Pulsating Heat Pipes (PHP) experiment on the International Space Station (ISS). The PHP is a passive, wickless thermal device, whereby a twophase fluid, forming liquid plugs and vapour slugs, moves with a pulsating or circulating motion inside a meandering tube or channel. The PHP may have a very broad range of geometries (flat, tubular, 3D structured), it can dissipate heat from large areas, and it can be suitable for high power applications with low/medium heat fluxes. PHP functioning is based on the capillary effect, which provides the existence of liquid plugs completely filling the channel cross-section, in a way that any expansion or contraction of the vapour slugs will naturally generate a movement of the fluid along the channel axis. For this, it is important that the channel has a cross-section size below a given threshold, which depends on the liquid surface tension and (for a static fluid) on the gravity acceleration. In space, when only residual accelerations are acting, such a static size threshold is virtually infinite, while a finite dynamic threshold exists even in the absence of gravity. The concept of a "Space PHP" was originally developed in 2014 by the team, and from then 17 Parabolic Flight Campaigns (PFC) and 3 Sounding Rocket (SR) experiments have been carried out to generate the data for the preparation of an experiment targeting a Low Earth Orbit (LEO) mission. Both a tubular and a flat plate PHP have been successfully tested in reduced gravity and on ground, by using different combinations of fluids and building materials. The need for having an experiment on a LEO environment is mainly because, during a PFC, only 22sec of reduced gravity are possible, which is a period below the characteristic time for reaching a steady state condition for almost all of the tested devices.

INTRODUCTION

Two-phase passive heat transfer devices have an important role in space thermal control systems, still gaining more and more relevancy because of their lightweight, high performances and reliability. When heat dissipation becomes very high, heat pipes are preferred to honeycomb structures in radiators panels and they have been widely used to reduce temperature gradients. The Capillary Pumped Loops (CPL) and Loop Heat Pipes (LHP) have been successfully used as thermal control systems in a large number of space missions because of their ability to cover tortuous and longer paths. Pulsating heat pipes (PHP) have been widely studied in the last 15 years, even if their concept was presented already 30 years ago by Akachi [START_REF] Akachi | Structure of a heat pipe[END_REF]. In particular, since more than 10 years, the authors have investigated the use of PHPs for space application, i.e. in a microgravity environment. In particular, they developed the new concept of a "Space PHP" which profits of the absence or a lower gravity acceleration to achieve higher performances in terms of higher heat fluxes, higher thermal power loads and lower thermal resistances. A PHP is made of a wickless capillary channel or tube bent in several turns between a heated and a cold zone. The tube is partially filled with a working fluid at the liquid/vapour saturation state, which is internally distributed by means of surface tension effects in the form of liquid slugs and vapour plugs. At the evaporator side, under a given heat flux, liquid film evaporation or flow boiling occur. At the condenser side, vapour condense toward the liquid film on the channels' walls. Flow patterns ranging from slug flow to annular flow are initiated in adjacent tubes by local pressure instabilities, and they effect the global heat flux transferred from the heated to the cooled region. When the flow patterns are considered in conjunction with related heat transfers, many parameters have a direct influence on PHP operation; while the most important is obviously the channel internal diameter that allows liquid/vapour phase division into liquid slugs and vapour plugs separated by menisci, other parameters are the number of turns, PHP length, filling ratio, the physical properties of the working fluid or, indirectly, the cold source temperature, heat power applied or heat flux, the inclination with respect to gravity and the closure, or not, of the overall loop. We address the reader to our reviews of the field for more information about the PHP principles and operations [START_REF] Nikolayev | Pulsating heat pipes: basics of functioning and numerical modelling[END_REF][START_REF] Marengo | Pulsating heat pipes: experimental analysis, design and applications[END_REF]. A PHP is working on the principle that a liquid slug fills completely the channel when its hydraulic diameter is below a critical capillary diameter. If we are not considering dynamic effects, a tube can be considered "capillary" if the surface tension forces overcome the gravity forces, maintaining the internal liquid in a confined mode. This concept can be expressed using the dimensionless number Bond 𝐵𝑜 = 𝜌𝑔𝐷 2 𝜎 ⁄ : When the Bond number is below one, the surface tension dominates, and therefore we are in the conditions written above, i.e. the liquid is not stratified and fills the channel. As we will see afterwards [START_REF] Pietrasanta | Developing flow pattern maps for accelerated two-phase capillary flows[END_REF], the critical threshold is not exactly equal to the unity, and varies with the liquid slug acceleration, wall wettabilities and other parameters. It is evident that, when the gravity acceleration is very small, the Bo number tends to be very small, i.e. the critical diameter tends to increase to very high values. In space, even big channels are capillary, in the sense In 2003, also Gu et al. [START_REF] Gu | Effects of gravity on the performances of pulsating heat pipes[END_REF] experimented two flat plate aluminium PHPs (250 x 60 x 2.2 mm 3 ) in PFC each with three different configurations. Each PHP was filled with R114 (FR 50%). A continuous channel, engraved on the bottom plate, had a cross section of 1x1 mm 2 and was laid out with 48 turns at both ends of the closed loop. The evaporator consisted of a 55x16 mm 2 Kapton heater, and for the condenser a fan of 80x80 mm 2 was used: in one configuration the evaporator is at the centre of the PHP, while for the other two, the heater is at the bottom (BHM) and at the top (THM), respectively. The authors observed that both PHPs operated more effectively in all the configurations under reduced gravity than under normal or hypergravity phases. Based on a dimensional analysis, these authors were the first to consider the use of larger channel diameters for space applications.

At the moment, two experiments were able to carry out a long duration test of a PHP in space, one from Japan Aerospace Exploration Agency (JAXA) in cooperation with Tohoku University in Japan, and the other one from the Air Force Research Laboratory partnered with Thermavant Technologies and Maxar Technologies (USA). Both experiments are with a standard capillary size. The on-orbit experiment of JAXA considered a stainless steel 5 turns 2D TPHP with Dint=0.8 mm, filled at 45% with HFC-134a, and has been conducted since 2012 as one of the Small Demonstration Satellite-4 (SDS-4) missions [START_REF] Ando | Onorbit demonstration of oscillating heat pipe with check valves for space application[END_REF]. The particularity of this experiment has been the use of a check-valve to promote the flow circulation. The US experiment intended to demonstrate the functionality and the limits of operation of a PHP [START_REF] Taft | ASETS-II oscillating heat pipe space flight experiment: the first six months on orbit[END_REF]. The experiment was installed on the 5 th flight of the Air Force's X-37B spaceplane and lasted 780 days until Oct 2019. The Advanced Structurally Embedded Thermal Spreader (ASETS-II) flight experiment consisted in three 2D TPHP (Dint = 1mm, 34 channels) with different varying configurations (centre heating with large and small heaters and with single-and double-ended cooling), using butane and R-134a. During six-week continuous operation test, the PHPs performed as expected with negligible differences between horizontal and microgravity flight data. As written above, both Gu et al. [START_REF] Gu | Effects of gravity on the performances of pulsating heat pipes[END_REF] and our research group [START_REF] Mameli | Thermal response of a closed loop pulsating heat pipe under a varying gravity force[END_REF][START_REF] Mangini | Hybrid Pulsating Heat Pipe for Space Applications with Non-Uniform Heating Patterns: Ground and Microgravity Experiments[END_REF] illustrated the possibility to build a PHP for space application with an internal diameter bigger than the static critical diameter on ground. Under reduced gravity, body forces are negligible and the threshold diameter to obtain a slug-plug configuration increases. Since the mass of the thermal fluid per unit length is proportional to the square of the Dint, increasing the inner diameter is also beneficial in terms of total heat exchanged. Theoretically, for negligible gravity accelerations, the capillary diameter tends to infinite, however, the limit to the increase of the inner diameter is also given by inertial and viscous effects. The whole work has been always related to the concept of confined flow, i.e. the condition of having the liquid slugs filling completely the channels, in order to trap the vapour, which is acting as a thermal pump during evaporation and condensation. Therefore, the classical use of the Bond Number cannot work for microgravity conditions, since the Bond number is always zero for any channel size. The inertial and viscous terms are becoming very important with respect to the capillarity phenomena, and authors in literature used dimensionless number such as Weber and Reynolds.

In two papers of the research group [START_REF] Pietrasanta | Developing flow pattern maps for accelerated two-phase capillary flows[END_REF][START_REF] Loyola-Fuentes | Machine Learning Algorithms for Flow Pattern Classification in Pulsating Heat Pipes[END_REF], the problem has been deeply analysed through the data collected on ground and in microgravity, where the gravity acceleration should not be considered inside the correlations. It resulted that the use of the actual bubble acceleration and the bubble length, leading to the constructions of modified dimensionless numbers, is more efficient in distinguishing the various PHP regimes. Despite the fact that it is not possible to use such criteria for the design of the PHP, since bubble length and acceleration are not known a priori, those limits can be implemented in lumped numerical codes for the simulation of Pulsating Heat Pipes [START_REF] Taft | ASETS-II oscillating heat pipe space flight experiment: the first six months on orbit[END_REF] or in VOF-CFD models, to eliminate the empirical assumptions on the existence of a slug-plug flow pattern.

IHTC-17 | ID: 228

TOWARD THE SPACE PULSATING HEAT PIPE

This paper summarises some of the results obtained by the authors in the last 10 years on the development of the concept and characterisation of the so-called Space Pulsating Heat Pipe. In Figure 1, it is possible to visualise all the microgravity experiments carried out until now by the research group on this technology.

The Planar Tubular Pulsating Heat Pipe

A first hybrid Thermosyphon/Pulsating Heat Pipe with a diameter bigger than the terrestrial capillary limit was tested both on ground and in hyper/micro gravity conditions during the 61st ESA PFC in 2014 [START_REF] Mangini | A Pulsating Heat Pipe For Space Applications: Ground and Microgravity Experiments[END_REF]. During each of the 3 flight days, 31 parabolic trajectories are performed: the first one, called parabola zero, is followed by six sequences of five consecutive parabolae. In between each sequence, there are five minutes of steady flight at normal gravity level, used here to vary the heat power level and let the system reach a steady state when possible. Each parabola is then composed by a first hypergravity period (20s at 1.8g), the micro-gravity period (20s at 0g) and a second hyper-gravity period equal to the first. This first device consisted in an aluminum tube (Dint = 3 mm) bent into a planar serpentine with five curves at the evaporator zone, while a transparent section closes the loop, allowing fluid flow visualizations in the condenser zone (see Fig. 2). Five heaters, mounted alternatively in the branches just above the curves at the evaporator zone, provide a possible asymmetrical heating thus promoting the fluid flow circulation in a preferential direction [START_REF] Mangini | Hybrid Pulsating Heat Pipe for Space Applications with Non-Uniform Heating Patterns: Ground and Microgravity Experiments[END_REF]. On ground, the device is first vacuumed by means of an ultra-high vacuum system down to 0.3 mPa, and then it is partially filled up with the working fluid (FC-72) with a volumetric ratio of 50% ± 3% and sealed by means of tin soldering. The fluid itself is previously degassed within a secondary tank, by continuous boiling and vacuuming cycles. Presence of NCG in the system was detectable only indirectly through verification of the pressure value, which should correspond to the saturation pressure at the given temperature. This kind of procedure is common to all the tests, and it will be not repeated anymore in the paper. The device has been tested at different positions (vertical and horizontal) and at different heat power input levels (from 10 W to 160 W). Ground tests showed that this device works as a thermosyphon when gravity assisted: in vertical position the device can reach an equivalent thermal resistance of 0.1 K/W with heat fluxes up to 17 W/cm 2 . In horizontal position the fluid motion is absent, thus the device works as a pure thermal conductive medium. In Fig. 3a and3b, it is evident that the hybrid PHP is working better in vertical BHM position, achieving 180W of heat power, still with less than 70ºC for the evaporator temperature. The PHP is still working in the horizontal position, and it is possible to verify that the thermal performance at microgravity resembles the ones at microgravity (Fig. 3b &c), and that in microgravity the liquid slugs are filling completely the tube as expected (Fig. 3d). Pseudo-steady state conditions can be reached in approximately 3 min, due to the low thermal inertia of the heating system and the device: In such a sense, during the parabolas the system never reached its steady state. In conclusion, these parabolic flight tests reveal that the device operates in a completely different way when the microgravity is reached: the images In some cases, the hyper-gravity period is able to eliminate partial dry-outs restoring the correct operation until the occurrence of the next microgravity period. A following experiment on the same kind of hybrid plan tubular PHP has been carried out in 2015 (63rd ESA PFC) considering the idea of different heating zones [START_REF] Mangini | Hybrid Pulsating Heat Pipe for Space Applications with Non-Uniform Heating Patterns: Ground and Microgravity Experiments[END_REF]. In most of the space applications, the heating can be localised in given positions linked to presence of CPU, IGBT and other electronic components. Moreover, it is possible that the switch on of heat fluxes with different patterns may facilitate the working operation of the PHP in circulation mode, increasing the thermal performances. For this experiment, the bubble velocity was estimated with an open source Particle Image Velocimetry (PIV) software partially modified in order to detect the liquid/vapor interface motion along the transparent section. The bubble velocity is calculated utilizing the OpenPIV software. With respect to the hybrid planar PHP with uniform heating, when the device is heated in a non-uniform way, a circulation in a preferential direction occurs, with an improvement of the overall thermal performance, especially for the lower heating power tested. In microgravity conditions, the sudden absence of the buoyancy force permits to activate a typical PHP oscillating slug/plug flow. Non-uniform heating configurations have a beneficial impact also in microgravity. Increasing the heating power level at the most lateral branches and decreasing it at the centre, the stop-over periods observed in microgravity for the uniform heating, are limited. In this way, the possibility to stabilize the two-phase flow motion in weightlessness conditions was proved through different heating configurations at the evaporator.

IHTC-17 | ID: 228

The Staggered 3D Tubular Pulsating Heat Pipe

A completely new device has been proposed in Nov 2017 for the 67th ESA PFC. It is made of an annealed aluminum (6060 alloy) tube with an inner and outer diameter of 3 and 5 mm, respectively. The closed loop is folded in a staggered 3D configuration with 14 turns (8 mm bending radius) in the evaporator zone, as shown in Fig. 4. The geometry (overall size of the PHP: 220 × 80 × 25 mm) was selected to fit this device in a Heat Transfer Host designed for the experiment on the International Space Station. Two brass connections allow to connect a transparent sapphire tube with the aluminum tube, and to host two K-type micro-thermocouples for the fluid temperature measurement (red and blue "F" bullet in Fig. 4b), as well as another miniature pressure transducer close to the evaporator section (green "P" bullet in Fig. 4b). Sapphire is chosen for its optical properties, being almost transparent also in the MWIR spectrum The heating power is provided by a programmable power supply from a minimum of 18 W to 182 W (1.10 to 11.43 W/cm 2 ). The condenser zone is embedded between two aluminium heat spreaders cooled down by means of an array of 8 Peltier cells (82.2 W each). A single-phase auxiliary cooling loop has been used to dissipate the waste heat of the Peltier cells, coupling the hot side of each element with a cold plate. Five T-type thermocouples are located between the evaporator spreader and the heater; six are located between the Peltier cold side and the condenser aluminium heat spreader; two on the condenser heat spreader just behind the sapphire tube; seven are located on the tube external wall (Fig. 4a). The PHP is filled with FC-72 (FR 50%). In comparison with a previous experiment with a smaller number of heated sections [START_REF] Mangini | A Pulsating Heat Pipe For Space Applications: Ground and Microgravity Experiments[END_REF][START_REF] Mangini | Hybrid Pulsating Heat Pipe for Space Applications with Non-Uniform Heating Patterns: Ground and Microgravity Experiments[END_REF], the absence of stop-over periods is beneficial in terms of heat transfer rate. Indeed, for the same wall-to-fluid heat input flux (the effective heating area passed from Aeff = 9.4cm 2 to 15.8cm 2 ) this 3D PHP is characterized by smaller transient temperature differences for all the heating powers. Start-up tests, where the heat load is provided just after the occurrence of microgravity, show that the 20s microgravity are enough for the device activation: the PHP operation is not primed by the flow inertial effect, still present when the device is activated before the

The Flat Plate Pulsating Heat Pipe

Another kind of PHP tested by the research group was the so-called Closed Loop Flat Plate Pulsating Heat Pipe (FPPHP) (Fig. 5) tested on ground and on board of an aircraft during the 60th ESA PFC in 2014 [START_REF] Ayel | Experimental study of a closed loop flat plate pulsating heat pipe under a varying gravity force[END_REF].

The purpose was to test an innovative FPPHP, which behaves differently from tubular PHPs due to two main characteristics: (i) the square/rectangular channels with sharp angles and (ii) the transverse heat spreading throughout the continuity of the plate bulk. The FPPHP consists of two brazed copper plates, into one of which a continuous rectangular channel (1.6 x1.7 mm 2 ) with 12 turns in the evaporator, is machined. The channel is filled with FC-72 as working fluid (FR 50%). Tests have been conducted with the FPPHP positioned both horizontally and vertically in BHM. The FPPHP presented an innovative design, involving the milling of grooves between the channels to avoid the heat spreading across the channel (Fig. 5b). The PHP and the heat sink are clamped against one another with throughbolts traversing nine screw holes. Experimental results on the ground show that the thermal device can transfer more than 180 W in both inclinations, and that the horizontal operation is characterized by repeated stopand-start phases and lower thermal performance. The FPPHP can operate under microgravity conditions and with a transient gravity force, with global thermal resistance reaching 50% and 25% of that of the empty plate, in horizontal and vertical orientation respectively. A minimum thermal resistance of 0.12K/W was recorded. Compared to the tubular PHP tested by Mameli et al. [START_REF] Mangini | A Pulsating Heat Pipe For Space Applications: Ground and Microgravity Experiments[END_REF][START_REF] Mangini | Hybrid Pulsating Heat Pipe for Space Applications with Non-Uniform Heating Patterns: Ground and Microgravity Experiments[END_REF], this FPPHP has proved to be quicker to respond to gravity variations in horizontal inclination, mainly on account of the higher hydraulic diameter of its channels. The FPPHP tested in horizontal inclination is not influenced by changes of gravity levels, proving that such a system can be an efficient candidate for thermal control in different space applications, once a greater channel size is combined with a sufficient number of turns. Another FPPHP, again filled with FC72, was tested during the 62 th ESA PFC under vertical orientation in 2015 [START_REF] Ayel | Visualization of Flow Patterns in Closed Loop Flat Plate Pulsating Heat Pipe Acting as Hybrid Thermosyphons under Various Gravity Levels[END_REF]. The FPPHP is made of a thin copper plate, in which a curved channel disposed with 11 U-turns is milled and closed on the top face by a transparent borosilicate plate. The particular characteristics of this FPHP is that the equivalent hydraulic diameter of the square channel (2.5×2.5 mm 2 ) is above the working fluid capillary diameter on ground. On ground, the heat transfer mode is represented either by pure pool boiling or by an annular flow pattern inside the channels mostly filled by the refrigerant vapor. Instead, during microgravity, the fluid regime naturally turns into a slug-plug flow pattern. During the transition from 1.8 g to 0 g a rapid dry-out may occur in some of the channels, followed by a similarly fast reaction of liquid plugs moving towards the evaporator from the condenser zone. Such stop-and-start motion events continue during the whole microgravity period, leading to strong temperature oscillations, but also to a still acceptable thermal performance of the device. The same FPPHP was tested during the 65 th ESA PFC under vertical orientation in 2016 [START_REF] Cecere | Experimental analysis of a Flat Plate Pulsating Heat Pipe with Self-ReWetting Fluids during a parabolic flight campaign[END_REF], but filled with pure water and a self-rewetting mixture (water/butanol), whereby for the last one, the surface tension increases with temperature in a small range around 60ºC, which may lead to a rewetting of the heated zones. Indeed, for low power input, the FPPHP filled with pure water was not able to work under low-g conditions, because dry-out condition appears in some channel, while the FPPHP filled with SRWF was able to operate. Furthermore, its thermal resistance was lower [START_REF] Cecere | Experimental analysis of a Flat Plate Pulsating Heat Pipe with Self-ReWetting Fluids during a parabolic flight campaign[END_REF]. Four similar prototypes of FPPHP were also tested during 64 th ESA PFC in 2016 [START_REF] Ayel | Thermo-hydraulic analysis of semi-transparent flat plate pulsating heat pipes tested in 1g and microgravity conditions[END_REF], with square channel dimensions ranging from 1.5 mm to 3 mm. It was observed that thermal performances increase with increasing diameter, both in vertical BHM and microgravity transient phases. The larger the channel diameter, the higher the number and cumulated time of reactivation or active phases, up to a limit that seemed to be situated between 2.5 mm and 3 mm of diameter. This suggests that there exists an optimal hydraulic diameter in such a configuration beyond which performance will not necessarily increase, due to higher inertial forces and/or the lower capillary forces necessary to maintain the menisci in slug/plug distribution.

DATA ANALYSIS

Infrared Analysis

The infrared analysis is essential for the characterisation of a PHP, not only because it is allowing to have a measure of the metal surface temperature or, in case of the sapphire insert, of the fluid inside the system, but because it is opening the route for the estimation of the heat transfer coefficients, which are very important for the tuning of numerical simulations.

There are rare research works on the evaluation of internal flow heat transfer in presence of an oscillation. The research group has first tested the use of the inverse heat conduction problem solution on a Single Loop Pulsating Heat Pipe (Fig. 6) partially filled with ethanol (FR 60%) and designed with two sapphire tubes as a connection between the evaporator and the condenser [START_REF] Cattani | An Original Look into Pulsating Heat Pipes: Inverse Heat Conduction Approach for Assessing the Thermal Behaviour[END_REF]. The local heat flux was estimated for the flow patterns usually observed during the PHP operation: annular flow and oscillating slug/plug flow. In the region between the evaporator and the condenser, the local wall-to fluid heat flux can reach values of up to 4000 W/m 2 in the case of annular flow and 2000W/m 2 in the case of an oscillating slug/plug flow. The same technique has been used for the Staggered 3D Tubular Pulsating Heat Pipe [START_REF] Mameli | Start-up in microgravity and local thermodynamic states of a hybrid loop thermosyphon/pulsating heat pipe[END_REF][START_REF] Pagliarini | Thermal characterization of a multi-turn pulsating heat pipe in microgravity conditions: Statistical approach to the local wall-to-fluid heat flux[END_REF] during the PFC67 in 2017.Two regimes were identified: intermittent flow (sporadic fluid motion, occurring in some channels) and full activation (stable fluid motion within the whole adiabatic section). During the intermittent flow regime, only some channels present fluid oscillations mainly in the frequency range 0.5-0.6 Hz; on the contrary, for high power inputs to the evaporator, the dominant frequency is generally similar in the whole adiabatic section and it is around 1.0 Hz, even though the oscillatory behaviour may still vary from channel to channel. This dominant frequency values are in good agreement with the ones estimated with pressure sensors by Perna et al. [START_REF] Perna | Flow characterization of a pulsating heat pipe through the wavelet analysis of pressure signals[END_REF]. The 80th percentile of the wall-to-fluid heat flux amplitude is almost constant (about 500 W/m 2 ) at low power inputs. From 100 W up to the maximum power input, the heat flux amplitude increases almost linearly with the power input, reaching about 1500 W/m 2 at 202 W.

Wavelet analysis

The research group has also used the wavelet transform to characterize the fluid pressure signal in the frequency domain varying the heat power input at the evaporator and in the condenser zone of a full-scale PHP tested in microgravity conditions in the 67th ESA PFC [START_REF] Perna | Flow characterization of a pulsating heat pipe through the wavelet analysis of pressure signals[END_REF]. During the slug-plug flow regime, the dominant frequencies falls in the range 0.6-0.9 Hz, showing an increasing trend with the heat load input. The Cross-Correlation reveals that the two signals at the evaporator and at the condenser are very similar.

CONCLUSIONS

Finally, the long journey of this collective work has proven that the concept of a Space PHP is viable for space applications, both in the Tubular and in the Flat Plate configuration, with a clear indication that more than 10 W/cm 2, less than 0.1K/W thermal resistance, more than 450 W/kg, can be achieved with a total power that reached 200W in some configurations. The systems have low start-up powers (from 30W), fast (below 20s reaction time, depending on thermal mass), and they are very reliable in terms of duration (until now a TPHP worked for longer than 1 year in laboratories of Pisa and Poitiers). It is also possible to design a specific Space PHP using a validated lumped parameter code (CASCO by CEA/CNRS). The team is now looking forward to being able to carry out a long-duration experiment in space in the next few years, as it has already been done for the capillary PHP by JAXA and AFRL teams. This work is also extremely important for the future improvement of the status-of-art PHP simulation code CASCO (French abbreviation for "Code avancé de simulation de caloduc oscillant"). In a very recent paper [START_REF] Abela | Experimental analysis and transient numerical simulation of a large diameter pulsating heat pipe in microgravity conditions[END_REF], CASCO code prediction ability was validated against the experimental data collected during the micro-gravity tests of 3D Staggered TPHP (67th ESA PFC). The advanced 1D Lumped parameter network code was able to predict the temperature temporal evolution of evaporator and tube walls with a maximum deviation of 7%, the pressure trends were reproduced, like the start-up time.

Fig. 1 A

 1 Fig. 1 A graphical representation of the most important research activities carried out by the research team in micro-, terrestrial and hyper-gravity from 2013. In this paper we will describe results obtained only in 7 campaigns, leaving the reader to consider the appropriate literature for more details.

Fig. 2

 2 Fig.2The first Planar Tubular Pulsating Heat Pipe with 5 turns analysed in 2014, with 5 independent heaters.

Fig. 3 A

 3 Fig. 3 A standard visualization of the temperature measurements and the fluid flow for the 2D Planar TPHP. In (a) and (b) the ground tests, in (c) and (d) one exemplary parabola. The lines with the different colours in (a) and (c) represent the values from the 14 thermocouples at the evaporator (reddish) and condenser (bluish and greenish). The hypergravity and microgravity zones are recognizable by looking at the gravity acceleration measurements (line in black in (c) and (d). In (d) the values of the fluid pressure are synchronised with the accelerometer and the visualizations of the high speed grey scale camera.

Fig. 4

 4 Fig. 4 The 3D staggered TPHP: thermocouples and pressure transducer locations: (a) the CAD exploded view, (b) the actual front view.

  period. The simultaneous measurement of the internal fluid pressure and temperature has been successfully implemented showing the existence of subcooled and superheated local thermodynamic states.

Fig. 5

 5 Fig.[START_REF] Mameli | Thermal response of a closed loop pulsating heat pipe under a varying gravity force[END_REF] The first FPPHP used in a PFC in 2014: (a) the engraved channel on the copper plate, (b) the blue silicon glue used to separate the channels and to seal the upped plate, (c) the section view with the thermal gaps to decrease the lateral heat flow connection.

Fig. 6

 6 Fig. 6 The Capillary Single Loop TPHP which served for the first experiments about the evaluation of the heat transfer convection coefficient using a high speed MWIR camera and the resolution of the Inverse Heat Transfer Problem.

Proceedings of the 17th International Heat Transfer Conference, IHTC-17

ACKNOWLEDGMENTS

This research has been a coral activity involving 7 academic European universities and the European Space Agency. Including M.Sc. and PhD students, 87 researchers have been involved in the construction of 19 PHP devices, tested in 21 ESA PFCs and 3 SR missions (https://eea.spaceflight.esa.int/portal/). It is worth also to acknowledge that, within this research framework, three student teams in Pisa and Brighton have been awarded for the Fly-Your-Thesis ESA and Student REXUS SR program. The work has been financially supported by UK EPSRC through the grant EP/ P013112/1 (HYHP), as well as the ESA MAP Projects INWIP and TOPDESS, Emilia-Romagna Region (POR-FESR 2014-2020 (NANOFANCOIL), the financial support from CNES in the framework of the GDR/MFA. Further acknowledgements should be given to Prof. P. Di Marco (UNIPI, Italy), Prof. K. Vieira de Paiva (UFSCA, Brazil), Prof. Sara Rainieri (UNIPR, Italy), Giuseppe Correale (ESA), Prof. J. Bonjour (INSA, Lyon, France), Prof. S. Khandekar (IIT, Kanpur, India) and Prof. S. Van Vaerenbergh (ULB, Belgium).