Yvan Burguin

Philippe Rauffet

David Espes

Christine Chauvin

Philippe Le Parc

Cyberdefense Adaptive Training Based on the Classification of Operator Cognitive State

Keywords: Adaptive Training, Cyberdefense, Human Factors, Decision Making Model, Physiologically Based Adaptive Training

To face the increasing number and the variety of cyberattacks, training and adaptation of cyberdefense operators become critical and should be managed all along their careers. Thus, it is necessary to develop adaptive training methods that are able to quickly detect operators' weaknesses and to propose a strategy to reinforce their skills on these points. This paper presents the choice of a cognitive model in order to guide the development of an adaptive training software. In this regard, the paper proposes a review of several elements that contributed to the development of the model.

INTRODUCTION

Cyberattacks are continuously increasing in variety and number, and therefore require a constant adaptation from the operator who must react to each attack with rapidity and efficiency. To face these changes, cyber operators must be trained regularly.

This training aims to: 1) maintain knowledge of cyber operators up to date, 2) train cyber operators to use new tools and 3) allow cyber operators to appropriately react to new attacks.

In this regard, adaptive training softwares support the training of cyberdefense operators in order to improve their performance in real conditions. To propose an adaptive training software, there are several requirements to satisfy [START_REF] Jones | Cognitive Agents for Adaptive Training in Cyber Operations[END_REF][START_REF] Trifonov | Possibilities for Improving the Quality of Cyber Security Education through Application of Artificial Intelligence Methods[END_REF] such as an ecological environment, a system to adapt the training scenario autonomously and a way to assess the difficulties experienced by the trainee. To support this dynamic and customised adaptation of the training scenario, it is important to detect or predict when errors may occur. For this purpose, behavioural and physiological data can be used to assess the variations in performance and mental workload that can lead to an error [START_REF] Dykstra | Cyber Operations Stress Survey: Studying fatigue, frustration, and cognitive workload in cybersecurity operations[END_REF][START_REF] Sawyer | Cyber vigilance : Effects of signal probability and event rate[END_REF]. This paper deals with the choice of a cognitive model that could support the design of a software for adaptive training in the cyberdefense field. Such a model would allow us to understand the different cognitive processes used by the operator to perform tasks, and to identify the factors that could contribute to performance decrement. This model can then orient the selection of appropriate physiological and behavioural indicators to measure what parts of the task cause difficulty to the operator.

The organisation of this paper is as follows. First, existing models and the different tasks performed by a cyberdefense operator during its daily activities are presented in the Related Work section. The section "model for an adaptive training" presents our model that fulfils all the requirements for an adaptive training model. The conclusion section concludes this paper.

RELATED WORK

Different approaches have been developed to identify the problems occurring in the cognitive activities of cyber operators. Several studies have focused on the definition of metrics that help to detect the human reliability of operators during the cyber tasks [START_REF] Henshel | Predicting proficiency in cyber defense team exercises[END_REF][START_REF] Klein | From detection to reaction-A holistic approach to cyber defense[END_REF][START_REF] Sawyer | Cyber vigilance : The human factor[END_REF], by especially considering response times (e.g., for detection, for threat solving) as well as categorization of attacks or threats. Some others have also investigated and demonstrated an effect of visual load or fatigue on an increased error rate of cyber operators in detection tasks (G. [START_REF] Klein | From detection to reaction-A holistic approach to cyber defense[END_REF][START_REF] Paul | Understanding operator fatigue, frustration, and cognitive workload in tactical cybersecurity operations[END_REF]. Nevertheless, these studies only focused on the performance metrics. If they can help for detecting some errors in the cyber tasks, these metrics do not provide insights on the root causes of the errors. It is therefore necessary to open the black box of the cognitive activities carried out by operators, to understand where the error or the difficulty experienced by the cyber operator can come from.

The following section will present a brief literature review on some cognitive models applied to the domain of cyberdefense operations. Some models are mainly inspired by the Natural Decision Making framework, with a focus on the situation awareness, while others are more focused on the planning and the organisation of the different cognitive tasks within structured workflows.

Situation Awareness Model in Cyber Activities

Endsley introduces the situation awareness (SA) concept as the combination of three levels: perception of the elements in the environment, the comprehension of the situation and the projection of future states of the situation [START_REF] Endsley | Toward a theory of situation awareness in dynamic systems[END_REF]. She defines the process of creating this SA as the situation assessment. An important point of the situation assessment process is that it is highly dependent on the actual SA. Based on this knowledge, the situation assessment process is performed as a cycle that analyses the situation. Another process aiming to enhance the SA is sensemaking (G. Klein et al., 2006a[START_REF] Klein | Making sense of sensemaking 2 : A macrocognitive model[END_REF]. This process tends to represent the understanding process of an unusual situation. It focuses on the inference of a frame that matches with the perceived data based on some pertinent points.

Various studies already attempt to analyse and model how the SA is built in the cyberdefense operations [START_REF] D'amico | Achieving cyber defense situational awareness : A cognitive task analysis of information assurance analysts[END_REF][START_REF] Endsley | Foundation and challenges[END_REF][START_REF] Franke | Cyber situational awareness-a systematic review of the literature[END_REF]. Especially, the work of d' Amico et al. (2005) brings a well detailed model of the progression through three stages of cyber situation awareness during the whole task of the operator. Moreover it also introduces the decision point during this process. The first stage is the detection stage, which consists in an analysis of the primary sensor data and processing of these data to transform them into information. The second stage is named the situation assessment stage and consists of including more data sources and processing all the information to finally obtain knowledge. Then the last stage is called threat assessment, which consists in the analysis of the incident, adding intelligence data, and finally in processing the information into knowledge and predictions. This model brings a precise description of the successive technical activities to build operator's situation awareness. Although this model describes precise actions to be taken, it does not detail the cognitive process involved to achieve the task. To build the SA, decision points are introduced in the model. They help to refine the SA based on the actions that are done by the operator. Indeed, decision making is a process that highly depends on the SA. It is particularly true in the cyber domain where it is highly related to the cyber SA [START_REF] Endsley | Foundation and challenges[END_REF]. The task of the operator is to recognise the threat and to apply the established procedure. Then the main skills to develop for the operator during the training should be its ability to correctly assess the situation and to correctly adhere to procedures.

System Analysis and Workflow in Cyber

Some studies lead to further investigation into the workflow of the operators [START_REF] Curnutt | KNOWLEDGE MANAGEMENT APPLICATION TO CYBER PROTECTION TEAM DEFENSE OPERATIONS[END_REF][START_REF] Franklin | Toward a visualization-supported workflow for cyber alert management using threat models and human-centered design[END_REF][START_REF] Gutzwiller | A task analysis toward characterizing cyber-cognitive situation awareness (CCSA) in cyber defense analysts[END_REF][START_REF] Trent | Modelling the cognitive work of cyber protection teams[END_REF]. Such studies bring a finely detailed model of the technical activity of the operators. The paper of Trent, Hoffman, Merritt and Smith especially, highlights four distinct phases in the cyber operator task [START_REF] Trent | Modelling the cognitive work of cyber protection teams[END_REF]. These four phases consist in a first step of planning and logistics to define the activities that can be done by the cyber operator. The second phase consists in the monitoring and the collection of data on the network's flow. Then these data are analysed in the third phase to identify and characterise the elements of interest. Finally a last phase consists in reporting the findings and defining a solution process. The two intermediate phases (i.e., monitoring and collection, and analysis) support the continuous sensemaking about the state of the network. Another main point highlighted by all these studies is the precise repartition of the task among the team [START_REF] Trent | Modelling the cognitive work of cyber protection teams[END_REF]. Indeed, the task of the operator includes the respect of the report procedure that allows for other parts of the team to continuously develop and adapt the procedures. This kind of study brings a very precise description of the theoretical workflow of the operator in a well organised organisation. Such precise details about the workflow have led to the development of several metrics like time needed to apply procedures or other metrics based on the procedure implementation [START_REF] Willett | Cybersecurity Decision Patterns as Adaptive Knowledge Encoding in Cybersecurity Operations[END_REF]. But, not all the organisations are well organised and the operators do not always follow precisely the established protocols.

Finally, this type of models only focus on the best way to realise the task without considering human or organisational failure. In order to take into account these points, there is a need to look forward to natural decision making model in cyberdefense, trying to highlight the cognitive process implied in the different parts of the task described in technical models.

Towards a Naturalistic Decision Making Model for Cyberdefense

The previous models presented above suffer from a lack of comprehension about the decision process itself and only focus on the elements that support the decision process, such as the building of the SA, or the organisation of the activity. Moreover they do not really explain how the team can influence the decision, or how the operators' expertise as well as their familiarity with the situation impact on the performance of the cognitive processes. These considerations are close to those of naturalistic decision making (NDM) approach [START_REF] Zsambok | Current and future applications of naturalistic decision making in aviation[END_REF]. Indeed, the NDM attempts to model the decision making closer to reality. To do so, based on the decision making process used by experts, the recognition primed decision theory (RPD) [START_REF] Zsambok | Current and future applications of naturalistic decision making in aviation[END_REF] proposes a model where the subject makes a decision based on a recognition of the situation. This model can be adapted in three variants depending on the situation.

The first variant is the simple match decision, and it is used when the operators immediately perceive the situation as typical. This straightforward recognition of the characteristics of the situation allows them to immediately implement the appropriate actions. The second variant corresponds to the situations where the operators do not recognize the situation as typical, and implement several mechanisms to further investigate it. Such mechanisms are also involved in case of anomaly detection after the first recognition of the situation to correct the diagnosis. Finally the last variant consists in the evaluation of the projected outcomes of the decision. In such a case the operators add a step of mental simulation before implementing the decision and if the projected results are not perceived as satisfying, they will try either to adapt the decision or to reconsider the situation if needed.

To model the decision making process in the cyberdefense operations, it could be useful to consider this model, along with the question of interactions with the rest of the team. Cybersecurity operations are rarely the burden of one single operator and have to be considered as a team activity, with collective decision making. Thus, to dispose of a global model of cognitive activity, there is a need for a model that synthesises all the cognitive operations involved in the decision making, from the situation assessment and sensemaking to the decision itself and its team dimension. The proposal presented in the next section will be based on RPD, with a few adaptations to take into account this question.

MODEL FOR AN ADAPTIVE TRAINING SOFTWARE

In order to propose a global model of the cognitive activity involved during the different cyberdefense tasks, we propose to add specific elements to a basic RPD scheme, related to the building of SA of cyber operators, the potential team interactions, the relationships between the decision quality and the technical skills of the operators, and finally the link between the benefits of training and experience acquired by the cyber operators during the different phases of the model.

As mentioned by [START_REF] Klein | A recognition-primed decision (RPD) model of rapid decision making[END_REF], when experts are confronted to highly dynamic situations with strong time pressure, they implement a process of recognition (see Figure 1. Recognition) of the situation which allows them to make a reactive decision based on their past experience.

However, if the situation is not immediately recognized, the cyber operators try to increase their comprehension of the anomaly i.e., they act to increase their SA (see Figure 1. Sensemaking loop). In such a case, the cyber operator can act by following two different ways. First, they use the team's resources which basically consist of the experience of their colleagues to recognize the anomaly. Second, they carry out a deeper analysis of the system in order to increase their SA. This process can be iterated several times until the anomaly is correctly identified.

Once the anomaly is recognized, the operators have to apply the established procedure (see Figure 1. Existing procedures). If they are able to implement the procedure, they will do it. Conversely, if they do not know how to implement the procedure, due to a lack of knowledge or experience, the operators have to find another solution to restore the safety of the system. At this step, there are two ways to find the best solution to mitigate the attacks. First, they can ask for help from the team and follow a report procedure. According to their experiences, they can propose a solution to the cyber operators that can deploy it. Second, the cyber operators can find themselves a solution to mitigate the cyber attacks, by adapting and slightly changing an established procedure. Usually, an analytical strategy is used to decide the course of action. The cyber operators look for all the possible ways they have at their disposal to prevent the cyber attacks. They select the one with which they are the most familiar (see Figure 1. Tool review).

For future events and to increase the experience of cyber operators, the previous accumulated knowledge has to be stored and reused for future training (see Figure 1. Experience / Knowledge). The aim is to limit the need to request assistance from the team that can be time consuming. In order to have a fast and accurate response, the cyber operators should be able to correctly understand the situation and to perform the best actions to prevent the cyber attack. The experience and knowledge block represents the database of the operators that allows them to recognise the situation. It is also implied in the construction of the SA because the cyber operators infer from this database the knowledge of the network architecture and their understanding of the cyber attack. Finally, this block is the main component for the training of cyber operators. The knowledge that it contains can be used for the mental simulation of the cyber operators and so to generate new scenarios for the adaptive training of cyber operators.

Unlike other existing models that only focus on a part required by adaptive training models, such as situation awareness or operator errors, our proposed model integrates all these components and describes how they interact between them. Our model is based on a decision block that is at the centre of the model. As in other models, the situation assessment corresponds to a loop that helps to recognize the situation. The novelty of our model resides in the use of a second loop that represents the process undertaken by the operator in reaction to the perceived situation. Moreover, this second loop admits potential deviations with existing procedures. This model, although specific to the activity of the operators, represents the cognitive functions achieved in cyberdefense operations without forgetting the tasks that he has to perform. It is sufficiently abstracted and avoids adding too much details on the tasks in order to keep a high modularity. A specificity of our model is also its ability to leverage team experience for a better understanding of the situation or improving the choice of the response that has to be done. Moreover the model highlights the link between the operators' process and their previous knowledge and experience. Using these indicators, the existing models propose a decision process that relies on the situation awareness of the cyber operator. However, these models suffer from a lack of usability in a context of adaptive training because 1) cyber operators do not interact sufficiently with other members of the team and 2) they do not take into account the errors that cyber operators can make. The contribution of our model on these points would gain to be reinforced through experimental validation.

Unlike other existing models, our proposed model is sufficiently flexible to support cyber operators during their training in an autonomous way. To reach this goal, we compensate for the loss of technical precisions with a cognitive approach of the operations. Moreover, the two loops emphasised in the paper open new perspectives to assess the difficulty experienced by cyber operators. The instantiation of these two loops could be assessed with traditional performance metrics (like temporal indicators or threat categorization accuracy), but we could also imagine to use physiological and behavioural indicators to measure the mental effort exerted by cyber operators to implement these loops, and to determine the causes of these difficulties. Our model is the cornerstone to design adaptive training software to automatically determine which type of training is suitable for the cyber operator in order to follow a pedagogical strategy that is necessary to respond appropriately to new threats in cybersecurity.

Figure 1 .

 1 Figure 1. Model of the cognitive and social process involved during the task


Margot Cessans

Eric Richard

Louise Lemonnier

Marie-Pierre Toquet

Ludovic Petit

Anne Courouce

Corre

Correlation between cytologic and microbial respiratory profiles and performance in french trotters

Objective: Investigate a possible association between cytological, mycological and bacteriological profiles of the respiratory tract of racehorses and the level of performance

Hinchcliff et al (2015) -Exercise Induced Pulmonary Hemorrhage in

Horses: American College of Veterinary Internal Medicine Consensus Statement.

Richard et al (2009) -Tracheal inflammation syndrome: clinical and bacteriological evaluation. 4. Crispe et al (2017) -The association between exercise-induced

pulmonary haemorrhage and race-day performance in Thoroughbred racehorses.

Morley et al (2015) -Exercise-induced pulmonary haemorrhage impairs

racing performance in Thoroughbred racehorses.

Malikides et al (2003) -Comparison of tracheal aspirates and

bronchoalveolar lavage in racehorses : Evaluation of the diagnostic significance of neutrophil percentage.

Depecker et al (2015) -Bronchoalveolar lavage fluid in Standardbred racehorses: influence of unilateral/bilateral profiles and cut-off values on lower airway disease diagnosis.

Good performers

  CORRELATION BETWEEN CYTOLOGIC AND MICROBIAL RESPIRATORY PROFILES AND PERFORMANCE IN FRENCH TROTTERS CESSANS M. 1 , RICHARD, E.A. 3 , LEMONNIER, L.C. 1 , TOQUET, M-P. 3 , PETIT, L. 3 , COUROUCE, A. 1,2

  (p-value = 0,041) • Innovative and interesting database • Methodological biases: number of individuals included, performance measurement • The relationship between airway colonization by bacterial or fungal elements and the link with performance remains to be proven : Pilot study • Search for fungal elements: Is the BAL more specific than the TW ? Conclusion & limits: 12 20 Statistical analytis : Mann-Whitney or Chi-square test (P-value < 0,05) 1. Couëtil et al (2016) -Inflammatory Airway Disease of Horses-Revised Consensus Statement.

• Fungal colonization of the airways was not associated with poor performance. • BAL seems to be more specific than TW 8 : Mycology (+) on BAL pool = 9% Mycology (+) on TW = 96% Crispe et al (2017) 4 : a low-grade EIPH would not decrease performance. It would be a sign of a more intense exercice. Morley et al (2015) 5 : a non-EIPH horse would be >2 times more likely to win races.

8. Lemonnier, L.C. et al (2022) -Variability of fungi detection in the respiratory tract of racehorses according to the sampling site

(ICEEP, Sweden)


Arlindo T Souza Netto

Arthur V S Oliveira

Michel Gradeck

Rogério Gonçalves

Dos Santos

Project of a pressure stabilizing closed system for fast boiling experiments

With neither the use of pumps (for excessive cost reasons) nor the use of compressed gas (for safety reasons), designing an experimental bench capable of operating at approximately constant pressure during vaporization of liquids in a closed system is challenging. We present in this paper the step-by-step project of an original pressurization solution for closed systems that has the final purpose of performing pool boiling experiments with water, gasoline, and ethanol at pressures up to 80 bar.

Nomenclature

Introduction

The reduction of pollutant emissions and the improvement of cold phase performance of internal combustion engines (ICEs) can be achieved by heating the liquid fuel prior to its injection into the cylinders [START_REF] Huang | Investigation of the effect of heated ethanol fuel on combustion and emissions of an ethanol direct injection plus gasoline port injection (EDI+ GPI) engine[END_REF][START_REF] Schulz | The effect of operating parameters on the formation of fuel wall films as a basis for the reduction of engine particulate emissions[END_REF]. Given the short heating time required as well as the power input, this technique results in the occurrence of boiling on the surface of the heater. The fuel heating phenomenon is especially important to be well understood during the phase shortly before the driver starts the engine, when the fuel rail and heater work in a closed system. This is the most critical stage of the project from a safety point of view, once it implies the rapid boiling of a small, closed volume of inflammable liquid. Moreover, the highest emission rates observed throughout the operation of an ICE normally occur during the cold phase [START_REF] Saliba | Comparison of gasoline direct-injection (GDI) and port fuel injection (PFI) vehicle emissions: emission certification standards, cold-start, secondary organic aerosol formation potential, and potential climate impacts[END_REF][START_REF] Du | Testing and evaluation of cold-start emissions in a real driving emissions test[END_REF][START_REF] Yusuf | Effect of cold start emissions from gasoline-fueled engines of lightduty vehicles at low and high ambient temperatures: Recent trends[END_REF]. Hence, heating the fuel until a target temperature becomes important to reduce pollutant emissions in this phase. Driven by this motivation, we present here the design of an experimental bench consisting of a closed system representative of a commercial heated fuel rail used in Brazil.

The project features a hand-operated pump for initial pressurization and a test section containing the test fluid and operating with a commercial fuel heater. The use of borosilicate glass in the test section, cameras, and adequate lighting make it possible to visualize and film the experiments. A condenser and a spring-piston system were designed externally to the test section as means to decrease the pressure variations during the tests caused by the vaporization of the liquid fuel. One of the major challenges in the modeling is dealing simultaneously with transient boiling and condensation. Simulations of the heating and cooling processes, which are presented in this paper, indicate that the closed system pressure control using these concepts is possible within the required operating ranges. Several parameters' behaviors such as the condensation length, spring displacement, and pressure throughout the experiments were used as the basis for dimensioning the piston diameter, the spring coefficient of elasticity, and the condenser tube length.

Experimental apparatus design

The experimental apparatus consists of a closed boiling chamber representative of a fuel heating system operating in a vehicle fuel gallery. It was projected to operate at pressures up to 80 bar and to work with water, ethanol, and gasoline. The test section has a square cross section of 12 mm edge, and its dimensions are displayed in Figure 1. Figure 2a shows a schematic illustration of the test bench. The pressurization of the system occurs by means of a manual pump (A) associated with the displacement (x) of a piston (B) and consequent compression of a spring (C). A condenser (D) was designed to reduce the pressure variation throughout the experiments due to the generation of vapor by boiling on the surface of the resistive heater (E) positioned inside the test section (F). Table 1 displays the values considered for some variables involved in the project and simulation of the experimental bench.

Modeling of the pressure control system

Figure 2b displays a control volume (CV) that contains the gas inside the pipeline and condenser tube. The same CV is illustrated in Fig. 2a, where we can notice that there are moments in which condensation occurs only in the pipeline and others in which it occurs both in the latter and in the condenser. In Fig. 2b, 𝑚̇! "#$ is the mass flow of steam entering the CV, 𝑚̇% is the mass flow of liquid water leaving the CV after being condensed and 𝑞̇% is the power leaving the CV by condensation on the inner walls of the pipeline (𝑞% ! ) and condenser (𝑞% " ), when such is the case. A mass balance for this CV can be analyzed to estimate the condensing length behavior during the experiments. The pipeline is externally exposed to ambient air and the condenser consists of a tube surrounded by a water jacket at ambient temperature (20°C). The inner and outer diameters are displayed in Table 1.

The following assumptions have been made with the objective of estimating the variables conservatively and ensuring proper operation of the project: the test section is considered adiabatic; all the heater power is applied for the test fluid vaporization in the test section (𝑚̇! "#$ in Fig. 2b), which is afterwards condensed on the internal surfaces of the pipeline and condenser; only film condensation takes place, whose heat transfer coefficients are lower than those observed for dropwise condensation; fully developed film condensation starts after one second, time during which an exponential function was used to represent a transition (see Section 2.3 for more details on the shape of the function); two uniform wall temperatures are considered, one for the pipeline (𝑇 &#'' ! ) and other for the condenser (𝑇 &#'' " ); the walls initial temperatures are equal to the ambient temperature (20°C); a minimum value 𝐿 ()* is used when the condensing length, either for the pipeline or the condenser, is less than this critical value; the minimum value for the difference between the saturation and wall temperatures is 𝑑𝑇 ()* .

The mass balance for the CV is:

𝑑𝑚 +, 𝑑𝑡 = 𝑚̇! "#$ -𝑚̇% = 𝑞̇-!#.!/ ℎ 01 - 𝑞̇% ! + 𝑞̇% " ℎ 01 (1)
where 𝑞-!#.!/ is the heater power (500 W), 𝑞% ! is the heat flux related to condensation on the pipeline, 𝑞̇% " is the heat flux related to the condenser and ℎ 01 is the fuel's heat of vaporization.

It follows that:

𝑑𝐿 𝑑𝑡 = ,𝑞̇-!#.!/ -𝑓 3!1 .𝑞̇% ! + 𝑞̇% " /0 𝜌 " 𝐴 %/455 ℎ 01 (2)
where 𝑓 3!1 is a degradation parameter better explained in Section 2.3, 𝜌 " is the vapor phase density and 𝐴 %/455 is the tube internal cross-sectional area.

The thermal-electrical analogy was considered to model the heat flux of condensation, which is dissipated by free convection with air for the pipeline and with water for the condenser. The conduction thermal resistances were neglected after performing some calculations that revealed a very small difference between the internal and external temperatures of the walls. The following iterative procedure was applied to model the transient phenomenon and estimate the wall temperatures:

𝑇 &#'' # (𝑖) = 𝑇 &#'' # (𝑖 -1) + 𝑑𝑡 𝑞 % # (𝑖) -𝑞 0% # (𝑖) 𝑚 .67! # (𝑖)𝑐 $ 5.!!' (3)
where 𝑘 can be equal to 1 (pipeline) or 2 (condenser), 𝑚 .67! # are the associated steel masses, and 𝑐 $ 5.!!' is the steel specific heat capacity.

Condensation and free convection correlations

The condensation heat fluxes can be represented by Newton's cooling law as follows:

𝑞 % # = ℎ 9 % # 𝜋𝐷 .67!,) 𝐿 9 (𝑇 5#. -𝑇 &#'' 9 ) (4)
With the intention of performing conservative calculations of the project variables, it was decided to disregard the initial drop-wise condensation and consider only film-wise condensation. Hence, the film condensation HTC (ℎ 9 % ) was estimated with the laminar film condensation for vertical plates theory formulated by [START_REF] Nusselt | The surface condensation of water vapour[END_REF] [START_REF] Nusselt | The surface condensation of water vapour[END_REF].

For the free convection on the external surface of the condenser (inside the water jacket), we have:

𝑞 0% # = ℎ 9 0% # 𝜋𝐷 .67!,! 𝐿 9 (𝑇 &#'' # -𝑇 5#. ) (5)
The Churchill and Chu (1975) [START_REF] Saliba | Comparison of gasoline direct-injection (GDI) and port fuel injection (PFI) vehicle emissions: emission certification standards, cold-start, secondary organic aerosol formation potential, and potential climate impacts[END_REF] correlation for free convection on vertical plates may be applied over the entire range of 𝑅𝑎 : and was used to estimate the free convection HTCs (ℎ 9 0% ).

Characteristic times of boiling and condensation phenomena

The characteristic time of a physical phenomenon can be associated with the time interval required for its onset from the moment the conditions for its occurrence are established. Boiling and condensation are phenomena with considerably low characteristic times. For condensation of pure steam at atmospheric pressure, the onsetting process is ultimately instantaneous [START_REF] Churchill | Correlating equations for laminar and turbulent free convection from a vertical plate[END_REF]. As one can see from the experimental study of [START_REF] Ma | Transient characteristics of initial droplet size distribution and effect of pressure on evolution of transient condensation on low thermal conductivity surface[END_REF] [9] on the transient characteristics of droplet size and effect of pressure on evolution of transient condensation of water on a low thermal conductivity surface, as the pressure increases, there is a decreasing tendency for the time required for all the stages of drop-wise condensation to be completed. For ambient pressure, it is possible to assume that the stage I (nucleation stage) is completed after 0.5 s. Therefore, also considering that in our model condensation takes place on steel, whose conductivity is higher than that of polycarbonate, the time interval conservatively considered for the film condensation to fully establish is one second. This assumption is made with the interest of performing a conservative calculation of the phenomena that ensures the proper operation of the pressure control system.

Toward that direction, aiming to account for a delay between vapor entering the condenser and the film condensation settling, but also aiming to smooth the onset of the phenomenon, the following degradation function was used to transition the heat removed by condensation between 0% (𝑡 = 0 𝑠) and 100% (𝑡 = 𝑡 ; = 1 𝑠) of that expected by the correlation of film-wise condensation.

𝑓 3!1 = exp 𝑡 𝑡 3!1 -1 exp 𝑡 ; 𝑡 3!1 -1 (6)
where 𝑡 3!1 is the shape parameter of the function.

Figure 6 shows how the shape of the function 𝑓 3!1 varies for different values of 𝑡 3!1 . Note that the smaller the value of 𝑡 3!1 , the closer to a step function 𝑓 3!1 becomes. The higher the value, the closer to a straight line.

Pressure variation estimation

The initial condition of the system is taken to be the pre-compression of the spring towards the experiment pressure, when all the fluid is in the liquid phase and at room temperature. The pressure variation is estimated for each time interval (𝑖) as follows:

𝑑𝑃 ) = 𝑘 5$/)*1 𝑑𝑥 ) 𝐴 $)5.4* (7)
𝑑𝑥 ) = 𝑉 1#5 $%(') -𝑉 ')< $%(') 𝐴 $)5.4* (8)
𝑉 1#5 $%(') = 𝜋𝐷 )*. = 4 𝑑𝐿(𝑖) (9)
where 𝑘 5$/)*1 is the spring elasticity coefficient, 𝑑𝑥 ) is the piston and spring displacement, 𝐷 )*. is the pipeline and condenser tube internal diameter, 𝑉 1#5 $%(') is the condensation volume variation at the time interval, 𝑉 ')< $%(') is the volume occupied by 𝑉 1#5 $%(') when at ambient conditions and 𝑑𝐿(𝑖) is the condensation length variation at the time interval estimated with Eq. 2.

Results and discussion

The variables of interest involved in the dimensioning of the spring-piston setups for the different pressure levels are the maximum force (𝐹 (#> ) and maximum displacement exerted on the spring (𝑥 (#> ), the maximum length of condensation (𝐿 (#> ), and the maximum pressure 17 𝑡 " reached during the experiments (𝑃 (#> ). To guide the selection of appropriate diameters for the pistons and spring elasticity coefficients, graphs were prepared to examine the behavior of these variables with varying piston diameter and for springs of different elasticity coefficients. A pressure tolerance parameter (𝛼) was included to the pressure variation analysis:

𝛼 = 𝑃 $*+ 𝑃 0 (10)
Therefore, accepting a tolerance of 20% variation of the test initial pressure (𝛼 = 1.2), a straight line with value 1 can be drawn as an upper limit reference on the graph shown in Fig. 4a. In this context, the region of Fig. 4a below the black line consists of projects with less than 20% variation in initial pressure, and the region above corresponds to more than 20% variation in initial pressure. Furthermore, based on the analysis of spring manufacturers' catalogs, it is possible to use maximum project values 𝐹 $,(#> (2200 N) and 𝑥 $,(#> (90 mm) above which no springs are commercially available. This approach is adopted in the graphs of Fig. 4, that exemplifies the analysis for experiments at 10 bar with 𝑡 3!1 = 0.3 and 𝛼 = 1.2. For Figs. 4a, 4c and4d, the regions of the graph with y-axis values smaller than 1 are considered as possible choices for the project variables, while Fig. 4b gives the maximum condensing length to be reached. For example, if 𝑘 5$/)*1 = 20 N/mm, the piston diameter must be between 35 mm, limited by the pressure criterion, and 45 mm, given by the spring compression criterion. In this case the maximum condensing length is shorter than 0.38 m. Based on this approach, Table 2 displays the selected feasible project values for the different pressure levels, while Table 3 shows springs A and B characteristics. We aimed to minimize the differences between the arrangements by using similar diameters and springs between them when possible. Figure 5 shows the transient results obtained for the different pressure levels. The filled lines represent the analysis with 𝑡 3!1 = 0.3 while the dashed lines are the system response without considering the initial condensation degradation. We can see that the lower the pressure, the greater the required condensing length (Fig 5b), given that for lower pressures a greater density variation is observed between the liquid and vapor phases. At 8 seconds of simulation the heater is turned off. Therefore, we see a rapid drop in the graphs at this instant until 𝐿 = 0.2 𝑚, when the condensed liquid fills the entire condenser tube, and then a slower drop for the fluid condensing in the pipeline exposed to air. It is possible to see that, even considering the initial degradation of the heat exchange, the system is nevertheless able to operate within the predefined project ranges. Considering Fig. 5b, and aiming for an oversized design, a 0.8 m condenser length is considered sufficient for the project.

Conclusion

In this paper the project of a test bench capable of operating at approximately constant pressure during fast pool boiling experiments was originally proposed. It includes a condenser and spring-piston assemblies. From its simulation and considering the operating characteristics of commercially available springs, the system could be conservatively dimensioned to perform experiments at pressures up to 80 bar.

Figure 1 :

 1 Figure 1: Front view of the test section (illustrative CAD).

Figure 2 :

 2 Figure 2: (a) Schematic drawing of the experimental apparatus, including the (A) manual pump, (B) piston, (C) spring, (D) condenser, (E) resistive heater, and (F) test section. (b) Control volume used for the mass balance analysis.

Figure 3 :

 3 Figure 3: Shape of the degradation function for different values of 𝑡 '(" and 𝑡 ) = 1 𝑠.

Figure 4 :

 4 Figure 4: Project variables analysis for tests at 10 bar.

Figure 5 :

 5 Figure 5: Transient behavior of the project variables for different pressure levels. (a) pressure, (b) condensing length, (c) force acting on spring, and (d) spring displacement.

Table 1 :

 1 Characteristics of the experimental bench.

	Heater power	500 W
	Pipeline internal diameter	6 mm
	Pipeline external diameter	8 mm
	Pipeline length (exposed in air)	200 mm
	Condenser internal diameter	6 mm
	Condenser external diameter	8 mm
	Ambient temperature	20 °C
	Temperature of the water in the jacket	20 °C
	𝐿 $%& (minimum condensing length)	5 mm
	𝑑𝑇 $%& (minimum temperature difference)	1 °C

Table 2 :

 2 Spring-piston setups.

	Pressure [bar]	Piston diameter [mm]	Spring	Number of springs [-]
		5	45		A	1
		10	45		A	1
		20	32		B	1
		40	32		A	2
		80	32		A	4
	Spring	Elasticity coefficient [N/mm]	Max. load [N]	Max. displacement [mm]
	A		23.333		2100	92
	B		31.111		2260	76

Table 3 :

 3 Springs characteristics.

Acknowledgements

This study was financed in part by LEMTA (Université de Lorraine), and by the Coordenação de aperfeiçoamento de Pessoal de Nível Superior -Brasil (CAPES) -Finance Code 001.


Adulam Jeyasothy
email: adulam.jeyasothy@lip6.fr

Thibault Laugel
email: thibault.laugel@axa.com

Marie-Jeanne Lesot
email: marie-jeanne.lesot@lip6.fr

Christophe Marsala
email: christophe.marsala@lip6.fr

Marcin Detyniecki
email: marcin.detyniecki@axa.com

Integrating Prior Knowledge in Post-hoc Explanations

Keywords: explainable artificial intelligence, prior knowledge, counterfactual explanation, compatibility

In the field of explainable artificial intelligence (XAI), posthoc interpretability methods aim at explaining to a user the predictions of a trained decision model. Integrating prior knowledge into such interpretability methods aims at improving the explanation understandability and allowing for personalised explanations adapted to each user. In this paper, we propose to define a cost function that explicitly integrates prior knowledge into the interpretability objectives: we present a general framework for the optimization problem of post-hoc interpretability methods, and show that user knowledge can thus be integrated to any method by adding a compatibility term in the cost function. We instantiate the proposed formalization in the case of counterfactual explanations and propose a new interpretability method called Knowledge Integration in Counterfactual Explanation (KICE) to optimize it. The paper performs an experimental study on several benchmark data sets to characterize the counterfactual instances generated by KICE, as compared to reference methods.

Introduction

Machine learning models are now present in many areas of everyday life. The XAI domain [START_REF] Burkart | A Survey on the Explainability of Supervised Machine Learning[END_REF][START_REF] Linardatos | Explainable AI: A Review of Machine Learning Interpretability Methods[END_REF] aims at answering questions raised by these models such as "Why do I get this prediction?" or "What do I need to do to get the prediction I want?". In particular, post-hoc methods focus on generating explanations for the predictions obtained from a trained classifier. They come in different formats such as feature importance (e.g LIME [START_REF] Ribeiro | Why should I trust you?" Explaining the predictions of any classifier[END_REF] and SHAP [START_REF] Lundberg | A unified approach to interpreting model predictions[END_REF]) or counterfactual examples [START_REF] Wachter | Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR[END_REF] (e.g. Growing Spheres [START_REF] Laugel | Comparisonbased Inverse Classification for Interpretability in Machine Learning[END_REF] and FACE [START_REF] Poyiadzi | FACE: Feasible and Actionable Counterfactual Explanations[END_REF]). They also vary depending on their input data: model agnostic and data agnostic approaches consider that no knowledge is available either on the model, nor on data. This lack of knowledge makes it difficult to generate explanations adapted to the context. As a result, they may then not be always understood by the user [START_REF] Rudin | Stop Explaining Black Box Machine Learning Models for High Stakes decisions and Use Interpretable Models Instead[END_REF]. To tackle this issue, some studies focus on enriching the considered input and integrating the human in the loop by taking into account expert knowledge [START_REF] Mahajan | Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers[END_REF][START_REF] Frye | Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability[END_REF][START_REF] Ustun | Actionable Recourse in Linear Classification[END_REF], so the explanation is more understandable.

In this context, this paper proposes a general formalization, through the definition of a cost function that integrates such prior knowledge in the search for explanation. It proposes to add to the classical penalty term, that aims at assessing the quality of a candidate explanation, a complementary compatibility term that takes as parameter the considered knowledge. We provide a discussion about this notion of compatibility for which two contradictory semantics can be considered. Indeed, the prior knowledge can be used either to find an explanation that is complementary to the knowledge or an explanation that follows the same language and is in agreement with the knowledge. It can be argued that an explanation in the knowledge language can increase the user's confidence in the explanation, and a complementary explanation can enrich the user's knowledge.

The paper discusses these cases when the considered knowledge provides information regarding features of interest. Then, it focuses on the case of explanation expressed within the knowledge language in the specific framework of counterfactual explanation: it considers that an explanation is compatible with the available knowledge if the features considered by the knowledge and the explanation are similar. In order to generate such a compatible explanation, it proposes a method called KICE which stands for Knowledge Integration in Counterfactual Explanation and presents the conducted illustrative experiments.

To summarize, the paper contribution consists in proposing a general framework for integrating expert knowledge into interpretability methods in Section 3, an instantiation of the framework in the context of counterfactual explanations and a new counterfactual explanation method to generate a knowledge-adaptive explanation in Section 4.

Background

This section provides background information regarding the framework to which the paper refers: it first describes some post-hoc explanation methods; then, it presents some existing approaches to integrate knowledge, after discussing different possible forms the latter can take.

Post-hoc Explanations

Numerous post-hoc methods which explain the prediction of a trained classifier have been proposed, addressing different problems, see e.g. [START_REF] Guidotti | A Survey of Methods for Explaining Black Box Models[END_REF]. A large portion of them relies on generating explanations as a solution to an optimization problem. Among those, we are particularly interested in counterfactuals and surrogates approaches. The former [START_REF] Lash | Generalized Inverse Classification[END_REF][START_REF] Laugel | Comparisonbased Inverse Classification for Interpretability in Machine Learning[END_REF][START_REF] Wachter | Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR[END_REF][START_REF] Poyiadzi | FACE: Feasible and Actionable Counterfactual Explanations[END_REF][START_REF] Mothilal | Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations[END_REF] aim at answering the question: "What do I have to do to get the prediction I want?". The answer is defined as the closest instance to the studied instance belonging to the desired class. It can be obtained by optimizing, under constraint, a cost function defined as the distance function between the considered instance and the explanation (usually the Euclidean distance [START_REF] Lash | Generalized Inverse Classification[END_REF][START_REF] Laugel | Comparisonbased Inverse Classification for Interpretability in Machine Learning[END_REF]). On the other hand, surrogate-based explanations [START_REF] Ribeiro | Why should I trust you?" Explaining the predictions of any classifier[END_REF][START_REF] Peltola | Local Interpretable Model-agnostic Explanations of Bayesian Predictive Models via Kullback-Leibler Projections[END_REF][START_REF] Sokol | bLIMEy: Surrogate Prediction Explanations Beyond LIME[END_REF] rather answer the question: "Why do I get this prediction?". They rely on an interpretable model, such as a decision tree, a linear regression or classification rules, that approximates the classifier under study to mimic its behavior. In the case of local surrogates we can cite LIME [START_REF] Ribeiro | Why should I trust you?" Explaining the predictions of any classifier[END_REF] that minimizes the fidelity to the black box classifier.

Representation of Knowledge

As discussed in the introduction, post-hoc explanations, as well as any explanation building method, can be enriched by available prior knowledge so as to improve their quality and intelligibility. The next section describes existing approaches, we discuss here several forms that can be considered to represent this knowledge; one can cite among others: class prototypes [START_REF] Van Looveren | Interpretable Counterfactual Explanations Guided by Prototypes[END_REF], distribution of data [START_REF] Poyiadzi | FACE: Feasible and Actionable Counterfactual Explanations[END_REF] or features-based knowledge [START_REF] Ustun | Actionable Recourse in Linear Classification[END_REF]. We are particularly interested in the latter, which is detailed in this section.

A first type of expert knowledge can be expressed by information on the individual features: the user can for instance indicate the so-called actionable features [START_REF] Ustun | Actionable Recourse in Linear Classification[END_REF], i.e. the features that can be modified. For instance, they can include the budget in a credit application, as opposed to the age [START_REF] Lash | Generalized Inverse Classification[END_REF]. A second type of expert knowledge on the features can provide information on their interactions through their covariation [START_REF] Drescher | Toward rigorous use of expert knowledge in ecological research[END_REF][START_REF] Mahajan | Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers[END_REF] (e.g. "An increase in school level leads to an increase in age") or through a causality graph [START_REF] Mahajan | Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers[END_REF][START_REF] Frye | Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability[END_REF]. In this paper, we consider knowledge provided by a set of expected features that can for instance be deduced from a rule-based model.

Integration of Knowledge

Beyond the question of the form the prior knowledge can take, the question of its exploitation and integration in the explanation must be considered. Existing works differ in the way they answer this question and the problem they address.

Considering the case where knowledge indicates which features are actionable and explanations take the form of counterfactual examples, Ustun et al. [START_REF] Ustun | Actionable Recourse in Linear Classification[END_REF] restrict the search set to forbid the use of some features or directions. This allows to avoid proposing inapplicable explanation such as "In order to get the credit, you need to decrease your age".

Also considering the case of counterfactual explanations, Mahajan et al. [START_REF] Mahajan | Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers[END_REF] consider knowledge about causal interactions between features. They propose to integrate it in the definition of a new distance measure that quantifies the extent to which the candidate explanation satisfies the causal relationships. Unlike Ustun et al., they do not exclude candidates that do not respect these constraints, but penalize them, in a more flexible approach. This method makes it possible to avoid unrealistic explanations of the form "In order to get the credit, you need to increase your budget and to decrease your salary".

Frye et al. [START_REF] Frye | Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability[END_REF] are also interested in the integration of causal relationships, but exploit a different information they induce; they considers the case of explanations in the form of local feature importance vector. He focuses on the asymmetry brought by the causal link orientation and proposes to exploit it by weighting the cost function.

These three approaches integrate different types of expert knowledge into different forms of explanation. They consider that this knowledge is an additional constraint in the explanation generation. This constraint is represented in different forms: reduction of the search space, addition of penalty in the cost function or weight in the cost function. In the following section, we propose a general formalization for all methods integrating knowledge.

Proposed General Framework

The considered objective is to explain the prediction of a classifier f : X -→ Y where X denotes the input space, included in R d , and Y the output space. We consider a domain where elements x ∈ X are described by means of a set of features (for instance, age, salary, weight,...). In this paper, we consider continuous data, the case of categorical data will be considered in future work. The prior knowledge to be integrated in the generation of the explanation is denoted E. In the following, we consider that an explanation is related to the prediction f (x) for an element x ∈ X .

Proposed Optimization Problem

This section describes the proposed general optimization problem to generate explanation integrating user knowledge. Let E be the set of explanations for a given type (e.g. counterfactual example or surrogate models), we propose the following optimization problem:

argmin e∈E agg(penalty x (e), incompatibility x (e, E)) (1)
where agg, penalty x and incompatibility x are three functions described in the following. This optimization problem allows to generate a knowledge-based explanation for any type of explanation and type of knowledge. As discussed below, the choice of the three functions depends on the studied context: the motivations of the user, the type of explanation to be generated or the considered type of knowledge. In our approach, we use a subscript x for functions that are related to the studied instance x.

Penalty Function As recalled in Section 2.1, most existing approaches to generate explanations minimize a cost function. Let penalty x : E -→ R be such a function: it takes as argument a candidate explanation e, and may depend on the studied instance x in the case of local explainer. It values the quality of the candidate explanation. For instance, it is defined as the distance between the candidate and the considered instance in the case of counterfactual examples and the fidelity of explanation to the black box classifier in the case of surrogate approaches.

Incompatibility Function To generate an explanation complying with available knowledge, we propose to add a function to measure how compatible the candidate explanation is with the knowledge. This incompatibility function depends on the type of explanation and the type of considered knowledge, as well as the user objective. For instance, an explanation generated for a non-expert customer asking how to get his credit application accepted needs to focus on common knowledge actions such as increasing his salary. On the other hand, in the case where the explanation is destined to a domain expert, the incompatibility can represent the level of fidelity of the explanation with respect to the expert's comprehension of the domain.

As mentioned in the introduction, we propose to distinguish two objectives: to propose an explanation that is complementary to the available knowledge, and to propose an explanation that compete with the knowledge language. We consider an explanation to be in the knowledge language if the features used in the explanation and the knowledge are similar, and an explanation to be complementary in the opposite case (i.e. when there is no redundancy). Below, we discuss these two possibilities in the case of surrogate-based explanations.

Let A e denote the set of features used in the explanation generated by a surrogate e which is an interpretable model, and E the set of features not considered in knowledge E. To build a surrogate-based explanation in the knowledge language, one possibility is to minimize the number of features it takes into account that are not part of E:

incompatibility x (e, E) = Card(A e ∩ E)
On the contrary, where the explanation needs to be complementary to the knowledge, another possibility is to minimize the number of features present in both A e and E: incompatibility x (e, E) = Card(A e ∩ E)

These two definitions can be both relevant depending on the motivations and needs of the user, as discussed above. Such a case is further discussed in Section 4 for the case of counterfactual explanation.

Aggregation Function

The aggregation function combines the penalty and incompatibility functions. There exists an abundant literature on aggregation operators, see e.g. [START_REF]Aggregation Operators: New Trends and Applications[END_REF][START_REF] Grabisch | Aggregation Functions[END_REF], which can be divided into three main categories: conjunctive, disjunctive or compromise. A conjunctive operator, for example the min function, returns a high value only if both penalty and incompatibility taken as input are high. A disjunctive operator can be represented by the max function, it requires at least one value to be high. Finally, trade-off functions such as weighted average result in the compensation of low values by high values.

The choice of this function can be made according to the user preferences, instead of necessarily considering the same function for everyone: one way to personalize the explanation to a user is to let him choose the aggregation he wants to perform.

An Example of Existing Method under this Formalism

To show how general the proposed framework is, we propose to express the stateof-the-art approach proposed by Ustun et al. [START_REF] Ustun | Actionable Recourse in Linear Classification[END_REF] in the proposed formalism highlighting the definition of the three involved functions. In this approach, the user knowledge E is denoted A(x) and defined as the set of modifications that can be applied to instance x, it is integrated to generate an actionable counterfactual explanation. The latter is the solution to an optimization problem of the form:

a * = argmin a∈A(x) cost f ct(a, x) with f (x + a) = f (x)
where cost f ct values the distance between x and x + a. The considered explanation expresses the move between the studied instance and the closest instance in the opposite class. This optimization problem can be rewritten as follows: let

E x = {x ∈ R d | f (x ) = f (x)}, e * = argmin e∈Ex cost f ct(e, x) + 1 |x-e| / ∈A(x) × Z
where Z is arbitrarily large. Here e * is the closest instance in the opposite class, and it enables to retrieve a as a = e * -x. This expression, equivalent to the previous one, makes it possible to identify the penalty, incompatibility and agg functions. The penalty function equals cost f ct defined in [START_REF] Ustun | Actionable Recourse in Linear Classification[END_REF]. The incompatibility function equals 1 |x-e| / ∈A(x) × Z and represents the presence or absence of the modified feature in the user knowledge; it takes only two values 0 or Z. An incompatible counterfactual explanation thus has a very high cost function, resulting in only compatible counterfactuals being considered. Finally, aggregation is performed by a weighted sum. However, as the incompatibility function is binary, only compatible counterfactuals are considered, so the resulting counterfactual is both compatible and of good quality.

Knowledge Integration in Counterfactual Explanation (KICE)

This section proposes a new method to generate counterfactual explanations taking into account prior knowledge, instantiating for counterfactuals the general framework introduced in the previous section. It then details the algorithm used to solve this problem. In the following, to value a distance we consider the l 2 norm.

The principle of the expected counterfactual example can be illustrated in the following toy scenario; let us consider the case of a patient who risks diabetes if he keeps his daily habits. Without knowledge integration, a plausible explanation is: "To predict a low risk, the patient has to decrease his blood sugar by 0.5g/L and increase his carbohydrate by 50g". However, the knowledge that the patient possesses is only reduced to less technical features, such as sport duration and weight: the proposed counterfactual explanation is thus not understandable to him. We aim to propose a counterfactual explanation more adapted to the user knowledge: "The patient has to run 30 minutes longer and lose 2kg".

Proposed Cost Function: Instantiation of the Framework

We propose in this section an instantiation of the general framework in the counterfactual case and considering prior knowledge in the form of a set of features denoted E = {a i , i = 1, ..., m}, considered as a subset of features of the input space X .

Penalty Function For the penalty function, we propose to use the classical counterfactual function which is the Euclidean square distance:

penalty x (e) = x -e 2 (2)
Incompatibility Function As specified in Section 3.1, the objective is to propose a counterfactual explanation in agreement with the user knowledge, meaning that ideally, the counterfactual modifications must be only performed according to the features appearing in E. However, for some instances, focusing only on a subset features raises the risk of never being able to meet the decision boundary of f , leading to no counterfactual explanation being generated. Therefore, we propose to relax this constraint by penalizing the modifications according to the features from E that are not present in the knowledge. This allows, when the boundary cannot be found along the sole features of E, to make sure that a solution can still be found. Thus, we propose the incompatibility function that computes the Euclidean square distance only along absent features:

incompatibility x (e, E) = x -e 2 E = i / ∈E (x i -e i ) 2 (3)
Minimizing this incompatibility avoids generating counterfactual explanations that greatly modify the unknown features.

Aggregation Function

The aggregation function combines the penalty function and the incompatibility function, we propose to use a compromise function:

agg(u, v) = u + λv (4)
with λ ∈ R + a hyper-parameter valued by the user. λ defines the weight given to the incompatibility function. In the case of categorical features, the l 2 distance is replaced with an appropriate distance or a dissimilarity measure.

We then obtain the following optimization problem:

Given E = {x ∈ R d | f (x ) = f (x)} and λ ∈ R + , e * = argmin e∈E cost x,E (e) (5)
with : cost x,E (e) = x -e 2 +λ x -e 2 E (6)

Algorithm Description

The algorithm we propose to solve optimization problem [START_REF] Grabisch | Aggregation Functions[END_REF], named KICE for Knowledge Integration in Counterfactual Explanation, uses the principle of iterative generation of instances, inspired by the Growing Spheres algorithm [START_REF] Laugel | Comparisonbased Inverse Classification for Interpretability in Machine Learning[END_REF], considering the additional term of compatibility to bias the generation. We consider a model and data agnostic case, in which no information about the data distribution nor the decision boundary is available. Therefore, we generate instances around the considered instance x (i.e. for which explanation related to f (x) should be found) in all directions: to find the point that minimizes the cost function, we generate points in increasingly larger spaces until we find one in the opposite class.

The equation cost x,E (e) = ν defines the equation of an ellipse with center x and radius ν 1+λ with respect to the features in E and √ ν with respect to the features in E. To make the ellipses grow, the value taken by ν is iteratively increased. As we cannot cover all possible values for ν, a hyper-parameter > 0 is used to generate elliptical layers defined by radius ν and ν + and generate uniformly in these layers.

In order to do so, we use a modified version of the HLG algorithm [START_REF] Muller | A Note on a Method for Generating Points Uniformly on N-Dimensional Spheres[END_REF] which consists in generating instances uniformly in the spherical layer SL(x, a 0 , a 1 ) which represents the set of points at a distance greater than a 0 and less than a 1 from x. In the HLG algorithm the rescaling step consists in generating a random variable U = {u i } ∼ U([0, 1]), and then calculating a 0 + a 1 U 1 dim(X ) . We distinguish the features E and E, and perform the following operations: a 0 + a 1 u i for i ∈ E and a0 √ 1+λ + a1 √ 1+λ u i for i ∈ E. The instances are then indeed generated in the desired ellipse.

The proposed KICE algorithm covers the space by generating instances iteratively: in the first step, n instances are generated in the ellipse of center x and radius ν 1+λ with respect to the features in E and

√

ν with respect to the features in E. This ellipse is represented in the left part of Figure 1. If none of these instances belong to the opposite class, we generate instances in the ellipsoidal layer between ν and ν + . This layer is represented in blue on Figure 1.

Experiments

This section presents the experiments conducted to evaluate the KICE algorithm: the purpose of these experiments is to show that the proposed method finds the expected counterfactual instance, thus it allows to achieve a trade-off between the quality and the compatibility of an explanation.

Experimental Protocol

Experiments are conducted on three classical benchmark tabular datasets: halfmoons, Boston and breast cancer. As a pre-treatment, dataset values are normalized and in the case of the Boston dataset, the regression value is transformed into a binary class by means of a discretization step: the price is "expensive" if it is greater than $21, 000, and "cheap" otherwise. Datasets are split into train and test data subsets (80%-20%). Within the considered post-hoc explanation framework, the classifier choice does not matter, we apply SVM with a Gaussian kernel, that obtains good accuracy on the three datasets: half-moons (0.99), Boston (0.98) and breast cancer (0.93). Regarding prior knowledge, we consider the user disposes of less features than the classifier. In order to build somehow realistic knowledge, we train a decision tree with low depth on the train data (more precisely maximal depth is set to half the number of data features), the set of features E is then the ones that occur in this tree. Counterfactual instances e * are then generated for each instance x of the test data set using KICE, with the value of λ being chosen to observe interesting results, respectively 4, 1 and 6 for the datasets half-moons, Boston and breast cancer.

We compare the proposed method KICE to two competitors. The first one is Growing Spheres [START_REF] Laugel | Comparisonbased Inverse Classification for Interpretability in Machine Learning[END_REF], which solves the reference counterfactual optimization problem that only minimizes the Euclidean distance and denote e ref the generated point. This corresponds to an extreme case of aggregation of Eq. 5 where the incompatibility term is ignored i.e. λ = 0. A second competitor is proposed by imposing to strictly comply with the knowledge. Its associated cost function thus minimizes the Euclidean distance according to the knowledge features only, a naive way of integrating knowledge in the explanation.

We denote e user the counterfactual instance that solves the associated problem: We can write this optimization problem in the proposed formulation given in Eq. ( 6) with λ arbitrarily large: In the figure, we observe that the counterfactual instance e ref is the closest point belonging to the opposite class. As expected, e user is further away than e ref and only modifies feature X 1 which is the knowledge feature: we observe that e * is a compromise between e ref and e user . It requires less modifications according to X 0 than e ref , hence it is more compatible. It is also closer to the studied instance than e user . In the graph, on the right, we notice that there is no e user . In this case, there is no counterfactual instance modifying only feature X 1 . The proposed method is then useful because it allows to obtain a more compatible counterfactual than e ref .

e user = argmin
e user = argmin

Evaluation of the KICE Method

In this section, we further compare the proposed method with its presented competitors and study quantitatively the gain in terms of the proposed cost function offered by KICE. 1. Results with the three considered approaches on the three datasets for metrics: penalty defined in Equation ( 2), incompatibility defined in Equation ( 3) and cost defined in Equation [START_REF] Guidotti | A Survey of Methods for Explaining Black Box Models[END_REF] We apply the three methods described in the previous section on the test set of the different datasets. Among the test set, some instances do not find counterfactual examples with the method that modifies only the features in E (e user ). For the datasets half-moons, Boston and breast cancer, this concerns respectively 20%, 0% and 11% of the instances. For the rest of the instances, Table 1 shows the mean and standard deviation values for the penalty, incompatibility and cost functions for the three approaches.

We notice that, as expected, the proposed counterfactual examples have a penalty greater than that of e ref but lower than that of e user . Moreover, the incompatibility function is much smaller than that of e ref .

Finally, e * cost function is the lowest. We notice that the standard deviations are high, which is due the fact that the instances are at different distances from the boundary.

To verify that KICE minimizes the cost function as opposed to the other two on all instances, Figure 3 shows the value of the cost function obtained by e * (that is defined as minimizing it) as compared to the value it takes for e ref (left) and e user (right), for each of the test instances of the half-moons dataset. The figures show that, as expected, all points are above the line y = x.

On the right hand graph, the points are more scattered but they remain above the line. We notice that the generated counterfactual instances are closer to the cost function of e ref than to e user . Only one feature is consider by the knowledge, it is difficult to get a close counterfactual instance only by modifying this feature. It is then necessary to move away from the studied instance to get 100% compatible one. It is also possible to increase λ so that the points are less scattered.

Conclusion and Perspectives

In this paper, a general framework is proposed to help defining an optimization problem to integrate prior knowledge in post-hoc model-agnostic explanations. Using this framework, we proposed a new method, KICE, to generate counter- factual explanations in the knowledge language by minimizing the modifications according to the unknown features.

Future works will include a study of the counterfactuals explanations generated for different values of λ. Thus, we will study the λ associated with the expected counterfactual examples. Another direction will focus on exploring and proposing instantiations of the framework for different models and applications, as well as real world experiments including real users.

Fig. 1 .

 1 Fig. 1. (left) Generation zone for the initial step, (right) Iteration zone.

e∈E x -e 2 EFig. 2 .

 22 Fig. 2. Examples of obtained results e ref , euser and e * for three instances x (+ + +: x, : e ref , : euser, •: e * )

e∈E x -e 2 +λ x -e 2 E 5 . 2

 252 Illustrative ExamplesFirst, we illustrate the behavior of the methods with examples in two dimensions denoted X 0 and X 1 . Figure2shows the counterfactual examples obtained for three different instances of the half-moons dataset. The figure shows the training set, the blue and red regions represent the predicted classes (darker points are the training examples); the decision boundary of the trained SVM classifier is shown in white, it achieves 0.99 accuracy. The considered knowledge system is a rule on a single feature, it is represented by the brown horizontal line : E = {X 1 }.

Fig. 3 .

 3 Fig. 3. Cost functions cost as defined in Equation (6) of e ref , euser and e * for the 80% of the test set for which all three counterfactual instances are defined.