Afef Awadid
email: afef.awadid@irt-systemx.fr

Kahina Amokrane-Ferka
email: kahina.amokrane-ferka@irt-systemx.fr

Henri Sohier
email: henri.sohier@irt-systemx.fr

Juliette Mattioli
email: juliette.mattioli@thalesgroup.com

Faouzi Adjed
email: faouzi.adjed@irt-systemx.fr

Martin Gonzalez
email: martin.gonzalez@irt-systemx.fr

Souhaiel Khalfaoui
email: souhaiel.khalfaoui@valeo.com

AI Systems Trustworthiness Assessment: State of the Art

Keywords: AI-Based Systems, Trustworthiness Assessment, Trustworthiness Attributes, Metrics, State of the Art Review

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Central to Model-based Systems Engineering (MBSE) is "the formalized application of modeling to support system requirements, design, analysis, verification, and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases" [START_REF] Incose | Systems engineering vision 2020. IN-COSE[END_REF]. MBSE, therefore, advocates "the use of models to perform systems engineering activities that are traditionally performed using documents" [START_REF] Mann | A practical guide to sysml: The systems modeling language[END_REF]. This promotes the understanding of complex systems engineering processes including Artificial Intelligence (AI) systems engineering as a multiengineering process [START_REF] Mattioli | Ai engineering to deploy reliable ai in industry[END_REF]. However, the successful application of MBSE requires the assessment of AI trustworthiness defined by the ISO/IEC DIS 30145-2 standard as the "ability to meet stakeholders' expectations in a verifiable way". Indeed, without an accompanying assessment of trustworthiness from the early stages of development, the deployment of an AI component within a safety critical systems such as in avionics, mobility, healthcare and defense becomes risky [START_REF] Mattioli | Towards a holistic approach for ai trustworthiness assessment based upon aids for multicriteria aggregation[END_REF].

In view of this, it is not surprising that the quantification of AI-based system trustworthiness has become a hot topic [START_REF] Braunschweig | The wall of safety for AI: approaches in the confiance.ai program[END_REF]. AI system trustworthiness is defined in terms of characteristics/ attributes such as reliability, safety, and resiliency [START_REF] Ai | A plan for federal engagement in oping technical standards and related tools[END_REF]. In this context, the paper at hand provides a state of the art review of AI trustworthiness assessment. Such review focuses on the main trustworthiness attributes as well as their evaluation metrics, and is based on academic and industrial literature conducted within the Confiance.ai research program.

The rest of the paper is organized as follows. Section 2 introduces the context and motivation of this work. Section 3 presents the state of the art review of AI systems trustworthiness assessment with respect to trustworthiness attributes and their evaluation metrics. Finally, Section 4 concludes the paper and opens up for future work.

CONTEXT AND MOTIVATION

Safety-critical systems, such as those used in avionics, mobility, healthcare, and defense, are designed to operate reliably and safely in dynamic environments where their failure could have severe consequences.

The adoption of Artificial Intelligence (AI) depends on their ability to deliver the expected service safely, to meet user expectations, and to maintain service continuity. Thus, such systems have to be valid, accountable, explainable, resilient, safe and secure, (Mattioli et al., 2023a).

compliant with respect to regulation and standardization (including ethics and sustainability). Assessing the trustworthiness of AI becomes the cornerstone of successful improvement in the design and operation of critical systems. However, obtaining trustworthiness measures remains a challenging task.

On the one hand, measuring trust can help identify problems with the system before they become critical and allow for corrective action to be taken before a failure occurs. On the other hand, measuring trust can help to improve the design of critical systems.

By understanding the factors that contribute to user trust in AI systems, designers can create ones that are more reliable, safe and secure. AI trustworthiness characterization is multi-dimensional and multicriteria as assessed by different stakeholders (Mattioli et al., 2023b) (regulators, developers, end-users). In this context, [START_REF] Felderer | Quality Assurance for AI-Based Systems: Overview and Challenges (Introduction to Interactive Session)[END_REF] proposed to consider three dimensions; the artifact type dimension (system, model and data perspective), the process dimension and the quality characteristic attributes, based on ISO/IEC 25023, that bear on software product or system quality.

AI-based systems, especially those using machine learning (ML), add a level of complexity to traditional systems, due to their inherent stochastic nature. Thus, to take into account the complexity of the ML-based systems engineering process, the set of trustworthiness properties illustrated in Figure 1 (Mattioli et al., 2023a), needs to be extended. Additionally, various experts and stakeholders are involved in the design of such systems.

Moreover, to assess AI trustworthiness, the choice of the relevant attributes is not easy, since the selection pertains to the context of application, which is modeled according to several elements (ODD, intended domain of use, nature and roles of the stakeholders...). The attributes can be quantitative (typically numerical values either derived from a measure or providing a comprehensive and statistical overview of a phenomenon) or qualitative (based on the detailed analysis and interpretation of a limited number of samples). Then once the list of relevant attributes has been defined, the aggregation of several attributes remains complex due to commensurability issues: indeed, this is equivalent with combining "oranges and apples", none of the attributes having the same unit. In addition, one aims at making trade-offs and arbitration between the attributes. This means that the value of each attribute should be transformed into a scale common to all attributes and representing the preferences of a stakeholder, and that the values of the scales for the different criteria should be aggregated. These elements constitute the main steps for solving the problem using a multi-criteria decision making (MCDA) approach.

MCDA is a generic term for a collection of systematic approaches developed specifically to help one or several decision makers to assess or compare some alternatives on the basis of several criteria [START_REF] Labreuche | A general framework for explaining the results of a multi-attribute preference model[END_REF]. The difficulty is that the decision criteria are frequently numerous, interdependent/overlapping and sometime conflicting. For example, effectiveness may be conflicting with robust-ness, explainability, or affordability. The viewpoints are quantified through attributes.

Aggregation functions are often used to compare alternatives evaluated on multiple conflicting criteria by synthesizing their performances into overall utility values [START_REF] Grabisch | A decade of application of the Choquet and Sugeno integrals in multicriteria decision aid[END_REF]. Such functions must be sufficiently expressive to fit the stakeholder's preferences, allowing for instance the determination of the preferred alternative or to make compromises among the criteria -improving a criterion implies that one shall deteriorate on another one.

MCDA provides a tool to specify the good compromises [START_REF] Labreuche | A general framework for explaining the results of a multi-attribute preference model[END_REF]. Our approach is based on the following steps: Given this, to improve trustworthiness, assessment, measures and processes are needed. Moreover, context, usage, levels of safety and security, regulations, (ethical) standards (including fairness, privacy), certification processes, and degrees of liability should be considered. In addition to measures and processes, various techniques and methodologies such as testing, evaluation, and validation of the system's performance against specified criteria, expert review, and stakeholder participation are required for trustworthiness assessment in AI-based critical systems.

Such assessment should be ongoing, with regular updates and monitoring of the system's performance and compliant with standards and regulations. Besides trustworthy attribute definitions [START_REF] Adedjouma | Towards the engineering of trustworthy AI applications for critical systems -the confiance[END_REF], the current work focuses on some examples of associated metrics that help to identify potential areas for improvement. It is important to note in this context that such attributes have a different meaning depending on the stakeholder's profile. For instance, a system engineer, a safety engineer, a data engineer, and an AI scientist may all have distinct perspectives on accuracy.

A NEW AI TRUSTWORTHINESS META-MODEL

A trustworthy software is defined [START_REF] Wing | Trustworthy ai[END_REF]) by a combination of overlapping properties: reliability, safety, security, privacy, availability and usability. For a ML-based system, this translates and extends to accuracy, robustness, fairness, accountability, transparency, explainability and ethics. [START_REF] Delseny | White paper machine learning in certified systems[END_REF]) also considers auditability.

To capture the type of considered information and the different inter-relations needed to assess ML trustworthiness, we proposed a meta-model with concepts in different abstraction levels (see Figure . 2). The red part describes the way the tree of attributes is built. It highlights the abstract concepts central to trustworthiness assessment. An attribute which aggregates other attributes is called a macro-attribute (e.g. robustness, dependability, etc.). It is assessed with an aggregation method. An atomic attribute (leaf attribute) is assessed with a clear and actionable observable which can take different forms (metric, "expected proof").

The green part of Figure . 2 is the meta-model fragment with concrete concepts. These concepts represent the different possible subjects and relations between them. For example, the product is developed following processes as technical processes (through which the product must go: design definition, implementation, operation, ...), agreement processes (with external organizations: acquisition, supply), and management processes (supporting the development of the product: quality management, risk management, etc.). Risk and quality management ensures the compliance with the specification which includes the different expected trustworthiness attributes. Processes are applied with tools by people respecting a certain governance.

The blue part summarizes systems engineering key concepts more precisely part of the nonfunctional specification: they do not define what the system "does" or how the system works, but what the system "is". The attributes are also commonly referred to as "-ilities" as they often have this suffix. They can also be referred to as quality requirements. Whether a specification is functional or nonfunctional, it is influenced by stakeholders such as the user, the operator, the developer, etc.

As opposed to non-functional requirements which define what the system is, functional requirements define what the system does: should it move? roll? roll fast? under what conditions? From this point of view, the Operational Design Domain (ODD), which characterizes the conditions of operation of the system, can be considered part of the functional specification relating to trustworthiness attributes in different ways: 1) Having transparency or clear visibility into the ODD permits to understand the system's capabilities and limits (which is part of the AI Act's requirements); 2) The ODD is the domain to consider for the different operational trustworthiness attributes; 3) Figure 2: A new AI trustworthiness meta-model [START_REF] Mattioli | Towards a holistic approach for ai trustworthiness assessment based upon aids for multicriteria aggregation[END_REF] The ODD has its own attributes (it should be complete, free of inconsistencies, human readable, etc.).

In contrast to non-functional requirements, which define what the system is, functional requirements define what the system does: does it move? does it roll? does it roll fast? under what conditions? From this point of view, the Operational Design Domain (ODD), which characterizes the operating conditions of the system/feature of interest, can be considered as part of the functional specification in relation to the reliability attributes in a number of ways: 1) the transparency of the ODD makes it possible to understand the limitations of the system (a requirement of the AI Act); 2) the ODD is the domain to be considered for the different operational reliability attributes; 3) the ODD has its own attributes (it should be complete, free of inconsistencies, human readable, etc.).

Thus, the trustworthy attributes can be assessed only if the ODD is clearly defined but many AI prototypes neglect to describe their ODD or leave it vaguely defined as the domain covered by the distribution of data used during training. In addition to the set of Requirements applicable to the System, one of the results of the System Specification phase is the ODD Definition/Specification, that aims at specifying the sub-domain where automation features are expected to operate according to their requirements, among the whole operational domain of At every stage of the system lifecycle, from engineering and design to operation, trustworthiness relationships must be established and maintained. According to the seven pillars of reliability (High-Level Expert Group on Artificial Intelligence, 2019), Confiance.ai specifies AI reliability (Mattioli et al., 2023a) by six macro-attributes: data/information/knowledge quality, dependability, operability, robustness, explainability/interpretability, and human control.

Through the system development, the desired ODD will be refined to fit the needs and constraints from different engineering. Here comes the question about where the ODD process stops. Figure 4 extends the ODD approaches with concepts related to ODD limits. As an example, the ODD is an important input for ML training, ML monitoring, etc. To be able to address the concerns of these engineering fields, one needs to define datasets, to define test scenarios, to identify the indicators to measure the algorithm performance, etc. A current expectation is that the ODD artefact must include all the features for deriving those elements (scenario definition, robustness metric, monitoring variables). However, following the definition of the ODD, it is not the case. Besides, that information is required by the engineering fields in general, regardless if the system is AI-based or not AI-based, which let induces that such information may come from another source.

AI-BASED SYSTEMS TRUSTWORTHINESS ASSESSMENT

Data Quality Assessment

In ML discipline, most of the research is focusing on model performance improvement more than on datasets [START_REF] Mazumder | Dataperf: Benchmarks for data-centric ai development[END_REF]. In the recent decade, ML techniques have advanced significantly and achieved a high maturity level [START_REF] Adedjouma | Towards the engineering of trustworthy AI applications for critical systems -the confiance[END_REF]. Classical ML practices consist typically in using the existing datasets and in leveraging performances challenges through techniques complexity enhancement. In the other hand, data-driven AI takes a broader approach by placing a greater emphasis on the data itself [START_REF] Jakubik | Data-centric artificial intelligence[END_REF][START_REF] Jarrahi | The Principles of Data-Centric AI[END_REF]. Instead of simply looking for patterns and relationships within the input features, data-driven AI involves collecting, processing, and analyzing large amounts of data to create more accurate and robust models [START_REF] Mattioli | Empowering the trustworthiness of ml-based critical systems through engineering activities[END_REF]. Moreover, a real challenge today is to associate datasets to the Operational Design Domain (ODD) 1 1 An ODD is concept created initially for automated driving system (ADS) used to restrict where the ADS is valid [START_REF] Gyllenhammar | Towards an operational design domain that supports the safety argumentation of an automated driving system[END_REF]. In the current work, ODD is a restriction of the domain where an AI-based system acts safely.

from the operational level of the system definition. Indeed, these datasets include several factors such as user needs [START_REF] Chapman | Dataset search: a survey[END_REF] and related metadata. Moreover, [START_REF] Mountrakis | Assessing reference dataset representativeness through confidence metrics based on information density[END_REF] highlights that dataset quality may have a more significant impact on performance than any model design choice. Many industrialization crisis often result from the data used to train the models instead of the model designs and architectures.

Without a systematic assessment of their quality, data-driven AI risks losing control of the various steps of data engineering such as collection, annotation and feature engineering. Doing without data quality assessment would result in assuming that data engineering can not be further improved and that problems will always be detected without systematic analysis. Thus, in a given end-to-end AI-based system process, the data quality assessment brings an evaluation of some ODD description aspects. These evaluation goes through a set of metrics, illustrated in Figure .1, such as data accuracy, data representativeness and data diversity.

Furthermore, to ensure conformity to the ODD specifications, well-founded metrics assess the reached data quality level. Both research and industrial practices have developed relevant data quality metrics in the AI-based system, such as accuracy and completeness. However, many of them still lack a sound foundation [START_REF] Heinrich | Requirements for data quality metrics[END_REF]. Thereafter, a definition and a brief technical description of five metrics for data quality assessment are given.

Data completeness for ML datasets refers to the degree to which it contains the necessary information required to accurately model the underlying patterns by the learning algorithm. Measuring dataset completeness includes evaluation of the amount of missing items, outliers and errors. Completeness metric could be based on the Ge and Helfert's ratio [START_REF] Ge | A framework to assess decision quality using information quality dimensions[END_REF] defined as: data completeness = ∑ N i=1 γ(d i )/N, where γ(d i ) is 0 if d i is a missing data, and 1 otherwise.

Data correctness refers to the accuracy of the data items to faithfully represent the real-world phenomena or objects they meant to capture. Dataset correctness could be defined as: data correctness = 1/(1+d(ω, ω m )) where ω is the data value to be assessed, ω m is the corresponding real value and d is a domain-specific distance measure such as the Euclidean or Hamming distance.

Data diversity is defined by the evaluation of the presence of all required information and quantifies how the dataset fits the environment and application domains described in the specifications. During ML model design, training and testing, the level Figure 4: The ODD analysis process of diversity should be equally distributed for the different data subsets being selected. This should ensure that the ML model is enough diversified so as to cover its domain of possible stimuli. According to [START_REF] Gong | Diversity in machine learning[END_REF], the only metric used for diversity for supervised ML is the Determinantal Point Process (DPP) introduced by [START_REF] Kulesza | Determinantal point processes for machine learning[END_REF]. Then, [START_REF] Dereziński | Fast determinantal point processes via distortion-free intermediate sampling[END_REF] regularizes the DPP (R-DPP) to accelerate the training process. Moreover, other diversity indexes, used in biology and ecology, could be adapted for ML models such as Shannon entropy and mean proportional species abundance [START_REF] Tuomisto | A diversity of beta diversities: straightening up a concept gone awry. part 1. defining beta diversity as a function of alpha and gamma diversity[END_REF].

Data representativeness [START_REF] Mamalet | White Paper Machine Learning in Certified Systems[END_REF]) refers in statistics to the notion of sample and population. Transposed to AI, the sample corresponds to the data-set available for the development of the model (training, validation, testing), and the population corresponds to all possible observations in the field of application. Moreover, a dataset is representative when it describes the environment observations, and the distribution of its key characteristics is conform to the specifications need, requirements and the ODD of the targeted application. There are multiple existing methods to quantify the representativeness of datasets, stemming from statistics and ML fields. Indeed, Student, Chi-square and Kolmogorov-Smirnov tests may be applied to assess the goodness of fit of specified distributions. Furthermore, in case of large datasets, the confidence interval combined with the maximum entropy probability could be used to determine, in terms of dataset size and acceptance thresholds, the suitable dataset for ML need [START_REF] Blatchford | Determining representative sample size for validation of continuous, large continental remote sensing data[END_REF].

Operability Assessment

By assessing operability, we can ensure that we deliver value to end-users and avoid problems at designtime, where operability is the ability to keep such AIbased system in a safe and reliable functioning condition, according to predefined operational requirements. Thus, (AI-based system) operability is a measure of how well the system works in a production environment, for both end users and developers. Good operability induces diagnosis and recovery for AI maintenance. In an operational context, it is also defined as the degree to which a product or system is easy to use, monitor, control and maintain and to use. Thereby, accuracy, trueness and precision are required for AI/ML operability assessment and considered as different concepts when referring to measurements.

Generally speaking, accuracy refers to how close a measured value is in relation to a known value. However, the ISO (International Organization for Standardization) uses trueness for the above definition while keeping the word accuracy to refer to the combination of trueness and precision. On the other hand, precision is related to how close several measurements of the same quantity are to each other. Thus, (model) accuracy is a fundamental metric for evaluating AI-based critical system, to measure how well the system performs its intended function compared to its ground truth or expected output.

Traditional operability metrics for regression include Mean Squared Error (MSE) or Mean Absolute Error (MAE), while classification problems can be evaluated through precision, accuracy and recall [START_REF] Davis | The relationship between precision-recall and roc curves[END_REF]. In classification, a confusion matrix (depicting the distribution of true/false negatives/positives for each class) is a practical tool for visualizing of the errors, and allows the computation of most metrics: precision, recall, sensitivity, specificity, F1 score, Calibration measures how well the AI system's predicted probabilities match the true probabilities of the outcomes. This can be evaluated using various metrics, such as the Brier score or calibration plot.

Let us denotes by T P (True Positive) to represent how many positive class samples your model predicted correctly; T N (True Negative) to represent how many negative class samples your model predicted correctly; FP (False Positive) to represent how many positive class samples your model predicted incorrectly and FN (False Negative) to represent how many negative class samples your model predicted incorrectly.

• Accuracy measures how often the Model produces correct results where Accuracy = (T P + T N)/(T P + FP + T N + FN).

• Precision measures the proportion of true positives out of all positive predictions: Precision = (T P)/(T P + FP).

• Recall measures the proportion of true positives out of all actual positives: Recall = T P/(T P + FN)

• F1 Score is a combination of precision and recall providing a single score to evaluate the overall performance of the AI system: F1 Score = 2 * (Precision * Recall)/(Precision + Recall).

• Specificity is the proportion of actual negatives that the model has correctly identified as such out of all negatives: Speci f icity = T N/(FP + T N)

• ROC Curve plots the true positive rate against the false positive rate at various classification thresholds, and can be used to evaluate the overall performance of the AI system. The area under the ROC curve is often used as a metric, with higher area indicating better performance.

Closely related to accuracy, trueness, and precision is correctness. Indeed, the latter is defined by ISO-25010 as the degree to which a product or system provides the correct results with the needed degree of precision. In ML, correctness measures the probability that the ML system under test "gets things right". Let D be the distribution of future unknown data. Let x be a data item belonging to D. Let h be the ML model that we are testing. h(x) is the predicted label of x, c(x) is the true label. [START_REF] Zhang | Machine learning testing: Survey, landscapes and horizons[END_REF]) defined the model correctness E(h) as the probability that h(x) and c(x) are identical, E(h)=Pr x∼D [h(x)=c(x)]. Note that there are many other metrics that can be used to assess the operability of AI systems, and the choice of metric(s) depends on the specific use case and application.

Dependability Assessment

As AI becomes prevalent in critical systems, their dependability takes on increasing importance. In systems engineering, dependability can be defined as the ability of a system to deliver a service that can be justifiably trusted [START_REF] Avizienis | Basic concepts and taxonomy of dependable and secure computing[END_REF]. But, this concept has evolved to integrate other attributes: Availability readiness for correct service; Reliability continuity of correct service; Safety for absence of catastrophic consequences on user(s) and environment; Security availability for authorized users, confidentiality, and integrity; Confidentiality for absence of unauthorized disclosure of information; Integrity for absence of improper system alterations; and Maintainability for ability to undergo modifications, and repairs. Moreover, the requirements on the AI system cannot be described completely, and the system must function dependably in an almost infinite application space. This is where established methods and techniques of classical systems and software engineering reach their limits and new, innovative approaches are required. A core element to assess dependability is to provide assurance [START_REF] Buckley | Software quality assurance[END_REF] that the system as a whole is dependable, i.e., that risk of failures is mitigate to an acceptable level. In a data-driven AI component, the functionality is not programmed in the traditional way, but created by applying algorithms to data. One challenge is to make this (learning) assurance case [START_REF] Byun | Manifold for machine learning assurance[END_REF] cleanly and to find appropriate evidence that demonstrates the dependability of the AI system.

• Availability and reliability are often used interchangeably but they actually refer to different things. Reliability refers to the probability of an AI-based component/system performing without failure under normal operating conditions over a given period of time. Thus, availability measure provides an indication of the percentage of the time that the system is actually available over the scheduled operational time. The first step in calculating availability is deciding the period we want to analyze. Then, it is calculated by dividing Uptime by the total sum of Uptime and Downtime: Availability = Uptime/(Uptime + Downtime), where Uptime (resp. Downtime) represents the time during the system is operational(resp. isn't operational). Downtime has the biggest impact on availability and is one of key KPIs for maintenance and in service support activities. Moreover, estimating AI-based software MTBF (Mean time between failure) is a tricky task. This interval may be estimated from the defect rate of the system or can also be based on previous experience with similar systems.

• Reliability is the probability that an asset will perform a required function under specified conditions, without failure, for a specified period. For AI systems, the definition of AI reliability is defined as [START_REF] Kaur | Software reliability, metrics, reliability improvement using agile process[END_REF]) "the probability of the failure-free software operation for a specified period of time in a specified environment".

Common measurements of reliability are MTBF and mean time to failure (MTTF). MTBF measures the average time between two consecutive failures, while MTTF accounts for the time elapsing from the beginning of operation to the detection of the first failure. Nevertheless, the measurement of the reliability of an AI algorithm is associated to its performance. Most data-driven AI algorithms are designed to solve problems of classification, regression, and clustering, [START_REF] Bosnić | An overview of advances in reliability estimation of individual predictions in ML[END_REF] used prediction accuracy from ML algorithms as a reliability measure.

• Repeatability and reproducibility are also characteristics of dependability. In the context of AI engineering, repeatability measures the variation in various runs of test plan under the same conditions, while reproducibility measures whether an entire experiment can be reproduced in its entirety. This verification facilitates the detection, analysis, and mitigation of potential risks in an AI system, such as a vulnerability on specific inputs or unintended bias. Therefore, reproducibility is emerging as a concern among AI Engineers.

Moreover, depending on the AI methods used, functional safety can still be measured and verified, insofar as such properties can be formally defined. For example, ML dependability properties have to be entirely verified in the field of aviation following the "EASA Concept Paper: guidance for Level 1 & 2 ML applications".

Robustness Assessment and Monitoring

AI-based critical systems should be robust, secure and safe throughout their entire life-cycle in conditions of normal use, foreseeable use or misuse, or other adverse conditions, they function appropriately and without unreasonable safety risk. To this end, robustness is mandatory to ensure that an invalid input data will not lead to an unsafe state of the system. This can be reached "by-design" and it can also be monitored "in operations" to enable analysis of the AI system's outcomes and responses appropriate to the context. Therefore, robustness and monitoring are two closely related topics in an AI-based system lifecycle. Robustness and stability are defined by [START_REF] Mamalet | White Paper Machine Learning in Certified Systems[END_REF] as an AI-based system's global robustness (out of distribution), the ability to perform its intended function in the presence of abnormal or unknown inputs; and local robustness (in distribution), the extent to which the system provides equivalent responses to similar inputs.

These definitions are made more precise by (SAE J3016, 2018) using the ODD concept. The global robustness is then called robustness and the local robustness is called stability where robustness is an AI asset's ability to maintain its expected/intended performance under well-characterized abnormalities or deviations in inputs and operating conditions under its ODD; and stability is the ability of an AI asset to maintain its expected/intended output(s) under wellcharacterized and bounded perturbations to its inputs and operating conditions within its ODD.

In addition, adversarial robustness refers to the ability of models to maintain their performance in the face of adversarial attacks and perturbations where perturbations are imperceptible, non-random changes to the input that alter a model's prediction, thereby maximizing its error [START_REF] Kapusta | Protecting ownership rights of ml models using watermarking in the light of adversarial attacks[END_REF]. Some tooled methods dedicated to local robustness assessment are based on evaluation or (formal) demonstration such as:

• Non-overlapping corruption on a dataset provides an assessment of the robustness of a given AIbased model [START_REF] Py | Real-time weather monitoring and desnowification through image purification[END_REF];

• AI Metamorphism Observing Software (AIMOS) [START_REF] Girard-Satabin | CAISAR: A platform for Characterizing Artificial Intelligence Safety and Robustness[END_REF] assesses metamorphic properties on AI models such as robustness to perturbations on the inputs but also relation between models' inputs and outputs;

• Time-series robustness characterization focuses and the assessment of the robustness w.r.t. perturbations on the inputs of regression models applied to time series;

• Adversarial attack characterization: [START_REF] Kapusta | Protecting ownership rights of ml models using watermarking in the light of adversarial attacks[END_REF] evaluates the impact and usability of adversarial attacks on AI models;

• Amplification methods evaluate the robustness of models using amplification methods on the dataset with noise functions.

Monitoring comprises methods for inspecting system in order to analyze and predict its behavior.

Enforcement, on the other hand, involves designing mechanisms for controlling and restricting the behavior of systems. Once an AI systems are deployed, we need tools to continuously monitor and adjust them. Thus, the main objective of online monitoring of AI models is to identify the output that does not fulfill the expectations by detecting any deviation in operation from the specified expected behavior, or from a predefined set of trustworthy operational properties [START_REF] Kaakai | Towards multi-timescale online monitoring of ai models: Principles and preliminary results[END_REF]. In Confiance.ai program, [START_REF] Adedjouma | Towards the engineering of trustworthy AI applications for critical systems -the confiance[END_REF] addresses both concepts, monitoring and enforcement, by combining several monitoring timescales (Present Time, Near-Past and Near-Future Monitoring) -with a rule-based approach to compute the final "safe output".

Explainability Assessment

The need to explain AI algorithms gave rise to the field of Explainable AI. In the literature, several studies argue that explanations positively affect user trust [START_REF] Biran | Explanation and justification in machine learning: A survey[END_REF] and inappropriate trust impairs human-machine interaction [START_REF] Ribeiro | Explaining the predictions of any classifier[END_REF]. For example, in data-driven AI, explainability is a main property to bring trust to models, given the black box nature of AI. This property is related to the notion of explanation as an interface between humans and AI. It involves AI systems that are accurate and understandable to people [START_REF] Philippe | Explainability benchmark v2 -the confiance[END_REF]. However, explanations do not necessarily have to provide accurate information about the algorithm of the ML process. In our study, we consider three key dimensions of explainability:

• Interpretability: assesses how easily human experts can understand the internal workings of an AI system; interpretable explanations need to use a representation that is understandable to humans, regardless of the actual features used by the model [START_REF] Ribeiro | Explaining the predictions of any classifier[END_REF]. In the context of ML systems, interpretability is defined as the ability to explain or to present in understandable terms to a human (Doshi-Velez and Kim, 2017);

• Fidelity measures how well the explanations provided accurately reflect the AI system behavior [START_REF] Yeh | On the (in) fidelity and sensitivity of explanations[END_REF]. Fidelity metrics measure the efficiency of the methods to explain models. Fidelity is also defined [START_REF] Plumb | Regularizing black-box models for improved interpretability[END_REF], when the explainer's output space is (ε s , (ε s := (g ∈ G|g : X → Y )), the explanation is defined as a function g : X → Y , and it is natural to evaluate how accurately g models f in a neighborhood

N x : F( f , g, N x ) := E x ′ ∼N x [(g(x ′ ) -f (x ′ )) 2 ]
which refer to the neighborhood-fidelity (NF) metric. This metric is sometimes evaluated with N x as a point mass on x, this version is called the point-fidelity (PF) metric.

• Usefulness: evaluates how effectively the explanations support human decision-making and action. This last dimension is qualitative. In the evaluation context, some questions can be asked to the user [START_REF] Tambwekar | Towards reconciling usability and usefulness of explainable ai methodologies[END_REF]: Using this explanation would be useful for me? Using this explanation will improve my effectiveness. Using this explanation will improve my performance.

• Faithfulness: measures the degree to which an interpretation method accurately reflects the reasoning of the model it interprets. It is important to note that explanations provided by an unfaithful method can conceal any biases that exist in the model's judgments, which may result in unwarranted trust or confidence in the model's predictions. Faithfulness is calculated using the following formula [START_REF] Du | On attribution of recurrent neural network predictions via additive decomposition[END_REF]: Faith f ulness = 1/N ∑(y x iy x i |a ), where y x i is the predicted probability for a given target class using the original inputs, and y x i |a is the predicted probability for the target class for the input with significant sentences/words removed. According to [START_REF] Arya | AI Explainability 360: Impact and design[END_REF], faithfulness is the inverse of the Pearson Product-Moment correlation and ranges from -1 to 1. A negative correlation of 1 indicates a perfect correlation, a positive correlation of -1 indicates the inverse, and 0 indicates no correlation. Faithfulness is calculated as follows: Faith f ulness = -σ xy /(σ x + σ y ), where σ 2 x (resp. σ 2 xy ) represents the variance of x (resp. the co-variance of (x, y)). This metric can be interchangeable with Fidelity metric in some methodes.

• Monotonicity: applies only to some explainable methods. It consists in progressively adding the values of x to a null vector, then looking if the probability of predicting the correct class with it is increasing [START_REF] Ribeiro | Explaining the predictions of any classifier[END_REF]; The interest in studying monotonicity in the context of MBSE lies in its ability to enhance the understanding and analysis of complex systems.

• Sensitivity: measures the degree of explanation changes to subtle input perturbations using Monte Carlo sampling-based approximation [START_REF] Yeh | On the (in) fidelity and sensitivity of explanations[END_REF].

Human-centered Quality & Human Oversight Assessment

To ensure trustworthy AI, it is important to go beyond the AI model itself (inputs, features and outputs) and consider dynamics of the model interacting with the overall system, including end-users. Humancentered quality involves meeting requirements for "usability, accessibility, user experience, and avoiding harm from use".

From such perspective, trustworthy AI should be both usable and explainable, meaning that it should not stop working at inappropriate times (which could create safety risks) and should be user-friendly for individuals with diverse backgrounds. Moreover, trustworthy AI must allow for human explanation and analysis to mitigate risks and empower users, as well as transparent to promote understanding of its workings mechanism. Human agency and oversight means that AI systems shall be developed and used as a tool that serves people, respects human dignity and personal autonomy, and being under human control and oversight. In that context, ethics guidelines for trustworthy AI were written by High-Level Expert Group on AI (High-Level Expert Group on Artificial ligence, 2019). The guidelines have 4 ethical principles: (1) Respect for human, (2) Prevention of harm, (3) Fairness, (4) Explainability; and seven key (ethical) requirements, among it, we can mention:

• Privacy: For IEEE-7000 privacy means that collection with unsolicited surveillance, processing with unexpected and unsolicited personal data aggregation, and the dissemination of personal information is carried out in such a way that it preserves the self-determination of the person with regard to information (breach of confidentiality, disclosure) and that any form of invasion is prevented (intrusion against the will). In practice, collection implies that data acquired are cleaned of private information. Once stored, the cleaned data may still fall within the scope of privacy when crossed with other data. Privacy rules must be explicit and respected throughout the data life cycle. When data is crossed and processed, information must be anonymized. This implies that data remains coherent, and that representativeness, diversity, and completeness are preserved.

In [START_REF] Fjeld | Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI[END_REF] eight principles of privacy are highlighted: control over the use of data, ability to restrict data processing, right to rectification, right to erasure, privacy by design, and recommends data protection laws, and privacy (other/general). • Respect for fundamental rights: During human-machine interaction, the machine is perceived as a) attentive, by replying in a reasonable amount of time, and b) responsive, by respecting user privacy, with appropriate decision criteria, transparency, fairness and politeness. For instance, fairness is unsatisfied when biases were introduced during model training in case of unreliable sources or distribution shifts over the time; transparency is neither met when data for learning were suppressed nor traced. Like the other data quality attributes, the definition of respect requirements with their thresholds is essential to regularly assess data and dataset quality -what must be considered at the beginning of the development of an AI system (High-Level Expert Group on Artificial Intelligence, 2019).

CONCLUSIONS AND PERSPECTIVES

This paper highlights the importance of assessing AI trustworthiness in the context of Model-based System Engineering (MBSE) for the development of AIbased systems. The complexity and uncertainties associated with AI necessitate a comprehensive evaluation of trustworthiness attributes and corresponding evaluation metrics. The state of the art review presented in this paper provides insights into the various trustworthiness attributes that need to be considered when assessing AI systems. These attributes include data quality, robustness, and explainability, among others. Each attribute plays a crucial role in ensuring the reliability, safety, and ethical implications of AI systems. The review is based on a thorough analysis of academic and industrial literature conducted within the Confiance.ai research program. This ensures that the findings are grounded in both theoretical and practical perspectives, making them relevant and applicable to real-world scenarios.

By considering the trustworthiness attributes and evaluation metrics identified in this review, MBSE practitioners can effectively assess the trustworthiness of AI-based systems. This assessment is essential for mitigating risks, addressing uncertainties, and building confidence in the deployment and utilization of AI technologies.

However, it is important to note that the field of AI trustworthiness assessment is rapidly evolving, and new attributes and evaluation metrics may emerge in the future. Therefore, future research will focus on keeping up with the advancements in AI technology and hence extending this work to include other trust-worthiness attributes and metrics.

Figure 1 :

 1 Figure 1: Trustworthy AI-based critical systems (Mattioli et al., 2023a).

1.

  Step 1: Structuring attributes in a semantic tree; 2. Step 2: Identification of numerical evaluations; 3. Step 3: Adapting attributes for commensurability; 4. Step 4: Definition of an aggregation methodology to capture operational trade-offs and evaluate higher-level attributes.

Figure 3 :

 3 Figure 3: Engineering Activities for "System Specification" from AI perspective the system/product. The diagram structures presented in fig. 3, the engineering activities needed to perform System Specification with AI/ML involvement in mind.At every stage of the system lifecycle, from engineering and design to operation, trustworthiness relationships must be established and maintained. According to the seven pillars of reliability (High-Level Expert Group on Artificial Intelligence, 2019), Confiance.ai specifies AI reliability(Mattioli et al., 2023a)

ACKNOWLEDGEMENTS

This work has been supported by the French government under the "France 2030" program, as part of the SystemX Technological Research Institute within the Confiance.ai Program (www.confiance.ai).


Olivier Martin
email: olivier.martin@gipsa-lab.grnoble-inp.fr

Emmanuel Witrant

Sufiyan N'yo

Balance control adjustment when load impact perturbation -A transfer function model-based analysis

Keywords: balance control, load perturbation, reactive adjustment, transfer function model

Introduction

Balance stability mainly depends on the central integration of the sensory signals to compute corrective motor commands and maintain the standing equilibrium. Whatever the postural perturbation, selfgenerated or externally triggered, the on-line sensorimotor integration determines the predictive or reactive adaptive responses to maintain the wholebody stability during perturbation. This study focuses on (1) the effects of an external load perturbation on the reactive postural control, triggered in different disturbance conditions, and (2) the comparison of the real balance data to the simulated data, obtained from the data set using a system identification method. The goal of this study is to relate a balance control dataset obtained with several subjects to a linear (after some transformations) dynamical system parameterized as an autoregressive model with exogenous input (ARX). A similar approach was proposed for vestibular patients [START_REF] Voda | Visual-vestibular compensation in balance recovery: A transfer function model-based analysis[END_REF]. In order to estimate the level of similarity between the real and modelized data and estimate the reliability of the modeling approach to varied or altered balance condition, as met in everyday life, sports, or therapy, different conditions of postural stability (variable stance), vision (available or not), and balance control expertise (Tai-Chi skill or not) [START_REF] Gatts | How Tai Chi improves balance: Biomechanics of recovery to a walking slip in impaired seniors[END_REF] was experienced.

Methods

The experimental conditions and data acquisition were obtained in 2019 and are briefly described here (see N'Yo, 2019, Master's theses, for more details).

Participants.

Four males and one female, aged 25 to 56 y.o., participated in the experiment and gave their informed consent according to the ethics research standards. Two subjects were Tai-Chi expert, three had no prior experience.

Postural task

Participants had to stand upright on a force plate, with a Tai-Chi posture with a protective-like attitude, i.e., arm-hand segment placed forward, at waist level. The postural task consisted to control, as well as possible, the disturbed whole-body equilibrium, due to a load perturbation applied at hand level by a passive inertial pendulum device (Fig. 1). It produces a sudden load impact on the hand-arm of the subject. The load was instantaneously discharge after impact. Subjects had to stabilize their balance, as quickly as possible, despite the charge disturbance.

Protocol

The standardized postural tasks sequence consists in a STANCE x VISION x CHARGE crossed-protocol: two STANCE conditions (Normal or Tai-Chi stance), repeated in two VISION conditions (Open-or Closed-Eyes), repeated with two CHARGE ("No load" on the pendulum, or "with load", i.e., with an additional 5kg on). This generates a trial-dependant kinetic energy (work) when the load-perturbation is applied on the subject's hand-arm segment. The tasks sequence is ordered as follow, with 10 trials per task: 1. Normal Stance, Closed Eyes, No load (NSCENL), 2. Normal Stance, Opened Eyes, No load (NSOENL), 3. Tai-Chi Stance, Closed Eyes, No load (TSCENL), 4. Tai-Chi Stance, Closed Eyes, Load (TSCEWL), 5. Tai-Chi Stance, Opened Eyes, No load (TSOENL), 6. Tai-Chi Stance, Opened Eyes, Load (TSOEWL).

Data acquisition

A MOCAP Qualysis system tracks both (1) the loadpendulum trajectory to calculate the perturbation force value in terms of kinetic energy (J), its trajectory and the impact time, and (2) the hand-arm and whole-body anatomical landmark (fig. 1). A Kistler force plate records the postural reaction at feet level to compute the resultant force from the 6 DoF components (Fig. 1). The electromyograms (EMGs) activities for elbow and ankle flexor and extensor muscles are recorded but not presented in this paper. All data acquisitions are timesynchronized.

Data analysis

The relation between the perturbation characteristics and the postural behaviour is characterized by an ARX transfer function model (Fig. 1, large black line). The input parameters are the values of the perturbation characterized by the kinetic energy of the charge applied on the hand-arm complex at the impact time (see fig. 1, impact work at time 0). The output parameters correspond to the postural reaction force values recorded at the platform level (see Fig. 1, force plate values). The transfer function is determined for each specific postural task (condition of perturbation).

Results and discussion

Results

Behavioural analysis. The comparative analysis of the work vs. force plate time-series data for all the postural conditions (Fig. 2) shows that the postural reaction force is functionally correlated to the intensity of the balance perturbation. The increasing of kinetic energy applied during the standing posture increases significantly the reaction force at feet level. More specifically, time-series data show both a main and an interdependent effect of the STANCE, VISION and CHARGE conditions on the postural reaction force: the disequilibrium compensatory are specifically correlated to the stance, the 'Tai-Chi stance' leading to a consistent regularity of the feet reaction force, favoured by either the 'Open Eye' or the 'No load' condition. Transfer function analysis. The result obtained from the transfer function model (TF) shows that the TF parameters fits well with the behavioral data (Fig. 2), as a representative mean estimation of the overall trials effect for each condition of stance, vision, and charge. Reasonable fits for each position and for each subject are obtained with ARX models of order 10 and by choosing the optimal fit (minimizing the least squares error) for a time-delay varied between 10 and 200 ms. Results show that the input work fits accurately either in amplitude or temporally with the output force. They also show that the complexity of the body dynamics, for a given subject and for a given posture, can be captured by a simple linear model with a reasonable number of degrees of freedom.

Discussion

This study demonstrates that the corrective processing of disequilibrium can be modelized by the input-output identification. The TF from ARX-model is able to characterize the effectiveness of the reactive postural adjustments. At a neuro-biomechanical level, results show that the postural adjustments are correlated to the level of the control balance constraints and perturbation, which trig instantaneous dynamic neuromotor reaction correlated to the postural characteristics of the stance, vision, and load. These on-line control strategies counteract the charge impact to correct the disturbed upright standing. The TF model is able to characterize the context-dependent evolution of the reactive postural adjustments. Hence, the identification parameters use in the TF model allows to clearly characterize the impact of all the postural conditions on postural motor commands.

Conclusions

The use of TF for a deeper analysis of the inner signs of the postural control processes is of great interest for study on balance impairment, restoration, or optimization. Exploring more efficiently the identification parameters could give us additional knowledge to make further assumption regarding the adaptive processes of balance control.

Figure 1 .

 1 Figure 1. Left frame: subject standing on the force plate, facing the load-perturbation device during the control postural task. Right frame: comparison plot of the input (work, blue left scale) and output data (postural response, red right scale). In black, the simulated curve from the transfer function.

Figure 2 .

 2 Figure 2. Comparison plot between the input work (perturbation values; blue right scale), the output force plate data (postural reaction; left red scale), and the transfer function model (black large line), for different Stance, Vision, and charge conditions: Tai-Chi Stance (With Vs. No load; four frames below) Vs. Normal Stance (No load; two upper frames), with Closed Eyes (left column) vs. Open Eyes (right column). Data for a Tai-Chi expert.

Acknowledgements. Thanks to the study's participants, to the MiSCIT students at University Grenoble-Alpes, to the Gipsa-lab technical team, and numerous students projects on this topic.


Kevin Doria

Lucor Didier. Anne Sergent

Lucor Didier

Temperature field inference using physics-informed neural networks in turbulent natural convection

come

Temperature field inference using physics-informed neural networks in turbulent natural convection K. Doria * , A. Sergent * , † and D. Lucor *

This work benefits from the French National Research Agency funding (THERMAL project).


est destinée au dépôt

In a recent publication in this journal, Laplaza, Contreras-Garcia, Fuster, Volatron and

Chaquin [1] (LCFVC) report an application of the Dynamical Orbital Forces (DOF) method, [2] which was used by the authors to challenge the presence of an inverted central C-C bond in [1.1.1]propellane (1 in Scheme 1), which was suggested long ago by Jackson and Allen, [3] then by Feller and Davidson, [4] and recently given theoretical support on the basis of ab initio valence bond calculations. [5] Scheme 1. The [1. 1.1]propellane and bicyclo[1.1.1]pentane molecules

The DOF of a molecular orbital (MO) is the derivative of its energy with respect to distance between two atoms of the molecule. This quantity is interpreted, on the basis of Koopmans' theorem, as the force exerted on the nuclei along this interatomic distance by removal of one electron from the MO, in the frozen MO approximation. The positive/negative sign of the DOF is used as a mean to characterize the bonding/antibonding nature of the MO with respect

[a] Dr. Benoît Braïda, Sorbonne Université, Laboratoire de Chimie Théorique, 75005, Paris, France. E-mail: benoit.braida@sorbonne-universite.fr [b] Prof. Dr. Sason Shaik, Institute of Chemistry, The Hebrew University of Jerusalem, 9190401 Jerusalem, Israel.

[c] Prof. Wei Wu, State Key Laboratory of Physical Chemistry of Solid Surfaces, Fujian Provincial Key Laboratory of Theoretical and Computational Chemistry, and Department of Chemistry, Xiamen University, Xiamen, Fujian 361005, China.

[d] Prof. Philippe C. Hiberty, Université Paris-Saclay, CNRS, Institut de Chimie Physique UMR8000, 91405, Orsay, France. E-mail: philippe.hiberty@u-psud.fr to the two atoms in question. As an extension of this model, the DOFs with respect to a given interatomic distance are summed over all the MOs (fi) of a molecule, and the result S(DOFi)

is considered as the central quantity that determines the nature and strength of a particular bond. A large positive S(DOFi) corresponds to a strong bond, [1,6,7] while a negative S(DOFi) signifies a repulsive interaction. This method was recently applied to a series of hydrocarbons and a linear correlation was found between the CC bonding energies and the CC S(DOFi). 7 Back to [1.1.1]propellane 1 in Scheme 1: this cage molecule was long supposed to possess a significant s bond between the two bridgehead carbons [3,4] owing to its remarkably weak enthalpy of formation (only 143 kcal mol -1 ) by extrusion of two hydrogen atoms from bicyclo[1.1.1]pentane 2. This value is significantly smaller than twice the standard bonding energy of a C-H bond. The presence of this central s bond in 1 was confirmed by ab initio calculations of the "breathing-orbital valence bond" (BOVB) type. [5] These calculations confirmed the order of magnitude of ca. 60 kcal mol -1 , that was estimated in previous studies for this bond based on the above thermodynamic considerations.

Two features of the bridgehead bond in 1 are noteworthy: (i) the bonding interaction takes place between two sp 3 -like hybrid atomic orbitals (HAOs) pointing away from the center of the molecule, hence its description as an "inverted bond" ; (ii) at variance with classical covalent bonds, the bonding strength of the bridgehead CC bond is mostly due to the stabilizing resonance interaction, C•-•C « [C + :C -+ C: -C + ], i.e. the resonance energy arising from mixing of the purely covalent component of the bond with its ionic components. At the same time, the covalent part alone C•-•C, involving singlet-coupling between the two HAOs, although being significantly stabilizing, cannot by itself overcome the large repulsions from the surrounding wing C-C bonds, and is thus overall only slightly repulsive. Such bonds that are dominated by the resonance energy of the constituent VB structures that differ by a shift of charge are qualified as "Charge-shift bonds" (CSBs) and constitute a bonding class on their own, quite different by many aspects from classical covalent bonds. [8] By contrast to the BOVB study, the bridgehead bond was diagnosed as being absent in the LCFVC paper, on account that the negative S(DOFi) of the MOs considered as being relevant to this bond (we will come back to this point later). Since the correlations of S(DOFi)

to bonding energies (BEs) were established for a series of CC bonds that are all of classical covalent bonding types, [7] whereas the bridgehead bond of [1.1.1]propellane was characterized as a typical CSB by our high level ab initio valence bond study, it is therefore important to check whether or not the DOF-BE relationship still holds for CSBs. Furthermore, since CSBs are generally found when the bond is surrounded by adjacent Pauli repulsion, it is also useful to test the adequacy of the DOF method for handling this latter kind of electronic interaction.

These two key questions are addressed here by means of our own DOF calculations in the three following sections, in the light of the DOF analysis of the prototypes ground state of difluorine anion F2 -(which is a pure CSB), and of the excited 3 Su triplet (which is repulsive and devoid of any bond) and ground 1 Sg states of difluorine F2 (which is a CSB in the ground state).

Is the DOF method reliable for Charge-shift bonds? The difluorine anion example

The difluorine anion, F2 -, is a typical example of a two-center three-electron (2c,3e) bond, usually represented as [F\F] -. In MO terms, such bonds display a doubly occupied bonding MO and a corresponding singly occupied antibonding one. In terms of Lewis structures,

[F\F] -is a resonating combination of two components, F: -•F and F• :F -. As each of these components is, by itself, dissociative, [9,10] the totality of the bonding energy arises from the electron fluctuation, i.e. the resonance between the charge-shift related structures F: -•F « F• :F. As such, the bond in F2 -, like all (2c,3e) bonds, [8,10,11] is a pure CSB. How does the DOF method account for the existence of this bond and its bonding energy?

To address this question, we performed DOF calculations on F2 -at the Restricted Open-Shell Hartree-Fock level (ROHF), using a quadruple-zeta Dunning basis set, aug-cc-pVQZ, with an interatomic distance optimized at the CCSD(T)/aug-cc-pVQZ level. The Gaussian 09 program was used throughout. [12] The results are displayed in Table 1.

Table 1. Energy derivatives of valence MOs of F2 -in its 2 Su anionic state, and of F2 in its 3 Su triplet and 1 Sg ground states, as calculated at the Hartree-Fock and density functional levels, in aug-cc-pVQZ basis set. The F-F interatomic distance in F2 is 1.412 Å, and 1.9202 Å in F2 (dropping the core MOs). It may first be noted that the DOFs correctly characterize the bonding/antibonding natures of the MOs (if one excepts the notable exception of the 1pu MO which is found non-bonding by the DOF), in agreement with the early aims [6] of this method: the 1sg and 2sg MOs are found to be bonding, all with positive DOFs, whereas 1su, 1pg and 2su have negative DOFs, hence antibonding. However, the sum of the DOFs over all occupied MOs, S DOFi, is significantly negative, (-0.291 a.u.), hence predicting the absence of any bond between the two atoms, and that F2 -would be dissociative. This is of course in absolute disagreement with the experimental bonding energy of F2 -, 30.2 kcal mol -1 . [13] It is noteworthy that the BOVB method, the same as the one that characterized the s bridgehead bond of [1.1.1]propellane and whose results are challenged in Ref. 1, yields a bonding energy of 30.4 kcal mol -1 for F2 -, [10] in perfect agreement with experiment.

To check the effect of changing the Hartree-Fock MOs to Kohn-Sham MOs, as done in Ref. 1, and argued to tentatively bring in electron correlation, we restarted the DOF calculations by using the density functional that was shown to yield the best (2c,3e) bonding energies, [14] the BHandHLYP functional, in the spin-unrestricted version to maximize accuracy. The results are displayed in Table 1, and are quite similar to the ROHF results, with a negative SDOFi of -0.251 a.u., hence still predicting a dissociative F2 -anion. Note that even if SDOFi is restricted to the valence MOs of s type only (those directly responsible for the 2c-3e bond), the value is found to be substantially negative. Thus, in view of these definite failures, one cannot avoid the conclusion that the SDOFi method, when used as means of predicting the existence or absence of bonds actually fails to do so when the bonds are CSB types. But this is not the only failure… Does the DOF method handle Pauli repulsion? The 3 S u triplet state of F2.

In the MO framework, the ground configuration of the 3 Su state of F2 is 1sg 2 1su 2 1pu 4 1pg 4 2sg 1 2su 1 . The elementary representations of this state are equivalent in the Hartree-Fock and valence bond frameworks, and in the latter the picture is that of two fluorine atoms displaying some Pauli repulsion between their respective s HAOs, each bearing a single electron with alpha spin. In accord, at the F…F distance of 1.412 Å (the equilibrium distance of singlet F2) the 3 Su state of F2 lies some 110 kcal mol -1 above the two separate F• atoms, as calculated at the ROHF level.

The results of our DOF calculations on the ROHF wave function of the 3 Su state at an interatomic distance of 1.412 Å are displayed in Table 1. Once again, the bonding/antibonding characters of the MOs are correctly assigned by the respective DOFs, positive for the bonding orbitals and negative for the antibonding ones. On the other hand, the strongly repulsive nature of this triplet state is definitely missed by the sum of the DOFs over all the occupied orbitals, as SDOFi is found to be slightly positive (0.069), which in this framework would characterize a slightly attractive interaction.

Furthermore, this SDOFi value can be decomposed into an interaction (SDOFi,s = 0.016) that is close to non-bonding in the s system, despite the expected repulsion between two electrons with parallel spins, and a slightly attractive interaction in both p systems (SDOFi,p = 0.053). These assignments are in contrast with the fact that each of these DOF components is a closed-shell system of p lone pairs which should undergo some fourelectron-two-orbital Pauli repulsion. So, somehow surprisingly, the errors of the SDOFi predictions are in opposite directions for the bonded F2 -vs. the repulsive triplet state of F2.

While in the first case SDOFi predicts a repulsion where there is a bond, in the second case it predicts a slight attraction where there is a strong repulsion.

The ground state of F2

In view of the above results, it is difficult to tell what would the DOF method predict for the F2 bond, which is a CSB. On the one hand, this molecule is found repulsive at the Hartree-Fock level (by ca. 31 kcal mol -1 at the experimental bonding distance), so in all rigor the DOF method should find it repulsive too, since it analyzes precisely the Hartree-Fock wave function. On the other hand, F2 is experimentally found to be bound by 38.3 kcal mol -1 . [START_REF]NIST data base[END_REF] Be it as it may, the DOF method finds F2 to possess a large positive SDOFi value (Table 1), and hence to be bound, despite its repulsive nature in the Hartree-Fock level. This latter result deserves comments.

First, the F2 bond is the site of a strong "lone pair bond weakening effect" (LPBWE) discovered long ago by Sanderson. [START_REF] Sanderson | Polar Covalence[END_REF] This effect originates primarily in the Pauli repulsion between the bonding electrons and the lone pairs that have the same symmetry as the bond, as shown in Scheme 2, and was estimated as ca. 77 kcal mol -1 in a computational study. [START_REF] Lauvergnat | [END_REF] Second, the charge-shift resonance energy (RECS) arising from the mixing of covalent and ionic structures is also very large, computed as 67 kcal mol -1 , making this molecule the prototype of pure CSB. [18] As was seen above, the neglect of these two effects by the DOF method is responsible for the DOF failure in triplet F2, as well as for the failure in F2 -. On the other hand, since the LPBWE and RECS are respectively repulsive and attractive, their joint absence in the DOF analysis of F2 somehow cancel out, so that the DOF result is correct, owing to cancellation of errors.

Scheme 2. The LPBWE in the ground state of F2

Interestingly, the S DOFi value for F2 is quite large, 0.661 a.u. and even 0.826 a.u. in Ref. 7, to be compared with 0.412 a.u. for ethane, [7] whereas the bond strengths (De values) are in the reverse order, 38.3 and 97.0 kcal mol -1 , respectively. [START_REF]NIST data base[END_REF] This apparent paradox can be explained by the fact that the DOF method is based on orbital-energy derivatives, while orbital energies are themselves dominated by the reduced resonance integral ß for bonding as well as antibonding MOs. Therefore, the DOF for a given MO is directly connected to the variation of ß with the interactomic distance. However, all other things being equal, this ß integral increases in absolute value from left to right of the Periodic Table [18] and so does its derivative, thus explaining the large S DOFi value found for F2, entirely based on a counting of orbital energy derivatives, but not quantifying the strength of the respective bond.

DOF Analysis of the inverted bond in [1.1.1]propellane

The ß-dependence of the DOF method has an interesting consequence. Apart from the abovenoted tendency in the Periodic Table, the ß integral is proportional to the overlap between the fragments that are involved in the bond. Now, because in such cases the hybrid atomic orbitals (HAOs) interact by their smaller lobes, this overlap is inevitably close to zero in the case of inverted bonds and is not expected to vary significantly with the carbon-carbon distance, leading to a quasi-zero ß integral derivative. It follows that the DOF method is, in principle, unable to detect an inverted bond, be it in bonds, as indicated by its shape, the absence of axial p atomic orbitals, and the low orbital energy (-0.533 hartrees, this work). Thus, the largely negative DOF of -0.109 a.u. for this MO is better interpreted as being due to the out-of-phase combination of bonding wing orbitals rather than to the minor contribution of the central bond. Now, keeping or removing this MO from the DOF analysis reverses conclusions of the analysis, which leads to a very small but positive DOF value, in accordance with the very small ß in this case. In view of such uncertainties, the attribution of a SDOFi of -0.029 a.u. to the central bridgehead bond alone appears as largely arbitrary. Incidentally, this difficulty to separate interactions from each other is also the problem that is encountered in the first section of the LCFVC paper, with the inverted C2H6 model. As the two methyl fragments approach each other in an inverted geometry, some H-H repulsions take place, blocking the approach at a long distance and preventing any CC bond formation.

However, this latter issue with the DOF analysis, which surfaces in systems that are more complex than simple diatomic molecules, is secondary. The main problems of DOF in analyzing the inverted bond in [1,1,1]propellane, are: i) the inability of the DOF method to account for the "charge-shift" component of bonding as illustrated with F2 -prototype case, ii) the inability of this method to probe situations of large Pauli repulsion, iii) the DOF's inability to probe bonding situation occurring when the overlap and thus ß resonance integral derivative is small. All these factors combine here in the case of the inverted bond of [1,1,1]propellane: large RECS has been previously quantified by means of accurate high level ab initio valence bond calculations, a large Pauli repulsion is present due to small-rings strain, and a quasi-zero ß resonance integral derivative is expected from the quasi-zero overlap between the hybrids involved in the inverted bond.

Conclusion

The DOF method which deals with derivatives of orbital energies may be useful and correlated to bond strengths in cases of small molecules displaying classical covalent bonds, where significant overlap between the respective fragment orbitals of the two atoms is the main factor of bonding, and furthermore if the bonding/antibonding character of MOs is well localized on the bond under study.

For more complex cases, when several factors come into play, the DOF model which is based on sums of MO energy derivatives inevitably misses some important features of the chemical bond, e.g., the Pauli repulsions or charge-shift resonance energies. [START_REF]type of interatomic forces which are present in molecules[END_REF] As a consequence, the DOF method may find bonding interactions where there is repulsion, and repulsive interactions where there is a significant bond. This was demonstrated in the case of difluorine in its triplet and anionic states. Finally, switching from Hartree-Fock MOs to Kohn-Sham orbitals as suggested in Ref. 1 does not improve the predictive accuracy of DOF, as shown in the F2 -example.

These severe failures, as well as the delocalized character of the MOs where wing bonds and the central bond are entangled, make the DOF method an inappropriate tool for characterizing the CC inverted bond in [1.1.1]propellane, which is revealed by direct ab initio valence bond calculations [5] as well as electronic stress tensor analysis, [20] density decomposition into paired and unpaired contributions, [21] and experimental Laplacian of the bond critical point. [22] A proof that [1. 1.1]propellane is devoid of an inverted bond, as claimed in Ref. 1, would require addressing of the foregoing four different studies, by methods that are more rigorous than DOF, which is demonstrated above to be inappropriate for such bonding problems. [START_REF]type of interatomic forces which are present in molecules[END_REF]

1 . 1 . 1 ]

 111 Comment on "The 'Inverted Bonds' Revisited. Analysis of 'in Silico' Models and of [1.1.1]Propellane Using Orbital Forces" Benoît Braïda, Sason Shaik, Wei Wu, Philippe C Hiberty To cite this version: Benoît Braïda, Sason Shaik, Wei Wu, Philippe C Hiberty. Comment on "The 'Inverted Bonds' Revisited. Analysis of 'in Silico' Models and of [1.1.1]Propellane Using Orbital Forces". Chemistry -A European Journal, 2020, 26, pp.6935 -6939. 10.1002/chem.201905666. hal-04400844 Comment on "The 'inverted Bonds' revisited. Analysis of 'in silico' models and of [Propellane using Orbital Forces" Benoît Braïda,* [a] Sason Shaik, [b] Wei Wu, [c] and Philippe C. Hiberty* [d] Keywords: [1.1.1]Propellane, Charge-Shift Bonds, Inverted Bonds, Orbital Forces, Valence Bond

[ 1 . 1 . 1 ]

 111 propellane or anywhere else.Another problem with the DOF analysis of[1.1.1]propellane is that it cannot cleanly separate the wing bonds from the central bridgehead bond in the canonical MOs representation, due to the fundamentally delocalized nature of these MOs. In Ref. 1, the MOs are divided into two categories, p (e' and e") and s (a'1 and a"2), and all the MOs of the latter category are arbitrarily associated to the central bond, whereas two of these MOs, 9a"2 and 13a'1, are in fact mixtures of wing and central bonds. In particular, the 9a"2 MO should rather be considered as a negative combination of bonding fragment orbitals localized on wing

  -. The S DOF values are calculated by summing up the individual DOFs of the orbitals multiplied by their respective occupancies. MOs 2 Su F2 -ROHF 2 Su F2 -UBHandHLYP

					3 Su F2 ROHF	1 Sg F2 HF
		DOF	DOF alpha	DOF beta	DOF	DOF
	1sg	0.021	0.016	0.019	0.241	0.262
	1su	-0.066	-0.050	-0.048	-0.159	-0.140
	1pu	-0.003	-0.003	0.000	0.108	0.108
	1pg	-0.058	-0.040	-0.040	-0.095	-0.103
	2sg	0.066	0.024	0.042	0.090	0.198
	2su	-0.090	-0.09		-0.238
	S DOF (s)	-0.048	-0.087	0.016	0.640
	S DOF (p)	-0.243	-0.164	0.053	0.021
	S DOF (total)	-0.291	-0.251	0.069	0.661

At the ROHF level, F2 -possesses the configuration 1sg 2 1su 2 1pu 4 1pg 4 2sg 2 2su 1


est destinée au dépôt

Organization of a spectroscopy tournament to engage students with structural identification of organic compounds

Frederic

  Hapiot, Stephane Menuel, François Delattre, Pierre-Edouard Danjou, Solen Josse, Véronique Bonnet To cite this version: Frederic Hapiot, Stephane Menuel, François Delattre, Pierre-Edouard Danjou, Solen Josse, et al.. Organization of a spectroscopy tournament to engage students with structural identification of organic compounds. LatinXchem 2022, Nov 2022, Online, France. . hal-04400864